internals-and-peps

[raven ridge]

or at least, that's all I've been able to find.

[weary garden]

^ why we need ISO standards

[signal tide]

im not surprised it's this easy to break it but in r you can apparently do '+' <- function(x,y) paste(x,y,sep="") which overloads the symbol itself

[spark magnet]

none of this would be changed by ISO standards.

[weary garden]

an ISO standard would fully describe the behaviour that godlygeek was trying to google so I disagree

[spark magnet]

ok, so he would get the answer from a standard, but it wouldn't change the answer.

[signal tide]

is that not a language construct?

[spark magnet]

(if the standard wasnt behind a paywall)

[raven ridge]

well, sure - but then I'd be searching for it in a standard, instead of searching for it on google, which would be at least as difficult, if not more difficult

[weary garden]

if Python had an ISO standard, for example, than a lot of the people in this chat would likely have a copy and be familiar with it enough to answer such questions

[raven ridge]

people are already familiar with the language, despite not having an ISO standard.

[weary garden]

pfft

[spark magnet]

we can explain how Python works.

[raven ridge]

just like people were familiar with C in 1988.

[weary garden]

actually when I went to uni back in the day my K&R was the ANSI version

[peak spoke]

Are the standards readily and freely available?
It's good to keep in mind that python's development is fairly open and has a relatively low barrier of entry for new contributors interested in doing that, which wouldn't work as nicely if it was standardized in that way

[raven ridge]

generally, no.

like, the C++ and C standards are paywalled, for instance.

[limpid marten]

their drafts are good enough.

[raven ridge]

in any event: this is mostly a problem of not knowing if I searched for the right things. Having a standard wouldn't help that; I'd still need to search within the thousand page document, and I still wouldn't know if i had used the right terms when I failed to find something relevant.

[raven ridge]

in other words, even where there are standards, most people don't bother with them 😄

[weary garden]

what is important is that implementors bother with them

[raven ridge]

well, that's not what you said a moment ago, about whether it would have helped me as a user learn whether the language had or didn't have double dispatch operator overloading.

[limpid marten]

I think having standards helps create competing implementations.

[weary garden]

agree

[spark magnet]

@limpid marten Ruby has an ISO Standard: how many implementations? Python doesn't: how many implementations?

[raven ridge]

I think it's almost the opposite of that - standards help unify implementations once they've already been created and start to diverge.

POSIX, for example. C, for another.

[spark magnet]

meanwhile, we are having this conversation because someone has chosen to make a new Python implementation, without a standard.

perhaps a standard does get more implementations, but Python doesn't seem to be hurting in this regard.

[signal tide]

is there a reason behind this?

and why it's 211 usd

[spark magnet]

ISO thinks it increases the value of the standards.

[signal tide]

does anyone actually buy it?

[weary garden]

heh

my life would be easier if it did have a standard tho, nedbat.

[spark magnet]

yeah, it might.

it;s just not a panacea, and it's a lot of effort and turmoil.

[weary garden]

I just wish there were more hours in the day and days in the week; I have a day job. :/

but at least in a month or so's time I will be able to do Python code reviews at work.

which doesn't happen that often as most of our stuff is C++

I guess power is an operator not a function in mathematics

[grave jolt]

is the SQL standard paywalled?..

[spark magnet]

judging from a google search, yes.

[raven ridge]

yep. https://webstore.ansi.org/Standards/ISO/ISOIEC90752016?source=blog

[spark magnet]

ECMA standards are open

https://www.ecma-international.org/publications-and-standards/standards/ecma-262/

[grave jolt]

$225

jesus

[feral cedar]

bruh

[grave jolt]

Minimum monthly wage in Russia is $171

why so expensive?

and... why is it even paywalled?!

[peak spoke]

I imagine it's because creating the technical documentation takes some manpower and I doubt they get many sales

[grave jolt]

ah, I see, all ISO standards have to be paid for

right

[quaint gulch]

Hello everyone. I created a python based trading bot and I’d like to share it. https://github.com/mhop2101/exoity it is completely free and can be used by anyone. I’d love to hear feedback.

[grave jolt]

it's in an IPython notebook? 👀

[peak spoke]

Don't think it makes much sense for programming standards as being open and easy to access is important, but the organizations encompass a lot more

[swift imp]

There was discussion on the mailing list about making a standard and the general concensus was that it's more work than it's worth

[grave jolt]

It would be more on-topic in #python-discussion, this channel is for discussing the Python language itself

[swift imp]

Literally happened a few weeks ago

[grave jolt]

PEPs are proposals to change the language. But they have to be sorta formal (e.g. PEP 634), so people refer to them as specification

[weary garden]

just had to change my JSON parser to support escapting forward slashes as divid floor (//) is a comment

[swift imp]

https://mail.python.org/archives/list/[e-mail]/thread/JZYW4JOTANYIOLYDQ6YHRUP2TWO52OAE/

[grave jolt]

You could do \/\/, right? 🙂

And wait... Does it treat //bar" in "foo//bar" as a comment? that's certainly not right. or is that what you meant?

[swift imp]

Json is by definition not supposed to support comments

[grave jolt]

well, it's relaxed json

[grave jolt]

What's the correct name -- "type hints" or "type annotations"?

!pep 484

[fallen slate]

**PEP 484 - Type Hints**

Link

Status

Provisional

Python-Version

3.5

Created

29-Sep-2014

Type

Standards Track

[grave jolt]

Well, I guess the PEP says 'type hints'

But is 'type annotations' a correct term as well? After all, they're stored in __annotations__, not __hints__

[raven ridge]

https://docs.python.org/3/glossary.html?highlight=glossary#term-annotation

[grave jolt]

ah, so it's like parameter vs argument?

wait, I'm confused

[raven ridge]

The piece of metadata attached to an object is an annotation. The conventional use for annotations is to provide type hints

[wide shuttle]

Annotation is a more general term and you can annotate objects with a type hint

[grave jolt]

aaaaaaaah

gotcha

so discord.py's abuse of annotations for converters is "annotations", but not "type hints"

I'm writing a tutorial on type hints

how do I say it so that the brain of the reader is not turned into mashed potatos?

I guess I'll just use them interchangeably, maybe clarify later

[raven ridge]

Just avoid the word annotation entirely, except perhaps to mention __annotations__

[grave jolt]

that seems reasonable

[raven ridge]

Everyone will understand what you mean if you say "a variable's type hint" or "add a type hint"

[grave jolt]

and I'll use the verbs typehint and annotate as synonyms

[raven ridge]

Seems reasonable.

[grave jolt]

alright, thanks 👍

[raven ridge]

Adding a type hint is a subset of all possible annotating

[grave jolt]

I just never got too pedantic about it, so I used the terms interchangeably in informal conversation

[raven ridge]

The terms are a bit muddled because originally the intention was for annotations to be usable for many different things, and then it was discovered that if a type checker doesn't know which annotations are meant as type hints and which aren't it can't complain about invalid type hints, and so we've arrived at our current situation where even though it's technically possible to use them for other things it's discouraged

And, coming round trip, typing.Annotated now exists, as a general purpose annotation that type checkers know to ignore

[grave jolt]

Hopefully discord.py will make use of Annotated

[wide shuttle]

The PEP on function annotations tries really hard to press the point that annotations are arbitrary expressions and have no special meaning within Python itself.

It's probably an interesting read on this topic: https://www.python.org/dev/peps/pep-3107/

[grave jolt]

Yeah, you can write your whole program as a one-liner in an annotation.

[wide shuttle]

And I'd just focus on type hints for the tutorial

[grave jolt]

that was my goal as well

[raven ridge]

That was superseded by https://www.python.org/dev/peps/pep-0484/#what-about-existing-uses-of-annotations though.

[grave jolt]

FastAPI made a clever move and made all the annotations correct type hints to the type checker:

@app.post("/foo/bar/{id}")
async def post_foo_bars(
    id:  int,  # gotten from the URL
    bib: int,  # gotten from the query string
    bob: str,  # same
    beb: MyPydanticModel, # gotten from the JSON body under they key `beb`
    x_quack: str = Header("X-Quack"), # gotten from the header
    # Header is thought to return Any, so it's assignable as a default to `str`:
):
    ...

[raven ridge]

I'm not sure if they've ever been explicitly deprecated, but non-typing uses of annotations are definitely discouraged. And https://www.python.org/dev/peps/pep-0593/#motivation specifically calls non-typing uses of annotations "not currently realistic given the prevalence of type annotations".

[wide shuttle]

I think it's more or less where we arrived now

[grave jolt]

In general, do you think such tutorial will be useful? I'm planning to mostly cover the "102" or even "103" parts -- like literals, generics, and protocols.

104 would be writing your own type checker, but I haven't ascended that high yet 🙂

[wide shuttle]

I think the 102 and 103 parts are useful, if done correctly. I think you can do that, but what I see a lot is that people in some of the more advanced (or "beyond beginner") discussions of type hints quickly posit dazzling blocks of code containing type hints that are fairly difficult to process for someone with less experience in the topic.

It would be great to see a solid tutorial to "ease" people into that

[grave jolt]

let's move to #799040290459615282

[unkempt rock]

where can i learn some real advanced shit like decorating the decorators

and decorators with mutex

[unkempt rock]

Regarding Leigh's python implementation I think it makes sense to support pure python only if its used in the sense of classical scripting on top of an engine. In that case the only "C API" needed is that engine's own API.

Lua is the perfect language for that kind of usage IMO. And implementing 5+ languages on your own sounds overly ambitious/ not realistic. Hats off if you pull it off though.

[nocturne delta]

what are the conditions for python to print none in terminal

like all the conditions

[charred wagon]

Are you asking under which circumstances it is possible for the terminal to output "None"?

[visual shadow]

There is just one. That is, when it's asked to print None. So understanding when a value is None is perhaps a better approach. There, the only advice I have is this : functions in python that don't return anything explicitly will return a None. And by convention, all builtin mutation methods and functions that aren't meant for creating a new value will return a None.

[nocturne delta]

oh ok thx!

[unkempt rock]

is concurrent.futures sone future module ?

whats the deal with the future?

[radiant scroll]

it's for unfinished tasks which will get finished in the future

it deals with async or threads

that's why it contains the concurrent word, it's for concurrent run, where you are running 1 function in a thread, that is not yet finished, and at the same time, in a different thread, you have the main process

[weary garden]

@grave jolt yeah \/\/ fixes it; my JSON parse didn't include an escape for forward slash tho so I had to fix it.

[atomic egret]

What is the difference between dataclasses and named tuples?

[peak spoke]

named tuples just wrap tuples and add properties that delegate the names to tuple item access; dataclasses are a bit heavier but offer a lot more flexibility and methods to reduce boilerplate

[grave jolt]

namedtuple also allow methods and custom properties, but they aren't as customizable (they're always immutable, for example)

[atomic egret]

so dataclass just makes __init__ with the required properties, and namedtuple actually subclasses tuple?

[peak spoke]

Yes, it generates a class that inherits from tuple directly

[atomic egret]

I see, thanks

[weary garden]

I've changed my mind; I will have a class semantic concept however its sub-concepts can vary depending on the language

[atomic egret]

How would I decorate every method in my class as staticmethod?

doing it manually seems very repetetive, is there some way to automate it or some pre-made class I can use to extend my class?

[radiant fulcrum]

The real question is why do you have a bunch of static methods in a class when they have no reason to be in there

[atomic egret]

I'm using the class as a holder for some methods related to 1 thing, that the class represents. I didn't want to use a whole file only for 5 methods, but it makes sense to group them separately (into class-like structure)

the class in my case is just a way to hold my functions, it's not meant to be instantiated

I would usually prefer making a file to hold them too, but in this case, that would just be a huge overkill, I only have 5 1-line methods, which take 1-2 variables and put them into a string, all of these methods are related to 1 concept and so I needed to group them in some way, class seems like a perfect structure to hold them, it's similar to just some regular class to hold some constants related to 1 thing, except in my case, I need them to be functions since I need a variable in the string they return.

[visual shadow]

Just use a module 😅

[atomic egret]

but that wouldn't make sense, the file i have is basically here for constant values, i.e.: ```py
class Color:
WHITE = 255,255,255
...

my class is a part of this file except it uses some functions instead of just constants, using module here would be weird

[modern night]

even if you do use a class as a namespace, you could use a metaclass to decorate all non-dunder callables in the class namespace but for the sake of 5 functions, doing them all individually is much easier to understand, still shorter and you wont get any issues with your linter expecting the functions to take self as a first parameter (this last issue may depend on your linter but i have this issue)

so in short, module is probably best choice, then manually decorating each mathod, then automatic decorating (because it's quite complex for a simple job)

[atomic egret]

but if I were to use a module, I'd have to split my whole config file, which is about 70 lines long into about 15 separate files, each only holding about 3-7 constants, I just don't find this worth the mess it makes in the filestructure

[visual shadow]

Hmm

[atomic egret]

I do know that I could use a metaclass, and it wouldn't be hard to do so, but the problems with linters are exactly what I wanted to avoid, that's why I asked if there wasn't some kind of module that was able to handle annotations for type checkers etc. already in some python library

[visual shadow]

I'm not actually sure if there's a recommendation or best practice for this

I'd like to hear someone else pitch in

[atomic egret]

I have seen that @fallen slate actually uses yaml config file and a metaclass that bunch of namespace classes inherit from and obtain the settings from that yaml file. Since similarly to my case, using dozens of files each only having few constant values didn't make much sense.

I'm in a similar situation, where splitting to modules simply doesn't make sense, but I can't use a yaml file like that, because as i said, some of those constant values are actually functions that only insert the passed arguments into a string in some form, and yaml can't store functions like this.

[feral cedar]

an enum?

[atomic egret]

what's an enum?

[feral cedar]

it's like a class of constants

[atomic egret]

that is basically what I need, except some of those are functions, how does enum handle those?

[feral cedar]

an enum is a class, so it can have methods

[atomic egret]

but i don't suppose it converts them to staticmethods, or does it?

[feral cedar]

oh,I see what you mean now. enums are for something like that Color class

for your functions, why do they need to be in a class

[atomic egret]

similarely to color, they're related to 1 concept

[feral cedar]

ok, I see. I'll let someone else answer since I'm not sure of the best practice here

[peak spoke]

I'd have to think about it a bit more to be sure on the best way, as mixing the constant class and functions that operate with its values is a bit weird imo but I don't think there's really a way to apply it to all the methods without doing it manually while having linters understand what you're doing. If you wouldn't be mind it being dynamic with no linter support then I suppose a class decorator would be the nicest

[atomic egret]

I agree that it is pretty unusual, but I do need to do it, because it basically belongs to constants, it only inserts given variable into a string constant to proper position, I've got quite a lot of these so manually writing out that they're staticmethods is a bit annoying, I will probably have to do it if there isn't any other, better way, since I do want to preserve proper linting support.

I was hoping that there was some solution to this problem, without breaking linting and too much repetition, but I suppose due to it's unusual nature, there won't be much else to do about it.

thanks for helping, and if something else does come up, please mention me

[peak spoke]

Linters will only understand thing that are static, which would go against no doing it explicitly for every method no matter the approach you take there. But there may be an another way to go around it

[unkempt rock]

Sorry, wrong channel for help

[sacred tinsel]

The functions don't necessarily need to live in a module of their own, they can just be in your constants module next to the other stuff 🤷‍♂️ that's probably what I'd do

[flat gazelle]

there is such a thing as too much namespacing. What is stopping you from having a single module that has everything?

[boreal umbra]

put everything in its own module

[atomic egret]

without the namespace I'd have to include the common name in a fucntion name, something like some_concept_specific_function, which is probably even worse than too many staticmethod decorators

[atomic egret]

#internals-and-peps message

[sacred tinsel]

I'm not sure if this is relevant to your situation but you could also have a global instance, like this:

config = {"animal": "cat"}

class _Constants:

    def make_thing(self):
        return f"Animal is: {config['animal']}"

constants = _Constants()

from ... import constants

constants.make_thing()

[boreal umbra]

couldn't this be an enum?

[atomic egret]

This was also already suggested (#internals-and-peps message), enums don't convert methods to static ones

[flat gazelle]

a bunch of singletons may be easier here, yeah

[atomic egret]

this does sound interesting

[sacred tinsel]

From outside the module it doesn't really matter whether the namespace you're looking into is a class or an instance thereof

it should be the same, basically

[atomic egret]

yeah, that would be a pretty good way around it

I think that's my best choice, thanks!

[unkempt rock]

if you import some python libraries like cv2, os in a module and then you import that module into a separate module (a main.py) and then run that separate module, will you be able to use the python libraries from the module you imported?

[sacred tinsel]

yes, if you access them via the imported module's namespace
a.py:

import pathlib

b.py

import a
a.pathlib

[boreal umbra]

Yes, importing a module puts that module into the namespace of the first module. and then importing the first module gives you everything in that namespace

however there will just be one instance of os that gets cached with the other modules for that interpreter session (or whatever module you import)

I saw this in a code sample for patma

def factorial(n):
    match n:
        case 0 | 1: return 1
        case _: return n * factorial(n - 1)

I didn't do line breaks because of laziness

is this wrong, or does it have a special order of operations? because wouldn't 0 | 1 to a bitwise operation?

[unkempt rock]

wait I thought python never had switch statements

[radiant fulcrum]

3.10 just added pattern matching

[unkempt rock]

yeah I won't be using that version for a long time

deep learning still uses old libraries

[radiant fulcrum]

most production systems etc... wont use pattern matching anyway

[grave jolt]

It's called "structural pattern matching" -- you capture the structure of something, and the syntax for the pattern is not a python expression

so 0 | 1 is not a python expression, it's a special form used in the case clause

[boreal umbra]

so if a pipe delimits one of the structures for which that case applies, how does it know when a pipe is a delimiter vs an infix operator?

can you do case (0 | 1) to force it to evaluate normally?

[grave jolt]

It can't be an infix operator because it's not a Python expression.

[boreal umbra]

so can patterns only be stand-alone tokens?

[undone hare]

I’d assume that the ast walker is having a special case for structural expressions

[grave jolt]

if you want to compare for 0 (BINARY OR) 1, you do

case x if x == 0 | 1:

[undone hare]

What is the ast tree for that one?

[boreal umbra]

I don't actually have 3.10 on my computer. Guess I might as well get it

[grave jolt]

Well... it's not a "special case". It's just not a Python expression. Just like the import path is not a Python expression.

from os.path import join

[undone hare]

Hmm interesting

You can probably see ot in the parser syntax then

[boreal umbra]

there's no 3.10 branch on cpython?

[true ridge]

See PEP 642 about this.

https://www.python.org/dev/peps/pep-0642/#representing-patterns-explicitly-in-the-abstract-syntax-tree

[undone hare]

Maybe 3.10a1?

Ah yes, I see them in the grammar specs https://docs.python.org/3.10/reference/grammar.html

[true ridge]

dev branch is always the master (main from now on)

when 3.10 goes into beta, 3.11 will be the new main branch

though there are tags for each release, and actually branches for every supported release (3.6+)

[undone hare]

Interesting, thanks!

[true ridge]

I don't like 642 in general though the part regarding properly representing ASTs in their own base was I believe a really goos fact.

[undone hare]

Hmm, why is that so?

[true ridge]

https://discuss.python.org/t/gauging-sentiment-on-pattern-matching/5770/21

"I see the match statement as a DSL contrived to look like Python, and to be used inside of Python, but with very different semantics."

This is probably the best and shortest answer to your question

[undone hare]

Interesting

[grave jolt]

how did patma get into Python with such poll results?

[radiant fulcrum]

no idea

honestly i was a i don't want pattern matching side

[weary garden]

@grave jolt what is that a screenshot of?

[radiant fulcrum]

the ast tree

of a match block

[weary garden]

I know what an AST is 🙂 is this a python feature CPython feature or what?

[radiant fulcrum]

well its a language specification so

[weary garden]

I know what an AST is! I am asking about whatever tool this is to display it

[grave jolt]

I guess It's a CPython feature

I'm actually not sure

Well, I mean, it's shipped with CPython

[radiant fulcrum]

I think every runtime exposes ast

[weary garden]

does PyPy have it too?

[radiant fulcrum]

yes

[weary garden]

in the same format?

[radiant fulcrum]

because it's a parsing fundamental

PyPy's whole thing is compatibility with pure python

generally, if Cpython has it, pypy has it

[weary garden]

so PyPy's interactive mode supports all the commands identically in CPython's interactive mode?

[radiant fulcrum]

pretty much

[true ridge]

yes

at least PyPy follows CPython, so it is quite similiar

[radiant fulcrum]

again you can just see what is and isnt supported identically with https://www.pypy.org/compat.html

[weary garden]

similar != same

[true ridge]

who knows ¯_(ツ)_/¯

[grave jolt]

||guido's laser eyes||

[true ridge]

quite similiar ~= same (I maintain both of them)

if you find an inconsistency, just feel free to report

[grave jolt]

Even if ast is not part of the "specification", I don't see a reason to expose a different format (since the AST is going to have the same shape, more or less)

[weary garden]

you maintain both CPython and PyPy?

[true ridge]

though there is no rule for other implementations to follow the same ast, or even ship an AST module

[true ridge]

AST modules

[weary garden]

ah

I might be pestering you in the future then given I am creating an implementation

[grave jolt]

Is there some indicator as to whether a package/feature is mandatory or not? Because in this case it's not clear

[true ridge]

RustPython is also uses a similiar AST though since they don't have an exact target python version, kinda hard to depend on it

[radiant fulcrum]

https://docs.python.org/3/library/ guessing this

well we can find out what version it targets lets go check le parser

[weary garden]

not sure how I will approach it for "neoPython"

[true ridge]

It is kinda the language reference + stdlib though it is complicated

[grave jolt]

yeah, it is complicated

[true ridge]

There are going to be (I presume) some work later this year regarding a "Developing a "Minimal Viable Python""

[weary garden]

I will effectively be doing that over the next 2 or 3 months

[true ridge]

Doing what?

[weary garden]

creating a Python implementation from scratch initially supporting pure Python language only

[true ridge]

that's quite interesting subject

good luck!

[weary garden]

@true ridge https://neos.dev

[radiant fulcrum]

The parser is targeting i think 3.7

i dont think atleast 3.8 is supported

[true ridge]

I kinda remember there was :=

though i might be wrong

[radiant fulcrum]

I dont think it does yet

it might be a PR

[true ridge]

https://github.com/RustPython/RustPython/blob/c63a429bdc192cbc51d0af87680c1e5b6a390ca8/parser/src/python.lalrpop#L675

weird

[radiant fulcrum]

oh?

hmm

[weary garden]

that would have been a plan B for me if I just wanted to learn 2 languages: Rust and Python

[radiant fulcrum]

let us try this

[true ridge]

I wish I knew rust :/

[weary garden]

but I have decided to go the universal compiler route

[true ridge]

that's a cool idea

like Graal

[radiant fulcrum]

Cool idea but idk how well it's gonna go down in the real world

[true ridge]

yeah, will be hard to pull it off. And making it efficient at the same time

[radiant fulcrum]

a language and it's implementation generally lives and dies by it's eco system

[weary garden]

well it is only part of my offering: my full offering is https://neogfx.org

neos is part of neoGFX

[stiff sky]

guys, i wrote class which makes progressline, please tell me your opinion about the code

class Progressbar():
    __slots__ = ['to_next_lvl', 'progress_now']

    def __init__(self, progress_now, to_next_lvl) -> int:
        self.progress_now = progress_now
        self.to_next_lvl = to_next_lvl

    def write_line(self, line, to_fill) -> str:
        progressbar = ""
        procent_bar = round(self.progress_now/self.to_next_lvl * 100)
        for i in range(round(procent_bar / 5)):
            progressbar += to_fill
        for i in range(20 - round(procent_bar / 5)):
            progressbar += line

        return progressbar


test = Progressbar(900, 1000).write_line('-', '+')
print(test)

[radiant fulcrum]

never mind it does support 3.8

calling the wrong parser function

[true ridge]

They also recently migrated to using ASDL which I guess is the best way to identify differences between ASTs of different implementations

[radiant fulcrum]

I should probably get more involved in the Rust Python stuff

the only think i do dislike with RustPython though is it seems to try imitate Cpython rather than necessarily using some of Rust's strong points and styles

[boreal umbra]

def fac(n):
    match n:
        case 0 | 1:
            return 1
        case _: return n * fan(n - 1)

match n: gives me a syntax error with Python 3.10.0a1. I guess I picked a 3.10 version that doesn't have patma?

[severe lichen]

Sooo, what's your opinion on pattern matching?
I haven't had the chance to play with it yet.

[radiant fulcrum]

personally i dislike it

[flat gazelle]

@boreal umbra IIRC its in a6

[radiant fulcrum]

to me atleast it just feels like a really good way to make code less readable easier

[flat gazelle]

I fully understand why it is pretty much needed, but I really wish there was a nicer solution

[radiant fulcrum]

I understand it's advantages but i think it's implementation is poor

[grave jolt]

I really don't like the whole captured variable vs capturing a constant by dot

[flat gazelle]

I feel like =var and ==var would be best

verbose, but actually clear

I feel like patma is about as good as it can really could be

[raven ridge]

One of the arguments against it was "if this is as good as it can be, maybe we just shouldn't have it"

[flat gazelle]

yeah, but it does make making functions that actually accept a wide variety of types much easier

[raven ridge]

I'm slightly negative on it. I wish they'd spent another year thinking about it. Rushing it into 3.10 all but guarantees that even if someone comes up with a way to do it better, they won't.

[flat gazelle]

ye, maybe given enough time it could be improved

[radiant fulcrum]

the fact it got into 3.10 does suprise me

[grave jolt]

Don't worry. We'll have Python 4 which will contain "legacy pattern matching" and "new pattern matching" 🙂

[raven ridge]

The bettermatch statement

It looks like Python, but doesn't behave like Python. It behaves more like regex, and there's fiddly cases that will just need to be memorized.

(like the different behavior that depends on whether the name has a dot)

[raven ridge]

!pep 636

[fallen slate]

**PEP 636 - Structural Pattern Matching: Tutorial**

Link

Status

Final

Python-Version

3.10

Created

12-Sep-2020

Type

Informational

[raven ridge]

You should really start here. Sounds like you haven't read this yet

[radiant fulcrum]

i wish you luck

[raven ridge]

It's not a typical PEP, it's structured as a tutorial instead, and it walks you through increasingly complicated patterns

[radiant fulcrum]

now this is pattern matching

[grave jolt]

Year 2074. "After much deliberation, we chose to deprecate PEP 642 in favor of putting https://github.com/decorator-factory/context_manager_patma into the standard library"

[limpid marten]

Wasn't the motivation of this current pattern matching to mimic that of other languages with pattern matching? Do they not have a similar problem?

[grave jolt]

what similar problem?

[limpid marten]

Named constants?

[flat gazelle]

haskell just doesn't allow matching on constants

IDK what rust does

[radiant fulcrum]

Rust is pretty good with stuff

you can pattern match against anything in reality

unwrapping enums, comparing constants etc...

[silk pawn]

so when we assign a value to _, like _ = 5, is _ automatically garbage collected? iirc there's something special about _ but i cant remember what

[grave jolt]

nope

!e
just a convention

_ = 42
print(_)
_____ = 1
print(_____)

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

001 | 42
002 | 1

[radiant fulcrum]

other than in the repl where it's the last returned value

[true ridge]

this is cool! Any reason not to use AST for parsing the case(...) contents tho?

[rose stream]

My vocab sucks, for typing especially.
Is there a way of changing partial_proxies type?

proxy =  partial_proxy | {'port': AVAILABLE_PORTS[0]} 

as it's causing this error:

(variable) partial_proxy: PartialProxyDict
Unsupported left operand type for | ("PartialProxyDict") mypy(error)

[boreal umbra]

@rose stream often people won't want to read screenshots of text. That one is too small for me to read on my screen.

The class of an object is writable for some types. So you might be able to do partial_proxy.__class__ = SomeOtherClass

[rose stream]

The class is a TypedDict* subclass, mypy is unfortunately still giving me issues

[grave jolt]

That is a good question... Well, I didn't think about it really. Maybe it's too many errors to filter out.

[unkempt rock]

Personally I think the pattern matching syntax will grow on most people. The kind of thing that makes you say ewww the first time you see it but after a few times using it you start to love it

Remember the controversy when f-strings came out?

I also think they will eventually introduce the leading . or some other syntax for loading a value in a case like in the original pep.

[solemn mountain]

Can anyone tell me what this is supposed to do

[oblique crystal]

ehm.... @solemn mountain

1. I think it's not the proper channel to ask
2. what language is that?

[wispy mirage]

looks like Haskell developers are infiltrating Cpython development.

[orchid karma]

@solemn mountain Please try out off-topic channels not python related questions. This channel in particular is discuss the uses cases, implementation, and future of Python itself.

[weary garden]

I now have the semantic concept concept.unimplemented 😄

correction: concept.unrealized 😄

[silk pawn]

🪄

[weary garden]

it allows me to get on with defining the grammar without getting bogged down at this stage with specific concepts

😄

actually it is a lowercase string; not sure why it is AWAIT in the grammar

better:

[radiant fulcrum]

man you are gonna love the asyncio massacre

[weary garden]

hi

why does '__peg_parser__' appear in the grammar?

[gleaming rover]

it would be nice

if pattern matching was an expression

😐

[swift imp]

It would be nice if descriptors played with docstrings better

[spark magnet]

it's not clear, but you might want to try typing __peg_parser__ into a Python interpreter.

[weary garden]

that doesn't really tell me much

[spark magnet]

Me either 🙂

it doesn't look like something you need to keep

[raven ridge]

It's an Easter egg, more than anything else, I think.

Technically you can use it to tell whether the old or new parser is in use, and I expect it'll go away when the old parser does.

[weary garden]

print(peg_parser) ???

[spark magnet]

what does that do?

[weary garden]

I was hoping you would try it

[spark magnet]

don't you have a Python to try it?

[weary garden]

I haven't started vmware tonight

[spark magnet]

it won't work, __peg_parser__ is a keyword that raises a syntaxerror.

why vmware?

[weary garden]

according to the grammar it is an atom

so print might work

[spark magnet]

From the actual grammar file:

atom[expr_ty]:
    | NAME
    | 'True' { _Py_Constant(Py_True, NULL, EXTRA) }
    | 'False' { _Py_Constant(Py_False, NULL, EXTRA) }
    | 'None' { _Py_Constant(Py_None, NULL, EXTRA) }
    | '__peg_parser__' { RAISE_SYNTAX_ERROR("You found it!") }
    | &STRING strings
    | NUMBER
    | &'(' (tuple | group | genexp)
    | &'[' (list | listcomp)
    | &'{' (dict | set | dictcomp | setcomp)
    | '...' { _Py_Constant(Py_Ellipsis, NULL, EXTRA) }

[weary garden]

I see

[raven ridge]

I believe it's only recognized as a special case at parse time, and always fails the parse if it's found

[sacred yew]

wait why do you need a vm to run python

[unkempt rock]

You don't need a virtual machine to execute python code.

[flat gazelle]

Is await as expression vs await as statement part of the spec? Or is that specific to yield?

[raven ridge]

yield used to be a statement, and now it's an expression. await has always been an expression.

[true ridge]

Since 3.9 shipped with 2 parsers, it was for testing out which one currently you are using

[grave jolt]

the wemake-python-services linter still prohibits f-strings

[grave jolt]

Would it make sense to create a sort of type-safe routing (in web apps)? As in, routing that can play nicely with type checkers.
(see <#esoteric-python message>)
TL;DR:

r = Const("foo", Str(Int(Const("bar", Int(Empty())))))
match = r.extract("/foo/some-string/1/bar/42")

-> match is inferred as Tuple[str, Tuple[int, Tuple[int, None]]] | None
I suppose that more complex scenarios like optional parameters could be more valuable.

The advantages being:

1. The editor will point to an error before you run the code through a manual/automated test
2. Autocomplete/intellisense can suggest what you can even do with the URL parsing results.

The disadvantages being:

1. Less readable than /foo/:str/:int/bar/:int
2. Quite complicated to name parts of the URL
3. The parts of the code that touch the URL parsing will probably be touched in many test cases anyway

[flat gazelle]

I feel like

Const("foo") / Str('special_name') / Int() / Const('bar') / Int() / End()
``` would look cleaner, but I have no idea if type checkers can use that correctly

[grave jolt]

I guess it is possible, since they should interpret / as a normal method call

I kinda wish there was some "universal typehint plugin" language that would allow writing your own domain-specific type rules, that would be declarative standard (i.e. work with not just mypy)

[weary garden]

I am using a VM as I have chose to not install CPython to my host OS (Windows); I prefer to test such things in Linux. @sacred yew

[flat gazelle]

at the same time, I don't think I ever made an URL too complex to require automated checks

generally, just looking at it did well for me

[grave jolt]

Yeah, that's true.

Yes, it's easy to see just by looking. But it's a bit sad that the URL is right there, like /images/{name:str}/png, and my editor has 0 chance of knowing that request.path_params has a single key name, and that request.path_params["name"] has type str.

[grave jolt]

Or take something more complicated, like attrs or sqlalchemy

currently, you have to write a plugin that interacts with the messy stateful type checking software

I'd imagine something like ```py

url.pyt

def interpret_path(path: StringLiteralType) -> DataclassType:
value: str = path.value
... # parse the path somehow
return DataclassType(
fields = ...,
methods = ...,
...
)

url.py

P = Type("P", bound=str)
def make_route(path: P) -> Apply["interpret_path", [P]]:
...

the language in .pyt doesn't have to be python, might be something more shallow/restricted

hm, that would be an interesting project

maybe it's even possible to somehow implement as a mypy plugin

[spring perch]

Correct me if this is the wrong place to post this, but I wrote a piece on how to use/test upcomming python pattern matching already
https://py.watch/get-started-with-pattern-matching-in-python-today-ef4d19c97b7a?sk=ee1b6f2842aaeb3dbd83ed8debe5f95c

[finite hare]

Someone asked me to help them in a beginner python class.
I've written a decent amount of python and I've been doing software development for a good 8 years by now.

I want to get my python knowledge up to par with my skillset in some other languages. Do people have a book on advanced python that they think could serve as a good read?

A lot of resources online seem to be targeted towards beginners. Is there anything like an "advanced python for programmers" book as there are for some other languages?

[strange heath]

Fluent Python

my friends recommended that

[grave jolt]

Fluent Python seems a good fit, judging from the table of contents
https://www.oreilly.com/library/view/fluent-python/9781491946237/

I should probably read it

Also check out the latest pattern matching and exception groups PEPs if you want to get the latest features

Gradual typing is also a big feature, I don't know any thorough reference, but

1. there's a tutorial on Real Python https://realpython.com/python-type-checking/
2. I'm planning to write an extended structured tutorial (length of a very short book, maybe?)

[spark magnet]

i don't imagine a beginner class will need to learn about pattern matching 🙂

[grave jolt]

Yeah, good point. I was just recommending general stuff to "get knowledge up to par"

Reading on advanced stuff might hinder the teaching, because you'll be tempted to share that with the students

and they will have hbox overfull

[finite hare]

it's unlikely I'll need to learn much more than what I already know, but I like to be prepared and have a good understanding of the underlying system

[spark magnet]

fancy TeX reference 🙂

[spark magnet]

I totally understand. it's easy to write Python and not get the deep structure.

[grave jolt]

I can't live without TeX references

[finite hare]

it's a good point, but I occasionally meet some really curious students who ask me questions that seem simple, but have super complex answers.
When I have a good understanding of the system, I can usually condense the answer as small as possible and then give resources if they're really curious. It's better than giving a bad answer or nothing at all some times

e.g. I am helping a math major with python, so he keeps asking about mathematical parallels in programming paradigms

they end up accidentally asking about functional programming and reactivity models

[grave jolt]

ah

[finite hare]

different student here btw

just giving an example

[grave jolt]

"Is async a monad?"

[spark magnet]

here's what i know about the relationship between async and monad: they both have five letters.

[finite hare]

I've read about a dozen articles on monads and I still don't understand exactly what they are. I've yet to do a deep enough dive into Haskell. I've got to finish some of those books I'm reading

[feral cedar]

a monad is just a monoid in the category of endofunctors

[finite hare]

I've got a much better understanding of async these days, however, which is nice

[finite hare]

ahhhh

now I get it

[swift imp]

It still boggles me that people use haskell

Like so few math majors even study categorical theory

And yet comp sci people, who def don't learn it in undergrad or grad school, use it

[finite hare]

There are some excellent articles on how Haskell solves problems that plague the entire rest of the comp sci field

or just functional programming in general

which I assume is why comp sci people pick it up

to continue the question, are there any other books people recommend?
I'll look into Fluent Python for now

Lyft's engineering blog has some interesting articles about monkeypatching

[swift imp]

Fluent is a good book

It took me about 5 months to really grasp the whole thing

And I'm still learning

[grave jolt]

Also two syllables.

[feral cedar]

they also have two letters in common :o

[grave jolt]

you don't really need CT to learn haskell 🙂

[feral cedar]

we might be on to something

[rich cradle]

You definitely are.

[grave jolt]

(as a side note, the answer is yes ```py
def unit(value: T) -> Awaitable[T]:
async def _coro():
return value
return _coro()

def bind(at: Awaitable[T], fn: Callable[[T], Awaitable[U]]) -> Awaitable[U]:
async def _coro():
t = await at
return await fn(t)
return _coro()

[grave jolt]

TL;DR (sorry for giving birth to a new monad tutorial):
Normal functions like A -> B are easily composable: you can stick A -> B to B -> C and get A -> C
But there are functions of a very particular shape, like A -> Optional[B] or A -> Awaitable[B] or A -> list[B]. They're not easily composable.

A 'monad' is a description of how to compose this kind of function for a particular generic type, like Optional. You might define a function like:

def compose(
    left: Callable[[A], Optional[B]],
    right: Callable[[B], Optional[C]]
) -> Callable[[A], Optional[C]]:
    def result(a: A) -> Optional[C]:
        b = left(a)
        if b is None:
            return None
        return right(b)
    return result

Now you can stick int -> Optional[str] and str -> Optional[bool] together

EXAMPLE

The idea is not very useful outside of functional languages, especially dynamically typed languages, but it leads to some neat patterns like

@null
def find_salary(user_id: int):
    user = yield get_user(user_id) # returns a user or None
    # but if result is `None`, it doesn't get out of the `yield`
    account = yield user.find_account()
    salary_entry = yield account.salary_entry()
    return salary_entry.salary()

>>> find_salary(-42)
None
>>> find_salary(51)
Salary(currency="USD", amount=45000.0)

(Optional is alias for Union[T, None] in Python)

[finite hare]

interesting, thank you

[sacred yew]

wsl?

much faster dev time b/c its not in a vm

[charred wagon]

WSL runs on Hyper-V which is virtualisation technology. So technically, it is still a VM.

[sacred yew]

thats wsl2?

[charred wagon]

Yes

[sacred yew]

oh i mainly use wsl 1

b/c its faster on windows files

wait hol up

isn't neos built using visual studio?

[spark magnet]

what's that got to do with it?

[sacred yew]

or are you just testing python on vm and neos on host

[spark magnet]

not me, @weary garden

[sacred yew]

you = leigh, oops

[grave jolt]

Example of when bool coercion introduces a silent defect.

def deliver(package):
    if not package.is_delivered:
        delivery_service.send(package)
        logger.info(
            "Package %s is sent to delivery",
            package.name if package.has_name else str(package.pno)
        )

See the bug? (Observed issue: no packages are delivered)

(concrete example is nonsense, but I did just this yesterday)

[signal tide]

logger should only have 2 g's?

[grave jolt]

nah, it's a typo

very loud bug

your IDE will scream at you

[charred wagon]

Looks fine to me. The logging statement is irrelevant since it's after the send() and t.f. could not be the cause of the observed issue

[grave jolt]

No log message either.

[signal tide]

"Package %s is sent to delivery" isn't that the message?

[charred wagon]

If is_delivered is True, then nothing happens because if not True is if False. Other case, is is_delivered is False so we have if True and the function must get called.

No idea what send() does but as far as I know it must be called if it's not delivered.

[grave jolt]

alright, spoiler

fix:

-     if not package.is_delivered:
+     if not package.is_delivered():

[charred wagon]

There is no way I would be able to deduce that without more context

[grave jolt]

Not saying you should 🙂

But it puzzled me for a while as well.

it reads correctly.

The hint was in that

Example of when bool coercion introduces a silent defect.

[peak spoke]

Not much that could be done here without context like mark said, more so when you already have package.has_name which I assume is an attr/property and the is_delivered seems to act in a similar way, but it is a straightforward debug

[charred wagon]

I think the problem was that I was reading it like it was correct, not like it was a bug

So I made assumptions I shouldn't have

[grave jolt]

sorry if the example was misleading

[charred wagon]

It did cross my mind to ask what is_delivered is or what send is but I just assumed it was a bool cause that's how it was being used

[grave jolt]

I guess it could also be missed in testing if you're using fakes -- you might make a FakePackage that has, according to the code, is_delivered as a property

same with monkeypatching

[raven ridge]

I had this bug in some code I wrote today. A condition that was always true because I checked the truthiness of a function, instead of a call to that function

[gleaming rover]

yeah

weak typing 😔

[limpid marten]

I'd recommend switching to WSL2! I believe I had some issues linking i386 compatability libraries until I switched.

[sacred yew]

i dont usually use 32-bit libs anyways

and whenever i need to i just switch temporarily to a wsl 2 enviro

[weary garden]

@sacred yew why would dev time be quicker if I ran Python on host OS rather than VM? I only need to use CPython occassionally to check something

[dense terrace]

Hello guys
anyone can help me about converting video from 2D to 3D.....
or provide Github Repo if possible

[spark magnet]

to be fair, it came up because you said you couldn't try a quick snippet because you hadn't started vmware yet, so there seems to be a small barrier there.

[undone hare]

Honestly it is a quite simple tool, I don't see why would one not install it on the host

Unless you want to run it in another host OS or so

[limpid marten]

Does Python have anything like Ruby's test suite https://github.com/ruby/spec?

[spark magnet]

There is a test suite in the CPython repo

https://github.com/python/cpython/tree/master/Lib/test

[limpid marten]

Thank you Ned.

[spark magnet]

np, what will you use it for?

[limpid marten]

I was curious on the use for __length_hint__ and how they'd use it in the test suite.

[thorny gorge]

Hey.. is scraping youtube legal? I need to get the number of views for a video..

[limpid forum]

scraping is against ToS
try their API https://developers.google.com/youtube/v3

also, I don't think it's the right channel

[thorny gorge]

Oh. Okay. Cool. Thanks

[unkempt rock]

https://github.com/beucismis/r2tg_bot

[limpid forum]

?
this seems unrelated to the discussion?

[pseudo cradle]

Man, singletons are really controversial aren't they. As well as global variables. I think global constants are okay, but apparently some people don't even like that.

[grave jolt]

global constants are okay 🙂

You can't make a program without global constants

literally

[flat gazelle]

Only for things that are actually global.

[limpid forum]

global constants are better than magic numbers used inside the code, for sure

[flat gazelle]

For example, if you put config options into global constants, you won't have a great time

[pseudo cradle]

I'm at a point in the development of a project where Singletons seem perfect for enforcing one instance of the class, caching, and logging. I'm trying to see if there's some overly compelling reason why I shouldn't use them, but I can't come up with anything. Like, I don't want to go through great lengths to avoid them for no other reason than that they're bad

Otherwise I'm really upping complexity

[flat gazelle]

That seems like a fair use. Some people prefer one logger and one cache per module so that a single module doesn't clog the whole cache up for all modules, but it isn't super important

[grave jolt]

Do people usually test the logging?

[pseudo cradle]

As in test it out to see that it works, or as in running it through a testing framework?

I don't do the second

[grave jolt]

As in, check logging through automated testing

[pseudo cradle]

Ah

I don't, at least not at this time

Too many other/larger project priorities

[grave jolt]

I think I've only seen them mentioned testing them at some talk by google, where they share how they failed by asserting a specific source code line in the log test

xD

[pseudo cradle]

haha

I'm sure it's worthwhile, I guess, but I find it hard to imagine a point in time where testing logging is a project priority

[spark magnet]

@pseudo cradle maybe we should clarify what you mean by Singleton

[pseudo cradle]

Sure. How can I clarify?

[sacred tinsel]

when I use global instances I don't usually enforce that there actually is only one instance, basically just instantiate something in module namespace and intend for it to be exported into the rest of the project, but I also don't write libraries

[grave jolt]

I have a hard time figuring out how to test message-constructing stuff, in general. If it's too vague, garbage like foobar459834958 can pass the test. If it's too rigid, it's close to hard-coding the format, so changing the message wording might break the test.

[pseudo cradle]

I have to write code that will both be used to interact with APIs (for developers) as well as used to run experiments and be used as a module (researchers/clinicians/other users)

[spark magnet]

@pseudo cradle there's "you can only make one" and "I will only make one". Which do you mean?

[pseudo cradle]

Primarily the first. I mean, you can make as many as you want, but I want them to all interact with the same underlying data.

[spark magnet]

why not just make one, and assign it to a global?

less magic, more honesty 🙂

[oblique crystal]

if you want the REST apibbackend to create only 1 DB pool - is it a singletone?

[pseudo cradle]

Ah, that's one more thing I need to work with/worry about, interacting with our ReST api

rest

I don't know enough about api, it's kind of embarassing

[pseudo cradle]

It's a good question, I'm still thinking about it, lol

I can't imagine a scenario where someone would instantiate more than one instance without it breaking everything

[spark magnet]

right, that's why you'll only make one

[pseudo cradle]

How do I stop other people from making more than one, complaining that it's broken everything, and coming to me to fix/debug? 😦

[spark magnet]

write good docs.

[limpid marten]

Perhaps I'm not understanding the complete scope, but assuming you've switched to dependency injection from singletons I can't really imagine a way a user of your API would do something wrong.

[radiant fulcrum]

Does anyone know the rough amount / minimum amount of dict lookups python does when interacting with an object?

[pseudo cradle]

I wish that was all that it took

[spark magnet]

I guess I'm wondering who is going to write code to read the config again? If you tell them, "get the config from import config; config.config," then why would they construct a new object? The right thing is easier.

[spark magnet]

"interacting with an object" is kind of vague.

[pseudo cradle]

Three people will be writing the code

Myself, other developers, then the more casual user of the module

[spark magnet]

OK, but what do they need to do, and what is the simple thing you can give them to do it the right way?

[undone hare]

Last time I fiddled with the __dict__ attribute it seemed like three interactions were done each time the interpreter wanted to read or write an attribute, although I may totally be wrong

[spark magnet]

why would they do import config; config = config.Config() when they could just do: from config import config or whatever

[pseudo cradle]

For a static config file it doesn't matter. But I do need some underlying dataclass that has the lifetime of the project for caching and outputting. It's just a matter of where it's being stored and how it's being interacted with

[radiant fulcrum]

Hmm true, basically any sort of general interactions e.g. calling functions, instance methods etc...

[pseudo cradle]

lifetime of the program, sorry

[radiant fulcrum]

though im assuming between the different operations they're going to have a different amount of calls

Tbh i should probably look at the ByteCode, might be a better idea

[spark magnet]

Make it a global variable.

[spark magnet]

the byte code won't tell you about dict accesses

[pseudo cradle]

Right now the underlying dataclass is a global variable. I'm considering singletons as a way to interact with it

[spark magnet]

how would the singletons improve the situation?

why not just tell people how to access the global variable?

[pseudo cradle]

There's a few concerns with having people access the global variable. One is that I deal with PHI that may be used in my algorithms where users shouldn't have direct access.

(personal health information, so hipaa considerations)

In theory I can allow people to interact with it, but I'd rather enforce it's integrity, tbh

Otherwise it's going to be a great big "Warning! You should not interact directly with this variable but should instead use X func or Y func to do so"

[radiant fulcrum]

can you think of any ways of measuring the amount of dict accesses?

[spark magnet]

@pseudo cradle ok, so you have functions that can access the global. I still don't see why anyone needs to "create an object" in order to access the configuration

[spark magnet]

i don't know a way, other than reasoning about the implementation. What's driving the question?

[pseudo cradle]

@spark magnet What are the alternatives?

[spark magnet]

def get_info_from_the_config():

[radiant fulcrum]

Im mostly curious just about how many lookups go into a single operation, as everything is built around them

[pseudo cradle]

Hmm

Well, the object will have other useful functions/parameters and such

[spark magnet]

then how about an object that has no data of its own, but works with the config object?

[pseudo cradle]

Yeah, that's a potential alternative

It's functionally the same, it just changes where the data lives

[grave jolt]

Isn't an object that has no data of its own just a bag of functions?

i.e. a module

[pseudo cradle]

It would add complexity to development if it had to get the data continuously instead of just getting some of its own to keep

Lots of interesting things to think about 🙂

I'm a fan of this channel and this discord

[grave jolt]

functional programmers have invented a nuclear pattern for configurations called 'reader'.
It's sort of like every function taking the config as an argument, but with a few conveniences

def send_mail(name):
    magic_password = yield "Mail:MagicPassword"
    mail_template = yield "Mail:Template"
    time = get_current_time()
    recipient yield "Mail:Recipient"
    mail_status = send_mail(
        magic_password,
        recipient,
        substitute(mail_template, name=name, time=time),
    )
    return mail_status.errno == 0

def send_emails(names):
    for name in names:
        ok = yield from extend_config(
            {"Mail:Template": "Hi $name, it's $time"},
            send_mail(name)
        )
        if not ok:
            msg = yield "Errors:Mail:CannotSend"
            raise MailException(msg)

def email_service():
    run_config({
        "Mail:MagicPassword": "123",
        "Mail:Recipient": "[e-mail]",
        "Errors:Mail:CannotSend": "Cannot send the email :-(",
    })

in Python it's more of an intellectual exercise, I suppose

Basically, run_config will interact with the generator by responding to the yields with appropriate config values

and helper functions like extend_config can alter the config locally

[spark magnet]

i do not understand this at all 🙂

[grave jolt]

well, the example isn't very self-documenting...

[gleaming rover]

ah

back to monads

[grave jolt]

It's very similar in spirit to this.
The null_aware function is very busy, but basically, it talks to the generator in an infinite loop, and it "filters out" the items it gets: on a non-None value it yields it back, otherwise it considers an error to have occured, and it just exits with None.
In other words, it's using generators for control flow

from typing import Optional

def null_aware(f):
    def new_f(*args, **kwargs):
        """
        Continuously get a new value from the generator.
        If it's None, exit early and return None.
        Otherwise, send the value back in.
        """
        generator = f(*args, **kwargs)
        query = None
        while True:
            try:
                query = generator.send(query)
                if query is None:
                    generator.close()
                    return None
            except StopIteration as e:
                return e.value
                # the `return`ed value from a generator
                # is saved in the `value` attribute of the exception
    return new_f

###

def safe_div(a: float, b: float) -> Optional[float]:
    if b == 0:
        return None
    return a / b

def safe_sqrt(a: float) -> Optional[float]:
    if a < 0:
        return None
    return a ** 0.5

@null_aware
def safe_inverse_sqrt(x: float):
    sqrt_x = yield safe_sqrt(x)
    print(f"{sqrt_x=}")
    inverse_sqrt_x = yield safe_div(1, sqrt_x)
    print(f"{inverse_sqrt_x=}")
    return inverse_sqrt_x

^ on a negative value, the yield safe_sqrt(x) never terminates, and None is returned immediately.
In the next case, the first yield terminates but the second does not.
So it does the same as:

def safe_inverse_sqrt(x: float):
    sqrt_x = safe_sqrt(x)
    if sqrt_x is None:
        return None
    print(f"{sqrt_x=}")
    inverse_sqrt_x = safe_div(1, sqrt_x)
    if inverse_sqrt_x is None:
        return None
    print(f"{inverse_sqrt_x=}")
    return inverse_sqrt_x

[grave jolt]

Maybe this will help... but it's more of an #esoteric-python territory

It was very challenging for me when I discovered this style of control flow in other languages, so hopefully I don't come off like this is easy or natural

Brandon Rhodes suggested a similar thing, where in order to not marry to a particular I/O (like requests), you can use generators
https://www.youtube.com/watch?v=DJtef410XaM

[whole light]

Hello, new here (but not to Python) and wondering if any Python core developers might hang out here? Submitted https://github.com/python/cpython/pull/23847 (which fixes bpo-31861: Add aiter and anext to builtins) a few months ago and still awaiting review. Hoping to get this landed in time for the 3.10 feature freeze coming up in May. Thanks for any advice.

[spark magnet]

@whole light some of the core devs are on this server, mostly because of a core sprint back in the fall.

[whole light]

Thanks @spark magnet! Could you please recommend any who might be interested in taking a look at that PR? (I've already tried pinging Yury a couple times, but he seems too busy to acknowledge, unfortunately.)

(Also, thanks for the great work on coverage!)

[spark magnet]

@whole light (thanks). I don't know who might be around, or in that area of the code.

[whole light]

Gotcha. In that case, any other advice?

[spark magnet]

i don't make pull requests against CPython, so I don't know the ins and outs. I know they get stuck sometimes.

[whole light]

Happen to know who is looking into that (meta) problem?

[spark magnet]

have you tried https://discuss.python.org/ ?

[cloud mirage]

Yoo nedbat

[spark magnet]

Yoo!

[whole light]

Yes, a month and a half ago: https://discuss.python.org/t/aiter-anext-review-request/6968 -- no response yet (though it got a few hearts, as did the PR -- users definitely seem interested in this)

[spark magnet]

@whole light sorry, that's all I got 😦

[whole light]

Thanks, @spark magnet. If you (or anyone else reading) happens to think of anything else in the future, please let me know.

[raven ridge]

Try asking about it in #async-and-concurrency - I know one of the asyncio core devs hangs out there sometimes

In fact... #async-and-concurrency message

That's a pinned message for the channel, so I don't feel bad 😋

Though it doesn't seem like they've been around much lately...

[unkempt rock]

when would I use is and use ==?

[spark magnet]

only use is with None. Use == for everything else.

[unkempt rock]

🤔

ok

[spark magnet]

• unless you are doing something very tricky

[feral cedar]

is is used for checking if two reference point to the same object. == just checks equality

[peak spoke]

True and False also pop up semi regularly and are a fitting use for is

[feral cedar]

yeah, False, True and None, are singletons, so you can use is with them

[boreal umbra]

come to think of it, I can't think of any times I've used is without None. I guess if I had an enum or an arbitrary sentinel object()

[unkempt rock]

ok

[weary garden]

Let the sunshine, let the sunshine in.

[spark magnet]

technically, x is False makes sense, but I would be very suspicious of it.

i don't see a reason to do that.

[peak spoke]

Need to differentiate between False and other falsy values in some cases

[spark magnet]

i'm not sure i've ever had to.

[raven ridge]

I certainly have.

[feral cedar]

it's just a codewars exercise, but it went something like "move all 0's to the end of the list", where False == 0 was an issue

[weary garden]

does my interpretation of slices syntax look OK?

[spark magnet]

@weary garden i'm not sure we know how to read your schema.

[weary garden]

I am dissapoint; I though it was obvious 😦

[spark magnet]

i'm not sure how "@" could be obvious 🙂

[weary garden]

@ are token matching rules

for the parant element

[spark magnet]

what are the '.' for? There's no dot in slice syntax.

[weary garden]

there is: look at last line

or am I misreading the grammar?

[gleaming rover]

how is this obvious??

[spark magnet]

i mean, in Python slice syntax, where are dots?

[weary garden]

look at last line

779 is copied from the grammar

[spark magnet]

This is in the Python grammar:

slices:
    | slice !',' 
    | ','.slice+ [',']

I don't know what the dot means in the last line. There's no dot in a Python program for slices.

[weary garden]

neither do I

[spark magnet]

why do you have dot literals in your grammar?

[weary garden]

because I assumed that dot in the grammar was part of the syntax

anyone know what that dot in the grammar means?

[spark magnet]

@weary garden have you written python programs with slices?

[weary garden]

maybe it means you need both the comma and the slice as an atom

[spark magnet]

yeah, idk. this is the new grammar for the new PEG parser.

[weary garden]

I only started learning Python last Friday so I haven't written any python programs yet

[spark magnet]

you are certainly taking an unusual approach

[weary garden]

how so?

my goal isn't to learn Python or to write Python programs, my goal is to create a Python implementation which requires me to learn Python as I go implementing each part of the Python language.

[spark magnet]

right. most people "learning Python as they go" would write some programs to help with the learning process. you've chosen a different approach.

[weary garden]

I don't have the time to do it that way.

I do not wish to become a Python developer so I have no need to write non-trivial Python programs

[spark magnet]

are you sure? transliterating grammars you don't understand seems like it would waste some time.

[strange heath]

interesting

I think the best way to learn is project based learning

[weary garden]

if I don't understand something I ask questions about it; as is the case here.

I have no desire to create any Python projects.

[spark magnet]

i'm not talking about projects. Just typing slices into the REPL would help elucidate the syntax.

will you have a test suite for your implementation?

[weary garden]

I would have thought any experienced Python programmer knows how slices work from a syntactical point of view so should be able to discern what the period represents in the grammar.

[spark magnet]

yes, i immediately noticed the dot literal in your grammar, and asked you about it.

[weary garden]

I got the dot from the PEG grammar

[spark magnet]

and i knew the dot in the Python grammar wasn't a literal, because slices don't have dots.

[weary garden]

stop being obtuse please

[spark magnet]

you mis-read the python grammar.

[weary garden]

so what does the dot in the grammar represent?

[spark magnet]

something meta-grammatical, i'm not sure what.

[weary garden]

not very helpful 😦

[spark magnet]

i've never read this grammar before.

[weary garden]

what PEG library is being used with this grammar?

[spark magnet]

i don't know.

based on this: global_stmt: 'global' ','.NAME+,

i think a.b+ means b or b a b or b a b a b, etc.

so ','.slice+ means, a comma-separated sequence of slice

[weary garden]

so period is acting as a contenation construct?

contatentation

pfft

[spark magnet]

more complicated than that

[weary garden]

doesn't foo.bar+ mean foo.bar foo.bar foo.bar?

[limpid marten]

I believe it's called a Gather.

[spark magnet]

it seems not to.

[limpid marten]

https://github.com/python/cpython/blob/58fb156edda1a0e924a38bfed494bd06cb09c9a3/Tools/peg_generator/pegen/grammar.py#L417

[spark magnet]

@weary garden cool, where is that from?

[weary garden]

https://www.python.org/dev/peps/pep-0617/#s-e

[spark magnet]

very nice

[weary garden]

also I was being stupid: I should have realised the period wasn't a literal as it wasn't in single quotes

[spark magnet]

@weary garden will you have a test suite for your implementation?

[weary garden]

not a Python one

[spark magnet]

How will you confirm that your python implementation works?

[weary garden]

initially the neos reference language will ensure full semantic concept coverage; I will only need Python test programs to ensure coverage of any Python-specific semantic concepts (if any)

[spark magnet]

you have 800 lines of a python schema like you showed above, and we just found a bug in it. Don't you need tests that confirm it is correct?

[weary garden]

I suppose the schema itself does need testing; I will think about a suitable approach to testing at a later date.

[limpid marten]

Could you run it against the CPython test suite, I imagine they use a lot of the available grammar.

[spark magnet]

usually it works better to test as you go

[weary garden]

my approach is to get as much of the grammar defined up front; then test it as I implement the semantic concepts; semantic concept folding should highlight any errors

[spark magnet]

ok, so you will have bits of python code in the tests that you write at that point, yes?

[weary garden]

oh I will be creating some python test programs sure

and there is plenty of extant Python code out there to test it with too

[spark magnet]

if you do that now, it could save you some time in this grammar

[weary garden]

I need to fully define the grammar first otherwise for any non-trivial program to parse

at the very least I need to be able to parse a print statement

including expressions

so the prority right now is statements and expressions

[spark magnet]

well, glad i could help with the slices and dots.

[weary garden]

the only slice I want right now is a slice of cake. I have no cake.

[spark magnet]

i can't help with that 🙂

[weary garden]

pfft

what does foo[0:1, 2:3,] mean? trailing comma?

[rich cradle]

Yes, never put a comma at the end of a list.

It's against the PEPs.

[feral cedar]

since when

trailing commas are awesome

[weary garden]

but this the grammar:

[rich cradle]

Isn't that what they said in the PEP 8 video?

[weary garden]

optional trailing comma

[rich cradle]

Or am I confused.

Never mind.

It's the other way around.

You should put it.

[weary garden]

@feral cedar what does trailing comma mean?

[rich cradle]

It means that there's a comma after the last list item.

[weary garden]

eh?

[limpid marten]

PEP8 is a style guide, not related to the grammar.

[rich cradle]

I screwed up. I confused myself. Ignore me.

[weary garden]

do you know what it is for @limpid marten

[limpid marten]

I don't believe this is allowed as it's considered a tuple which isn't a valid index.

[rich cradle]

This is a trailing comma
[1,2**,**]

[weary garden]

but [','] is in the grammar

[rich cradle]

That bolded comma is the trailing comma

[weary garden]

I know what a trailing comma is I am asking what it means in the context of a list of slices

[raven ridge]

This is a strange assumption. Experienced programmers don't necessarily know all the nuances of the grammar. Programmers know the conventional ways to write something, not all the possible ways to write something. Grammars are exhaustive, and full of edge cases

!e I just learned this week that you can do this:

print(5 . bit_length())

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

3

[weary garden]

@raven ridge however I am creating a Python implementation so I need to know what ALL of the grammar is referring to and "advanced-discussion" channel is the appropriate place to ask

[raven ridge]

oh, it's definitely the right place to ask, and we can help - but most experienced Python users aren't experts in the grammar.

[rich cradle]

Frankly, you'll never know everything about the grammar.

[weary garden]

that isn't acceptable

[rich cradle]

You can learn as much as you can, but not everything.

[weary garden]

disagree

[rich cradle]

Then you have unrealistic expectations.

[weary garden]

last resort will be to examine CPython source code. I need to know what ALL of the grammar refers to.

so not unrealistic at all just tedious trawling through CPython implementation to get an answer to my question

however the documentation for slices might provide the answer

[raven ridge]

https://docs.python.org/3/reference/expressions.html#slicings

a comma is allowed at the end of a slice list, it seems - but it's not ignored; it results in the expression being treated as a tuple, I believe

!e ```py
d = {(1,): 2}
print(d.keys())
print(d[1,])
print(d[1])

[fallen slate]

@raven ridge :x: Your eval job has completed with return code 1.

001 | dict_keys([(1,)])
002 | 2
003 | Traceback (most recent call last):
004 |   File "<string>", line 4, in <module>
005 | KeyError: 1

[limpid marten]

>>> class A:
...     def __getitem__(self, i):
...         print(type(i))
...         return i
...
>>> a = A()
>>> a[1, 2]
<class 'tuple'>
(1, 2)

So it's valid and treated as a tuple.

But the object has to well, take tuples as an index I guess.

[raven ridge]

@weary garden ^ lest you miss this link

[weary garden]

thanks

[raven ridge]

the /reference/ section of the docs is the closest thing to a language standard that exists. You'll definitely get further reading that than the grammar.

that points out that the grammar for slices is ambiguous, and how the ambiguity is resolved, for instance.

[weary garden]

well I will need to refer to that when dealing with semantics; at the moment I am more concerned with syntax

[raven ridge]

it explains the syntax in a more human-friendly way as well - the confusion about the dot wouldn't have happened.

[true ridge]

'<sep>'.<rule>+ expands to <sep> (<rule> <sep>)*

[visual shadow]

Depends on whether you're writing the list or container in a single line or multiple lines. For multiple lines the trailing comma is nice. For single, not really.

[visual shadow]

Not unrealistic. Remember that someone has to write the Grammar for example. Or tools that rely on it. Sure it's a very small subset of the python developers, and it's unfair to expect most people to know it. But there's people who probably are comfortable reading and parsing grammar and making use of it.

So it's not really a question of memorizing it all, more like "can you read and parse what this portion means"

[flat gazelle]

Trailing comma is equivalent to not having a trailing comma, unless it's the only comma, then it creates a singleton tuple in slices

[grave jolt]

https://mail.python.org/archives/list/[e-mail]/thread/HL3P7CXZX3U5SMNIJODL45BE6E72MWTI/
explicit exports?

[brave badger]

Yes plz

[grave jolt]

well, it's sad that all features in 3.10/3.11 will only see use in a few years

And even if you make a library that uses them, someone will complain how it doesn't work on their python 2.7...

[hoary jetty]

Hey I found some strange behaviour the property decorator and class inheritance

idk if its "advanced" enough for this thread but id love to get some ideas on this script I have

[raven ridge]

This isn't a help channel, but if it's something about the language itself, it's on topic here. Sounds like it would be.

[hoary jetty]

Well it's kind of help thing, but I have documented it in the readme of this repo https://github.com/YellowSub17/funky-class-properties

I guess you could say it's unexpected behavior? I made a help channel in help-berylium

Essentially, I have a parent and child class. Some of the parent attributes are "aliased" in the child class. Using property.setter methods, I can change the value of the parent attribute in the child class, but I can't change the value of the child attribute that is aliased to the parent attribute.

But the strangest part is that with ipython, I can can try to increment the value of the child attributed that is aliased, it throws an attribute error, but when I print the variable, I can see that it is incremented?

[flat gazelle]

the reason child.cvol += 1 still increments is because it is roughly equivalent to

res = child.cvol.__iadd__(1) #fine, not assigning to an attribute
child.cvol = res # problem, property has no setter

!e
you can observe the same behaviour with tuples of lists

a = ([],)
try:
    a[0] += [1]
except TypeError:
    print(a)

[fallen slate]

@flat gazelle :white_check_mark: Your eval job has completed with return code 0.

([1],)

[hoary jetty]

thanks @flat gazelle
I solved my problem in the help-berylium thread, but I was still interested in how the values change even though it threw an error at me

[quaint onyx]

Hello everyone! Can sombody help me or give me and idea about this question?I've learnt the python to do this github project but I know the calculations and only thing that I didn't success is how do I visualize the results like in this example.
https://github.com/EmilienDupont/openPitMining
GitHub
EmilienDupont/openPitMining
Open Pit Mining example with Gurobi. Contribute to EmilienDupont/openPitMining development by creating an account on GitHub.

He uses the Gurobi optimization library for optimization but how did he do this interactive webpage which it calculates the bloks according to economic values and produces the bloks from surface to bottom of pit
my question is what should I do to do webpage like this working with python and What should I learnd to do this
thank you

[spark magnet]

i don't understand why people want explicit exports. We tell people not to use "from x import *" and then we make new keywords to help do it?

[hoary jetty]

If we tell people not to use from x import *, why do we have the option to do it anyway?

[spark magnet]

There are lots of things (in any language) that are available but not recommended. Perhaps just for historical reasons.

[zenith topaz]

backwards compatibility.

[spark magnet]

speaking of contextual keywords like "export", coverage.py uses stdlib's keyword module to do syntax highlighting. Now we have case and match....

[flat gazelle]

• imports are useful for some things too, such as writing quick scripts around small modules or beginner code

[unkempt rock]

I won't like export, from two views, unfortunately there are still people who use python 2, what about them? , also there are still people who use a bit newer but still old python's (I always use the latest) so backporting newer projects would be such a pain, But if someone needs export like, You could define them in __all__ as a list or a tuple

[spark magnet]

and in python3.10a6 (which implements match), keyword.kwlist doesn't mention case or match 😦

[spark magnet]

you could argue against any new feature by saying existing python versions won't have it.

[unkempt rock]

Yeah i could, But doing that will open up new hacks which devs might use to kill a bug, Then backporting it would be awful

I will still say that defining them in __all__ does the job

[spark magnet]

i don't understand what hacks and bugs you are imagining

[flat gazelle]

I feel like the _ vs no underscore works fine, the issue is only reexporting

[spark magnet]

oh, case and match are in keyword.softkwlist, but so is _?

[flat gazelle]

_ in a case has a special meaning

it means ignore value

[spark magnet]

i thought it was an assignment in case just like everywhere else. Why a keyword?

[unkempt rock]

I'm not imagining, Bugs happen everytime, and sometimes hacks are needed to fix them, using export as a hack (maybe overriding a variable or function's view to other modules, to fix some unknown bug)

[brave badger]

unfortunately there are still people who use python 2, what about them
Python 2 has been deprecated for over a year now, it's unrealistic to expect new features to pop up, plus refusing to migrate to an actual supported version of Python seems to be a conscious choice most of the time, in a general developer context at least.

In terms of backporting in library development though, wouldn't be developing in the last supported version and a build matrix the proper way to go about making sure nothing breaks?

[flat gazelle]

because otherwise

case Class(_, _, v)
``` would be an error

[spark magnet]

ok, but what has that got to do with older versions of python? Or how is it different than any new feature being added?

[spark magnet]

i see.

[flat gazelle]

actually, I am not 100% sure it would be an error, I didn't read the PEP yet

but I wouldn't expect it do what _, _, a = cls does

[spark magnet]

this isn't an error: _, _, x = 1, 2, 3

[flat gazelle]

ye, it is only an error in function definitions

but I don't think that behaviour is desired in a case pattern

[spark magnet]

i'm not sure what to do with coverage.py "token coloring" now that we have soft keywords

[flat gazelle]

I don't think anything too bad will happen from miscoloring _, but the issue may become worse as more are added

[spark magnet]

i'm not worried about _, it's case and match

[unkempt rock]

Then what would happen if the export key gets real, Any other name (variable, functions) would become unexported, then in one release 99% of libraries out there need to write different code for the newer release

[spark magnet]

and yes, it's open season, so there will be more (like "export")

[spark magnet]

they would not use those semantics, and that might be a reason the proposal isn't accepted

[brave badger]

If a 3.x bump breaks 99% of Python code then that's bad design

I'm sure there's better ways of supporting such a feature without breaking existing code

[spark magnet]

the proposal will be, "only if an export keyword is used in the file, then unexported names are not exported"

which kinda sucks, so maybe it won't happen.

[flat gazelle]

I do wish there was a nicer way to reexport things

from ns import * is a bit meh

[unkempt rock]

If export gets real, it does make sense that every name which is not marked with export, will be left local, So a nightmare like python version based code will happen, In a large scal

[spark magnet]

they wouldn't do that, for the exact reasons you mention.

[brave badger]

This

import from x.y.z
```-ish but even then it's a tad bit confusing for beginners maybe

[unkempt rock]

Wait, why should we need it, We already have __all__ ?

[spark magnet]

i agree with you there.

[flat gazelle]

__all__ requires restating all the names

which isn't great

[spark magnet]

@unkempt rock if you read a proposal and think, "this will break 99% of programs", then either the proposal won't be accepted, or you haven't understood the proposal.

[flat gazelle]

but you can just have a insert __all__ lsp code action, so it isn't that horrible

not sure if it actually exists though

haskell also can do something very similar IIRC

[grave jolt]

I particularly dislike re-exporting typing stuff. That gives rise to things like prefixing typevars with _, or using explicit export of typing (e.g. typing.List), which makes annotations even less readable IMO

[spark magnet]

i think the @export decorator idea is good enough

[unkempt rock]

Yeah but has a lot of benefits, what if you want to avoid exporting a variable ? In version with export you need to find the line which you defined it, Then remove the export keyword, But with __all__, You need to just open the file and remove a single string

[grave jolt]

yep, if IDEs and such will understand it and give a warning if you import some 'unexported' stuff, it's good

(not necessarily IDEs, a linter perhaps)

[spark magnet]

@grave jolt would it be disallowed to explicitly import an unexported name?

[unkempt rock]

we already have __all__, pycharm shows a warning if you import a name which is not defined in __all__

[grave jolt]

# noqa
```, I suppose

(since it's purely for a linter, just like typing.overload)

[spark magnet]

i think python would allow it, if linters don't

[grave jolt]

yeah, it shouldn't actually hide it

[unkempt rock]

If export becomes real, Many (even who use the latest stuff), will still rely on __all__

[grave jolt]

That's fair, but that would lead to restating all the names, as lakmatiol said. And if you only want to unexport one name, it's unnecessary work.

Maybe an alternative is the opposite decorator, like @private

[flat gazelle]

the solution for that already exists, it involves renaming the name to start with _

[grave jolt]

yeah, maybe...

[unkempt rock]

You just need to append an string to a list

[spark magnet]

restating the name just doesn't seem that bad to me. It's an infrequent operation (how often do you add a new thing to export?)

[grave jolt]

that's true

[unkempt rock]

fare point

[flat gazelle]

yeah, it is a minor inconvenience all things considered

and mostly comes up in __init__.py, which are likely to be boilerplate anyway

[unkempt rock]

or a tuple, or whatever sequence you used

export is a bad idea, It has a single benefit, But a lot of bad points

[grave jolt]

Another thing I've seen people ask about at least a few times is exporting a class, but only for the purposes of typing (as in, "make the constructor private").
I guess you can do that with typing.TYPE_CHECKING somehow

[spark magnet]

how does exporting a class make the constructor private?

[grave jolt]

Not directly related to explicit exporting, just a side note

[unkempt rock]

A good point about python is that it doesn't enforce you a lot, unlike other languages, It just says, Hey, you shouldn't do that, That dev who added a underscore before the name had a point

[raven ridge]

!e Guido's suggestion goes beyond from module import * and would affect the behavior of from module import foo as well. I think that's really intriguing. It's a fairly serious problem in the language that this works:

from dataclasses import sys

[fallen slate]

@raven ridge :warning: Your eval job has completed with return code 0.

[No output]

[spark magnet]

i don't see that as a serious problem

[unkempt rock]

Nor me

[brave badger]

I suppose if unqualified imports had better semantics from the beginning and the global namespace wasn't too polluted then from module import * wouldn't be discouraged, but that's too far off from reality

[spark magnet]

@brave badger it's discouraged because it's hard to tell where names came from, not because it pulls in more than you want.

[brave badger]

Fair point

[unkempt rock]

If the writer of that code respected conventions, he/she couldn't do that

[raven ridge]

How not? If names that aren't prefixed with underscore are intended to be your public interface, isn't it a problem that you need to rename every module you import to avoid it becoming part of your public interface?

[unkempt rock]

put an __all__ to top of your module, then that code will break

[unkempt rock]

not that break

[raven ridge]

No, it won't...

[unkempt rock]

.

[brave badger]

Although such isn't the case for other languages like Haskell where qualified imports are explicit, but I do understand that there are different rationales for these types of things

[unkempt rock]

Other language enforce you don't do that, But this process needs to get a little more soft, And python did that

[flat gazelle]

I feel like there is very little to use __all__ in most cases, since it is quite obvious what you should and shouldn't use, and in cases where it isn't, you have __all__. I do agree that it isn't a perfect system, but it allows making quick and dirty libraries as well as large frameworks with clear exports

and well, I have yet to see a perfect import system

[unkempt rock]

because there are always exceptions, so this softness is needed

[unkempt rock]

also me, but export is not the solution

[raven ridge]

We provide introspection for modules today, and you can see which attributes they have. There's no way to use reflection to see which attributes are part of the public interface of a module unless it uses __all__

And there's no pushback if you accidentally import something that's not part of the public API. That seems quite problematic to me.

[flat gazelle]

[a for a in dir() if not a.startswith('_')] is simple enough for that

[grave jolt]

well, it's the same as with _underscored properties, right?

[flat gazelle]

it isn't perfect, but it is good enough for quickly creating a separate module

[raven ridge]

That would say that you can import json from dataclasses, which is wrong.

[grave jolt]

hm right, there's no direct equivalent with properties/attributes

[raven ridge]

Or, not wrong, but it's not part of the public interface

[unkempt rock]

an __all__ could solve this

[flat gazelle]

ye, it is an imperfect solution

but it is good enough

[unkempt rock]

I agree to that

[flat gazelle]

there is no perfect solution

[grave jolt]

||[a for a in dir() if not a.startswith('_') and a != "json"]||

[unkempt rock]

😂

[flat gazelle]

if there was, there would be a language that uses it

[unkempt rock]

There are ways around that problem, You could implement at _dataclasses then defien whatever you want to be public in __all__, then you could write a file named dataclasses.py with one stmt from _dataclasses import *, then josn won't be available

[raven ridge]

The status quo makes it easy for end users to accidentally import something that isn't part of the public API without realizing it, and makes it more difficult than necessary for library authors to declare their public interface (who among us has never renamed a function and forgotten to update __all__, because it's a list of string literals in a different part of the file?). I think it's entirely reasonable to say that the status quo is not perfect, and a better system could exist.

[flat gazelle]

I am not convinced there is a better system

[unkempt rock]

maybe there is one, but until that moment, all does the job, and python core developers should be pretty careful about changing it

[unkempt rock]

read my message above yours

[raven ridge]

No one is proposing removing __all__, don't be silly.

[unkempt rock]

Maybe one day, But until that, all is good

[raven ridge]

Any new explicit export system would be on top of/an alternative to __all__

[unkempt rock]

but what if a new non-explicit export system comes better than all ?

[raven ridge]

Then we'd tell people they should use it

[spark magnet]

@unkempt rock they aren;t going to break 99% of programs

[raven ridge]

And maybe remove __all__ in Python 4

[flat gazelle]

introducing export in some variant, even disregarding backwards compat now means you have to repeat export many times and its easy to forget to export something (which is a difficult bug to catch as well), an explicit hide keyword doesn't improve on leading _.

[unkempt rock]

with what change, we talked about 2 things here

[flat gazelle]

in general

[spark magnet]

with any change

[flat gazelle]

the disaster that was py2 -> py3 has taught psf not to do that

[unkempt rock]

ofc there are not going to, python core developers are professionals at their job, But what we can do is to say our feedback

[spark magnet]

sure, but to my earlier point, if you think a proposal will break 99% of programs, then you might have misunderstood the proposal.

[unkempt rock]

Then what is the reason of adding something which at least makes python developers worry about it ???

[raven ridge]

No one but you is worrying about it, because you're misunderstanding the proposal

[flat gazelle]

I don't think anyone is supporting that proposal here, there is just the idea that there is room for improvement on the currect state of things

[spark magnet]

did you worry that "match/case" would make re.match break because now match is a keyword?

[unkempt rock]

nope

[unkempt rock]

maybe someone who is reading all these will ???

[unkempt rock]

and we are here to talk about those proposals

[raven ridge]

I wonder if it would it be reasonable to warn when importing a module from a module that isn't a parent package of the imported module... Or maybe this is a thing linters should complain about (are there any that do?)

[brave badger]

The proposal is meh at best, but there's always a better way to improve upon something, even if that means borrowing semantics from other languages

[unkempt rock]

Discussion on the use cases, implementation and future of the Python programming language including PEPs, advanced language concepts, new releases, the standard library, and the overall design of the language.

[spark magnet]

yes, this is on-topic.

[raven ridge]

That is, I should be able to from collections import abc to import collections.abc because collections is a direct parent of collections.abc, but something sound warn me if I from dataclasses import json.

[unkempt rock]

if dataclasses has __all__ (i don't really know it does or not), Your linter (or at least pycharm) should generate a warning

[brave badger]

I personally don't bother with __all__ nor import *, __init__.py is boilerplate by nature so I just stick with manually importing stuff so following them works fine

[unkempt rock]

Maybe a internal linter to get these warnings, would be a nice idea

[flat gazelle]

__init__.py is only really useful if making public packages

anyway, neither flake8 nor pylance warn about this

[raven ridge]

We have those, in the form of warnings emitted by the interpreter.

[flat gazelle]

so that you don't make your users do

from package.name import Name
from package.another import Another
...

[unkempt rock]

but it is not as advanced as pycharm's, So that would really help

[raven ridge]

PyCharm's is static analysis, the interpreter's warnings are dynamic at runtime

They're not the same type of thing, so it doesn't make sense to compare which is "smarter"

[unkempt rock]

even through that python interpreter's warnings are "smarter", it is weak againts the static one

[flat gazelle]

the interpreter can't warn about things that might be correct

[raven ridge]

Static analysis needs to guess what your code will do when it runs. The interpreter knows exactly what it will do, because it's doing it.

[flat gazelle]

or well, the things pycharm warns about the interpreter will just error about

unless it's breaking convention

[unkempt rock]

so it will be easier for them emit that kind of warnings

[raven ridge]

Right.

[flat gazelle]

def fun():
    print(i_am_not_defined)
```pycharm will warn you about this name not being defined, but python will just error with a NameError

[raven ridge]

The interpreter can know when you've imported a module from a non-parent package, because it keeps track of where things have been imported from. It can emit a warning when the happens.

[unkempt rock]

you can configure your pycharm to show errors

[flat gazelle]

anyway, the warn about importing a non-parent package is not a bad idea

[raven ridge]

Or maybe not. PyCharm could be wrong, the name could be monkeypatched in as a module global from somewhere else.

[flat gazelle]

yup

we were completely disregarding the whole "module imports are non-deterministic" thing during this discussion

[raven ridge]

This isn't entirely innocuous. Accidentally importing something that wasn't meant to be public means your code can break when the author of a module restructures something they believed was an implementation detail. I've seen someone's code break because they were importing json from a library that didn't mean to be exporting json. They thought the library was providing a better replacement for the built-in json module. When the library updated to drop or reorganize its dependency on json, their code broke.

[unkempt rock]

we already have it (thanks to pycharm), But it would be nice if python interpreter was verbose as a c++ compiler

[flat gazelle]

it can't do that

because pycharm warnings are not errors at runtime

they are guesses what could maybe error/be incorrect

but it's just that, a guess

[unkempt rock]

but those guesses might get pretty accurate (at the runtime) if the python interpreter does those guesses

[flat gazelle]

those guesses are errors if they were right

[visual shadow]

Python interpreter doesn't guess.

But at that stage you're literally running code.

[flat gazelle]

def fun():
    globals()[input()] = 5
    print(hello)
import sys
from io import StringIO
sys.stdin = StringIO('hello\n')
fun()
```this program doesn't error

I don't think pycharm will agree

[unkempt rock]

but there will be no errors if you import something which isn't supposed to be imported

[unkempt rock]

read the above please

[raven ridge]

But... There could be warnings.

[flat gazelle]

yeah, that is a place where a warning may be suited

[unkempt rock]

that's what i am talking about

[flat gazelle]

but it can't be as verbose as the C++ compiler

or well, it already is

[unkempt rock]

ofc it can't, But a little more verbose then it is

[unkempt rock]

?

[raven ridge]

And ^ is exactly why. It's impossible to know what Python code will do until it's running.

[visual shadow]

How do we define an import that isn't supposed to be imported btw? I'm missing context.

[flat gazelle]

there isn't a whole lot that actually makes sense as a warning

[unkempt rock]

read the above please

[raven ridge]

from dataclasses import json

[flat gazelle]

importing a module from from a package that isn't its parent

[brave badger]

Tad tangential but was there ever a need to block importing transitive dependencies from application code? Through static analysis/compiler warning at least

[visual shadow]

Ah, so like a package that was importing one of its dependency libraries?

[flat gazelle]

yes

[visual shadow]

Got it, thanks

[raven ridge]

Picking up transitive dependencies of the headers you include is a problem in C and C++ as well...

Not in Java or C#, though.

[unkempt rock]

i mean static analysis is good, but if those analysis did actually happen at run time, It could get really better then static ones (I'm talking about dynamic analysis)

[flat gazelle]

I do wonder if there are things that use that correctly as well

[visual shadow]

Dynamic analysis? Sounds "expensive"