internals-and-peps

[paper echo]

not that cpython shouldn't also be fast but

[swift imp]

I have no idea, and honestly i don't fully understand cpython, cython, pypy, etc.

whatever JIT is

[main lynx]

i'm pretty sure cpython has immensely more buy-in than any other impl

having other impls is healthy but trying to make another the flagship would be a disaster

[prime estuary]

Well PyPy can't really run all the C-extensions efficiently.

[main lynx]

and most of the best python software is built on C extensions

[raven ridge]

PyPy is faster than CPython at running Python code, but slower than CPython at running C extensions.

which, paradoxically, makes it fast at running code that no one thought was important enough to optimize, but slower at running the optimized code.

[main lynx]

the next Best Thing for fast python might be mypyc, i can get behind its development for sure, because it's just compiling type-annotated Python directly to C for massive gains

[swift imp]

why would type annotations in C be so good? Thought the whole point was that you dont use annotations at runtime

[main lynx]

type annotations make the generated C faster

[swift imp]

How

[prime estuary]

Well there's also HPy, aiming to be a new design for the C API which is efficient on all implementations.

[dusty verge]

I mean, what do you mean by untyped C annotations, everything in C must be typed

[paper echo]

cpython is the original python, written by guido van rossum and implemented in c, hence "cpython". python source code is compiled to bytecode, and the bytecode is then interpreted.

cython is a python-like language that compiles to c code that uses the cpython c api, which can be turned into a shared object that uses the cpython abi.

pypy is an entirely standalone implementation of python that is itself written in a restricted subset of python called rpython. it uses a jit compiler and does not use cpython bytecode.

mypyc compiles type-annotated python to a shared object that uses the cpython abi (?) to c code that uses the cpython api, which can be turned into a shared object that uses the cpython abi.

nuitka does the same as mypyc, and also can generate a standalone executable.

[main lynx]

the same way C extensions are faster, you don't have to handle potentially any type of data in every signature

not using annotations at runtime isn't the point, it's a decision

[raven ridge]

that's not really what makes C extensions faster. They're faster because they can side-step the generic Python interfaces and use more efficient, lower level interfaces instead.

[main lynx]

mypyc does use cpython abi, the generated C is an extension module with a ton of statics

[paper echo]

i wasn't sure if mypyc used c as an ir

[raven ridge]

Executing Python bytecode is somewhat slow, but manipulating Python objects is also very slow. C extensions can minimize the places where Python objects need to be manipulated, and that's where the biggest part of their speed gains come from.

[main lynx]

mypyc does have its own IR, before it translates to C then to binary

[paper echo]

i wonder if you can write a python extension in nim, since apparently nim has good interop with c

[main lynx]

that would be the dream, but even better i think would be just writing type-annotated python and compiling it

[paper echo]

yeah, mypyc is pretty cool

[main lynx]

i'm in love with python type annotation in case you couldn't tell 😄 it really starts taking off in 3.9

[paper echo]

same, i think 3.9 was a big step forward

have you tried nuitka?

[main lynx]

nop

[paper echo]

(i haven't but i've been hearing more about it)

[swift imp]

I still don't understand how type annotations would help. Doesn't GvR explicitly refer to them as not type declarations lol

[paper echo]

https://nuitka.net/

[swift imp]

How does the annotation trickle down to C to improve anything?

[main lynx]

the annotations are hints that allow the C to make assumptions about the data and do less work

[swift imp]

So what happens when you pass the wrong data type then?

[paper echo]

fwiw entirely untyped cython still is sometimes 2x faster than the equivalent python, or more, on straightforward "loopy" workloads

[main lynx]

imagine the difference between a C++ class and a Python class, when you use a Python class it has to do a bunch of map lookups every time you get or set an attribute, a C++ class already knows the offset of that member so it goes straight there

[paper echo]

i think gvr has changed his attitude recently about type annotations. it's clear that types are the primary use for "annotations", to the point where we now have https://www.python.org/dev/peps/pep-0593/

[main lynx]

it probably explodes, as it should

if you checked your code with strict mypy the wrong data type will never be passed

[paper echo]

same thing that happens if you pass the wrong data type to any other c extension maybe

[swift imp]

Its actually in a talk explicitly about this Pep that he refers to them as "the thing that isn't a type declaration"

[main lynx]

it's really not hard to write end-to-end fully typed Python with no use of Any, and to prove it

[paper echo]

cpython i think installs some type checking guards

[paper echo]

lol

[swift imp]

mypy still can't handle descriptors, i wont hold my breath for these things

[paper echo]

not in a new project, but you're f'ed if you depend on a library that isn't "typing-friendly"

mypy still can't handle a lot of things

[main lynx]

he probably changed his mind around the time he was working on mypyc and realizing how good it could be

[swift imp]

Fwiw I like annotations, but I do find myself using them less for static type checking and more for applications of typing.Annotated and the like.

[paper echo]

my favorite mypy-isn't-ready-yet bug:

from collections.abc import Callable
class Foo:
    callback: Callable[[int], int]
    def __init__(self, callback: Callable[[int], int]):
        self.callback = callback

main.py:7: error: Cannot assign to a method
main.py:7: error: Invalid self argument "Foo" to attribute function "callback" with type "Callable[[int], int]"
main.py:7: error: Incompatible types in assignment (expression has type "Callable[[int], int]", variable has type "Callable[[], int]")
Found 3 errors in 1 file (checked 1 source file)

or:

from collections.abc import Sequence, Mapping
from typing import Union
JSON = Union[
    None,
    str,
    int,
    float,
    Sequence[JSON],
    Mapping[str, JSON]
]

main.py:4: error: Cannot resolve name "JSON" (possible cyclic definition)
main.py:9: error: Cannot resolve name "JSON" (possible cyclic definition)
main.py:10: error: Cannot resolve name "JSON" (possible cyclic definition)
Found 3 errors in 1 file (checked 1 source file)

(both of these have workarounds using Protocol, but they are moderately unpleasant)

[swift imp]

isn't that a from __future__ import annotations fix?

[paper echo]

nope, this is under 3.10 where that behavior is default

there's a longstanding bug on the mypy issue tracker about it

[main lynx]

fixed

from collections.abc import Callable
class Foo:
    def __init__(self, callback: Callable[[int], int]):
        self.callback = callback

[surreal sun]

Why was everyone so mad at Guido btw? about the assignment expresion operator

or the walrus operator

[paper echo]

i didn't realize mypy was inferring instance attribute types now, that's very nice

from collections.abc import Callable
class Foo:
    def __init__(self, callback: Callable[[int], int]):
        self.callback = callback
reveal_type(Foo(lambda x: x+1).callback)

main.py:16: note: Revealed type is "def (builtins.int) -> builtins.int"

[main lynx]

yup passes in strict mode

[paper echo]

i'm not sure, i think some people are just difficult

that said, i regret not complaining more about pattern matching outside of this channel

[main lynx]

alternative:

from dataclasses import dataclass
from typing import Callable

Callback = Callable[[int], int]

@dataclass
class Foo:
    callback: Callback

[paper echo]

my complaints wouldn't have amounted to more than a +1 on the existing complaints anyway

[paper echo]

not always feasible though. and the bug is still the bug

but good point

the JSON one is worse

[raven ridge]

You don't have to avoid Any. Code that uses Any just won't get any speedups.

[main lynx]

avoiding Any more about being able to prove type correctness with static analysis

[raven ridge]

sure. Any is just an opt-out from the type system.

[main lynx]

combine with liberal use of assert and you've got something approaching DBC

[raven ridge]

which is very frequently needed, because Python's type system is... limited.

[main lynx]

most of the typing problems i face are when i'm doing something i shouldn't be doing

trying to make sugary syntax, that kind of thing

or making things overly generic

so if you're going for systems programming, typing everything is a nice leash

[paper echo]

imagine a documentation generator that could parse out certain asserts from the start and end of the function body to use as documentation

[main lynx]

that would be cool, and probably not too difficult with ast

[grave jolt]

!pypi deal

[fallen slate]

deal v4.7.2

**Deal** is a Python library for [design by contract][wiki] (DbC) programming.

[paper echo]

oh that's the one i was wondering about

[grave jolt]

although I'm not sure it affects the docs

[paper echo]

i knew there was a dbc library i forgot about

[main lynx]

you still can't really do post-conditions with assert, too much duplication

i did try deal and icontract this week and it was ok until i tried to use mypy with it

[paper echo]

what went wrong with mypy?

[main lynx]

probably lack of stubs, didn't dig too deep

[paper echo]

hopefully with paramspec that'd be fixed

[main lynx]

assert is fine for pre-conditions and with some finagling probably ok for post-conditions too

[paper echo]

what if you use a special name for the return value w/ post conditions?

e.g. only asserts that use parameter names, nonlocal names, or the name _return are valid parts of the "contract"

__return__ or __result__ or something

[main lynx]

i think any name would work, as long as you see that the return value is a simple statement and read backwards from there (any assert before the return with that name in the expression)

[paper echo]

hmmm i see

ignore any returns not of the form "return" NAME

then use NAME to figure out the return value

@pure
def count_nonempty(items: List[str], item: str) -> int:
    assert len(items) > 0, 'List should not be empty.'

    result = items.count(item)

    assert result >= 0, 'Result should be non-negative.'

    return result

go does that funny thing with named returns

imagine 😆

@pure
def count_nonempty(items: List[str], item: str) -> (n: int):
    assert len(items) > 0, 'List should not be empty.'

    n = items.count(item)

    assert result >= 0, 'Result should be non-negative.'

[main lynx]

i've been looking into solving the issues people have with python for mission-critical applications

• type annotation and DBC close some of the formal verification gap
• making a pytest fixtures to check that tests do not produce garbage, later looking into static analysis for circular references over annotated code, with goal of being able to turn off the gc
• mypyc closes a big chunk of the performance gap

[paper echo]

...or use idris 2

mmm, tuple literals in annotations could be fun, is there a technical reason why this isn't possible?

def f() -> (int, int):
    return 3, 5

we've been down this road with other literals though, i know some people don't like the idea

[surreal sun]

Just guessing but maybe because __annotations__ needs classes not instances of classes?, and (int, int) will create an instance of the class tuple with two integers,
I don't know much thoguh so I could very well be wrong

[main lynx]

legacy of annotations is [] for better or worse

[paper echo]

as long as it's valid python syntax i don't know if it has semantic requirements beyond that

def a() -> (int, int):
    return 3, 5

def b() -> [int]:
    return [3, 5]

def c() -> {str: int}:
    return {"a": 9}

def d() -> {str}:
    return {"x", "y"}

# this clashes with existing syntax so it probably wouldn't ever work
def e() -> "this is a Literal":
    return "this is a Literal"

[main lynx]

def a() -> (int, int):
    return (int, int)

def b() -> [int]:
    return [int]

def c() -> {str: int}:
    return {str: int}

def d() -> {str}:
    return set([str])

which is correct?

[grave jolt]

Dict literals would be weird, because | means a different thing on types than on dicts

same with sets

actually, on other types | would be just invalid

[surreal sun]

what is bitwise or on a dict again? does it create a set?

[grave jolt]

a | b == {**a, **b}

[main lynx]

>>> {str: int}
{<class 'str'>: <class 'int'>}

[grave jolt]

And how often do you really need to typehint a list? Most of the time it's Sequence, Iterable or Collection

for dicts you can use Mapping or MutableMapping, or even TypedDict

[surreal sun]

ah, so it joins too, like a dict.update(dict1) except it isn't in place and it returns it

!e

dct = {'e': 2}
dct2 = {'f': 2}
print(dct | dct2)

[fallen slate]

@surreal sun :white_check_mark: Your eval job has completed with return code 0.

{'e': 2, 'f': 2}

[main lynx]

you often want to typehint a list, tuple, or iterator because there's big differences between all of these types

[paper echo]

good point, i would treat dict literals purely as sugar for dict[a, b] and make any forms other than {a: b} invalid

so the only variadic one would be tuples

[main lynx]

i don't think there are currently any type hints that could possibly be confused with non-primitive values?

[paper echo]

that, or you could have typed dict literals...

def hypot(point: {"a": float, "b": float}) -> float:
    return math.sqrt(a**2 + b**2)

[main lynx]

None is the only sketchy annotation

[paper echo]

you can use NoneType though right?

here's a fun question: should unions of typeddicts be treated like untagged unions?

[main lynx]

if you can figure out where to import NoneType from

actually NoneType = type(None) if that's your cup of tea

[paper echo]

right, you can't actually write it in an annotation

...this limitation is probably a good thing

from typing import TypedDict

class A(TypedDict):
    x: float
    y: float

class B(TypedDict):
    z: float

datum: A | B = {"x": 3.5, "y": 2.5, "z": 1.5}

main.py:10: error: Incompatible types in assignment (expression has type "Dict[str, float]", variable has type "Union[A, B]")
Found 1 error in 1 file (checked 1 source file)

[main lynx]

!e

NoneType = type(None)
def foo() -> NoneType:
    print("foobar")

mypy strict scans it

[paper echo]

heh

i wonder why None, True, and False got title-cased in the first place

[grave jolt]

🤔🤔🤔

from a practical standpoint, maybe

[paper echo]

from typing import TypedDict

class A(TypedDict):
    x: float
    y: float

class B(TypedDict):
    z: float

#datum: A | B = {"x": 3.5, "y": 2.5, "z": 1.5}

class AB(A, B):
    ...

datum: A | B | AB = {"x": 3.5, "y": 2.5, "z": 1.5}

this one passes

typeddicts are incredibly frustrating

i still want MyData(TypedDict, frozen=True) but have no idea how to implement such a thing

[main lynx]

you want frozen dataclass

[grave jolt]

Am I right that given py class C(A): z: float py datum: C | B = {"x": 3.5, "y": 2.5, "z": 1.5} passes, but ```py
datum: A | B = {"x": 3.5, "y": 2.5, "z": 1.5}

[paper echo]

i think so yes

[main lynx]

yes, because you are making a literal C

[paper echo]

or attrs, but i have some cases at work where it makes the most sense to leave the data as a dict and assume that it's type-valid when coming from an external source

[grave jolt]

What about ```py
c: C = {"x": 3.5, "y": 2.5, "z": 1.5}
datum: A | B = c

[paper echo]

but yes @main lynx, attrs+marshmallow+desert or attrs+cattrs or pydantic is probably a better way to go

[paper echo]

this works and i don't like it

(i think it's because typeddicts don't disallow extra fields)

(it's a good default but it should be disable-able)

[grave jolt]

Well, it's supposed to work -- after all, C is a subtype of A

[paper echo]

oh wait

right

variance

[grave jolt]

flashbacks...

[paper echo]

@main lynx this was my use case https://mail.python.org/archives/list/[e-mail]/thread/26MWFD46WP4OZKNLMQ3K2PMYP6M2UZ3L/#26MWFD46WP4OZKNLMQ3K2PMYP6M2UZ3L i was interacting with an external API where it would have been very useful

[grave jolt]

Hmmmph?

[paper echo]

🤷‍♂️

[main lynx]

and attrs supports that?

i should probably be using attrs

[paper echo]

mypy has a plugin for attrs, and attr.define uses the same protocol as pydantic and dataclasses

[main lynx]

from dataclasses import dataclass
from typing import Literal

@dataclass
class BaseDatum:
        id: str
        datumType: str

@dataclass
class Thing(BaseDatum):
        datumType: Literal["thing"]

@dataclass
class Widget(BaseDatum):
        datumType: Literal["widget"]

though i think the obvious TypedDict solution is not include datumType in BaseDatum

and rename it _BaseDatum so nobody gets wise

[paper echo]

yeah that would be the "nice" way to do it, with dataclasses/attrs/pydantic

this is what i ended up doing in my actual code, which happened to work for my use case but probably not other people's:

from typing import Literal, TypedDict, Union

class _BaseDatum(TypedDict):
        id: str

class Thing(_BaseDatum):
        datumType: Literal["thing"]

class Widget(_BaseDatum):
        datumType: Literal["widget"]

Datum = Union[Thing, Widget]

[fleet ice]

What does | do?

[verbal escarp]

it's binary OR

https://en.wikipedia.org/wiki/Bitwise_operation#OR

[spice pecan]

For integers it's a bitwise OR, for dicts - merging, for sets - unions, and starting with 3.10, a way to create type unions

[fallen slate]

@unkempt rock :white_check_mark: Your eval job has completed with return code 0.

True

[boreal umbra]

Traditionally it represents bitwise OR, like this

!e

a = 0b10011
b = 0b01001
print(bin(a | b))

[fallen slate]

@boreal umbra :white_check_mark: Your eval job has completed with return code 0.

0b11011

[boreal umbra]

but since its functionality for a given class can be defined via __or__, the convention is that you use it for set union-like operations.

(as opposed to +, which is used for addition or concatenation)

its sibling is &

[verbal escarp]

so serious?

[verbal escarp]

and its cousin ^

[peak spoke]

I'm still a bit concerned that pattern matching will be overused from the reception it got (compared to say the walrus which is now really only used where it makes the code more readable), and the (imo) comparatively complicated syntax won't help

[verbal escarp]

you mean for code structure?

[weak tusk]

Any one who is practicing data analysis here ?

[verbal escarp]

yeah, but you might want to check #data-science-and-ml or #algos-and-data-structs with that primer

[weak tusk]

Thanks

[verbal escarp]

so, basically the same concerns with pipes? 😉

[peak spoke]

Well, I don't see pipes coming into the language in its current state, but pattern matching in inevitable at this point

I didn't follow the discussion yesterday but with other languages with proper lambdas and expression based syntax it sometimes feels like I'm reading an operator soup.
Admittedly the language is usually not being properly utilized at that point but it seems common enough

[verbal escarp]

hm.. well, i think it's mostly a cultural thing

it's not so much about what kind of tools people have at their disposal but more what they want to build with them

the other day the phrase about hammers and nails came up, but with a chisel, you can use hammers to build pyramids

on the other hand, you can use a hammer to smash statues into dust

if we don't set our goals to be like "code should be as short and cryptic as possible" (i'm looking at you #esoteric-python ) and let that get into everyday-code, i don't see any danger with introducing new shiny tools into the language

[quaint pike]

What pattern matching are we talking about? Python has had a regex library since forever

[verbal escarp]

https://www.python.org/dev/peps/pep-0622/

[peak spoke]

From my experience reading the source of dependencies when I need something from them, it does look like some of the authors really don't particularly care about how readable something is and with new syntax that has many uses it just expands how they can be mis-used to produce more code that is borderline unworkable.

But I do agree that new features in the language are nice with the assumption that people will use them properly, which can be applied in a controlled environment like your own project or something that's properly maintained by a larger community; for the pattern matching I'll wait and see how it's utilized as I didn't look into the functionality in depth myself

[quaint pike]

ah, ok, you mean pattern matching Python syntax itself. It's a very cool feature in Mathematica!

But in the proposed code examples, I'm seeing a lot of things condensed into cryptic symbols that would have previously been written out in (horrors!) isinstance checks. Not sure if I like that.

[verbal escarp]

there definitely will be a lot of code that will abuse features at first, simply because people will want to experiment and see if they useful applications (hammer and nail applies)

once they get the hang of things, it'll settle down and cut back

natural order 🙂

[quaint pike]

I think it will take some special care to keep this from being heavily reliant on cryptic symbols. One of the nice things about reading Python is seeing words for and, or, not, and x if p else y

[verbal escarp]

cryptic symbols?

did you have a look at the PEP?

[quaint pike]

just did

[verbal escarp]

match, case =, *

i think that's all the symbols it uses

[pallid moon]

How I wish for had a clause to deal with an empty iterable

Quite handy when the iterable is not a variable (although it would make the code slightly more tidy) but a generator.

[quaint pike]

What sort of use case do you have in mind, exactly?

[pallid moon]

@quaint pike For example:

for item in get_items():
    # work with item
otherwise:
    logger.warning('No items found')

[fallen slate]

@warm kernel :white_check_mark: Your eval job has completed with return code 0.

no

[pallid moon]

The else clause is executed except if a break is found in the for clause

!e

for x in [1]:
  print(x)
else:
  print("no")

[fallen slate]

@pallid moon :white_check_mark: Your eval job has completed with return code 0.

001 | 1
002 | no

[pallid moon]

Now that we have soft keywords, this could be implemented in a way that doesn't break peoples's code (I believe). Haven't proposed to python-ideas because I can't think of a good keyword

Not that I like soft keywords, but they are a handy tool

[pale juniper]

!e py print("hi")

[austere rapids]

Hi. How to install python module for development environment?

is it pip3 install -e package_name ?

[main lynx]

python/pip have no concept of a development environment, consider using virtualenv and having a separate requirements.txt for development deps (dev-requirements.txt maybe)

deal doesn't have an issue tracker, but it turns out it has humongous overhead when run in not-debug mode, very sad

icontract is much slower in debug mode but very low overhead in not-debug mode

so looks like i'm optimizing icontract instead of deal :>

[elder blade]

But venv?

[main lynx]

tell Bassam about it if you like

[bronze briar]

Hi, does anyone know a bit about floating binary numbers? When the matissa is negative, I don't know if I take Two's complement OR just assume the bit on the left is negative and do the rest as normal? Every tutorial I have watched does it differently! Thanks

[dark wasp]

Big brain

[charred pilot]

6 ns is incredibly small, i don't think that's accurate

they technically don't do the same thing, but i don't think that's what is causing the issue

i'm not sure what's causing that, but when i replicate it, i get more reasonable results.

seq = "1"*1_000_000
d = {i: int(i) for i in "1234567890"}

%%timeit
res = []
for ch in seq:
    if d.get(ch, ""):
        res.append(ch)
110 ms ± 5.87 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

%%timeit
res = [d.get(ch, "") for ch in seq if ch in d]
105 ms ± 10.2 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

ah, interesting:

%timeit "r=[]; for char in prog: if a.get(char, ""): r.append(char)"
7.19 ns ± 0.11 ns per loop (mean ± std. dev. of 7 runs, 100000000 loops each)
``` it probably has something to do with how timeit is running the code you give it

[peak spoke]

you're timing the loading of the string literal

[charred pilot]

ah.

😔

[fleet ice]

What is mantissa?

How does range() function takes args

[visual shadow]

!d range

[fallen slate]

range

class range(stop)``````py

class range(start, stop[, step])```
The arguments to the range constructor must be integers (either built-in [`int`](https://docs.python.org/3.10/library/functions.html#int "int") or any object that implements the `__index__` special method). If the *step* argument is omitted, it defaults to `1`. If the *start* argument is omitted, it defaults to `0`. If *step* is zero, [`ValueError`](https://docs.python.org/3.10/library/exceptions.html#ValueError "ValueError") is raised.

For a positive *step*, the contents of a range `r` are determined by the formula `r[i] = start + step*i` where `i >= 0` and `r[i] < stop`.

For a negative *step*, the contents of the range are still determined by the formula `r[i] = start + step*i`, but the constraints are `i >= 0` and `r[i] > stop`.

Similar names: pyside2.range, django.range

[visual shadow]

It basically looks at number of args passed and infers the rest.

[raven ridge]

Roughly like:

def range(*args):
    if len(args) == 1:
        start, stop, step = 0, args[0], 1
    elif len(args) == 2:
        start, stop, step = args[0], args[1], 1
    elif len(args) == 3:
        start, stop, step = args[0], args[1], args[2]
    else:
        raise TypeError("Wrong number of arguments")

[prime estuary]

It's a unique/tricky one, actually CPython has a tool called "Argument Clinic" which autogenerates the C argument parsing code (all C builtins get given *args/**kwargs and have to deal with it themselves). But range has bespoke parsing logic.

[lilac vapor]

Hello! Is it better to use image generators from Keras or is loading the images in a numpy array better? For Jupyter NN thanks!

[nova iris]

meanwhile, in the future:```py
def range(*args):
match len(args):
case 1:
start, stop, step = 0, args[0], 1
case 2:
start, stop, step = args[0], args[1], 1
case 3:
start, stop, step = args
case _:
raise TypeError("Wrong number of arguments")

really reduces some boilerplate and makes it easier to understand

but the double indentation is kinda ugly in this case (no pun intended)

[worldly ridge]

much better looking tbh.

[flat gazelle]

why not

match args:
    case [stop]:
        start, step = 0, 1
    case [start, stop]:
        step = 1
    case [start, stop, step]:
        pass

[raven ridge]

Yeah, that's definitely not the way pattern matching is meant to be used. It's called "structural pattern matching" because you're meant to use the pattern to validate the structure of the object. lakmatiol has it right.

[nova iris]

wait a second

that's like 10000000000000000000000 times more elegant

orz

incredible

[nova iris]

yep yep, i was still in that switch case mind lmao

[flat gazelle]

this is the python alternative to function overloading/multiple dispatch

[raven ridge]

I've been explaining it by analogy: pattern matching is to arbitrary objects as regex is to strings.

not perfect, but neither is the function overloading version, since pattern matching can obviously be used for more than just dispatch.

[swift imp]

I think with 3.9 function overloading can be used for more than dispatching.

typing.Annotations seems very strong

[white nexus]

Help direct me to a different channel if one suits it better, but why does flake8 not support pyproject.toml yet? I'm looking at the repo and it seems like they just don't want to or something, but that was also 2 years ago.

Are we doomed to have to continue to use extra configuration files for .flake8?

[main lynx]

yet another single config file for all python tools?

[white nexus]

.... 👀

riiight, tox.ini used to be the defacto config file lol

[main lynx]

and flake8 reads that, sounds like you're not doomed after all

it would break the laws of pythonics if you only needed a single configuration file in your python project

[nova iris]

pov: reversed uses __len__ but iter doesn't

i mean, it kinda makes sense

but at the same time, why don't they make it symmetric

🧂🎲 LMFAO

THAT IS GENIUS

[deft pagoda]

no me

probably because __reversed__ was added later? and you can't use IndexError the same way you do going forward

In [3]: class A:
   ...:     def __len__(self):
   ...:         return 5
   ...: 
   ...:     def __getitem__(self, i):
   ...:         if i > 10:
   ...:             raise IndexError(i)
   ...:         return 0
   ...: 

In [4]: for i in A():
   ...:     print(i)
   ...: 
0
0
0
0
0
0
0
0
0
0
0

[peak spoke]

it has to know where to start from

[nova iris]

yeah, that's why i said it makes sense - it needs to know the length so it knows where to start (@peak spoke mentioned that too), but why doesn't iter do that as well, so that when __getitem__ doesn't have indexerror raising, it'd still work if __len__ is there?

[deft pagoda]

probably to use as few methods as it needs

[nova iris]

hm, mayhaps

[deft pagoda]

you can iterate over infinite iterators with just __getitem__

[nova iris]

yep

[deft pagoda]

but how do you reverse an infinite iterator

[nova iris]

...

wait

a

minute

[nova iris]

so THAT's why... this asymmetry is deliberate to support infinite iterators having a reversed

but also maybe efficiency, like you said earlier

[deft pagoda]

if my iterator is all the natural numbers and i want to reverse it, i'll just start at the last natural number, ez

[peak spoke]

why would an infinite iterator have a finite length and how would it help

[nova iris]

no idea

but it could happen

[peak spoke]

the iter behaviour is a remnant of how iteration worked before

I guess it could be expanded to utilize the length but that could be incompatible with how it was used previously and is really not important when __iter__ exists and should be used

[deft pagoda]

yeah, it's very py2-ish

of course __reversed__ exists so one can always customize

[misty dust]

@analog tendon

[unkempt rock]

was trying to think about why @overload exists. i think it's because when using TypeVar you're restricted to the same type in and out of the function. whereas using @overload you could use any combination of type(s) going in and out of a function, and with @overload they can be different types that go in and out so it's less restrictive than TypeVar. anything else im missing?

[prime estuary]

Overload also exists so you can indicate which combination of parameters are allowed. For instance here's the definition of sorted() in typeshed:

@overload
def sorted(__iterable: Iterable[SupportsLessThanT], *, key: None = ..., reverse: bool = ...) -> list[SupportsLessThanT]: ...
@overload
def sorted(__iterable: Iterable[_T], *, key: Callable[[_T], SupportsLessThan], reverse: bool = ...) -> list[_T]: ...

If no key parameter is provided the iterable must support __lt__() so it can be ordered. But if the key is provided, instead the iterable can produce anything, but the key function must accept that and produce an orderable value. Another example is int(), where it converts stuff to integer if one argument, but if you provide a base the input must be only string/bytes.

You could also do mutually exclusive stuff, defining it so if one argument is parsed another can't be.

Or the strange behaviour of range, where it's range(start) or range(start, stop, step=1).

[paper echo]

You can overload on literals too

@overload
def append(
    seq: Sequence[A],
    val: A,
    inplace: Literal[True] = ...
) -> None: ...

@overload
def append(
    seq: Sequence[A],
    val: A,
    inplace: Literal[False] = ...
) -> Sequence[A]: ...

[deft pagoda]

branchless programming go

[paper echo]

i always wondered how branchless programming worked

pandas has an "inplace" api like that

data2 = data1.set_index('y')

data3.set_index('y', inplace=True)

and numpy with its optional out=

[fallen slate]

@unkempt rock :white_check_mark: Your eval job has completed with return code 0.

I am true

@unkempt rock :white_check_mark: Your eval job has completed with return code 0.

I am true

[paper echo]

Aha

Hey that's a monoid

A set with a binary operation and an "identity" element, like 6 + 0 = 6

[gleaming rover]

#category-theory-discussion

[paper echo]

Hey algebra is useful

Even if you don't go to the extreme levels of abstraction of categories

It's a tool for thought

Data types that form a monoid can be used for branchless programming

Now that's a general principle you can use

You can probably do even better if you have an inverse, forming an algebraic group

[fallen slate]

@unkempt rock :warning: Your eval job has completed with return code 0.

[No output]

[paper echo]

dis already prints

I'm not sure why it didn't print in that output

[fallen slate]

@unkempt rock :white_check_mark: Your eval job has completed with return code 0.

001 |   4           0 LOAD_CONST               1 ('true')
002 |               2 LOAD_FAST                0 (val)
003 |               4 BINARY_MULTIPLY
004 |               6 LOAD_CONST               2 ('false')
005 |               8 LOAD_FAST                0 (val)
006 |              10 UNARY_NOT
007 |              12 BINARY_MULTIPLY
008 |              14 BINARY_ADD
009 |              16 RETURN_VALUE
010 |   7           0 LOAD_FAST                0 (val)
011 |               2 POP_JUMP_IF_FALSE        8
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/inarebuxix.txt?noredirect

[paper echo]

So this is a micro optimization right? In cases where the short circuiting behavior of a branch is less valuable than avoiding branch prediction in the CPU?

So for really "small" branching in hot areas of code

Because arithmetic is stupidly fast

It blows my mind that doing "more" work is faster, sometimes significantly so

Given the overhead of cpython I wonder if that's a valid optimization strategy

I'm extrapolating from what you found!

This could be interesting to benchmark

I had only heard of this before in some forum thread about go I think

Maybe an article on hacker news

Benchmark it, see if there's a performance gain, and then see if there is still a performance gain on other python implementations

I would do it myself but I'm not at a computer

[charred wagon]

I wish there was a way to use TypedDict with **kwargs. Right now I have to choose between lacking type information and having to manually construct a dict from parameters.

The latter is especially tedious if I don't strictly require all parameters to be filled.

I know that for *args it treats it as a homogeneous type e.g. *args: int implicitly means a tuple[int] or "each thing in args is an int". Not sure about **kwargs though.

It doesn't work with TypedDict as expected though

[swift imp]

**kwargs accepts anything so how are supposed to know apriori what valid keywords are to use with TypedDict?

[charred wagon]

I'm not sure what you are trying to say. It seems clear to me what would be valid:

class KeywordArgs(TypedDict):
  a: int
  b: str

def f(**kwargs: KeywordArgs) -> None:
  ...

f(a=1, b="hello")  # valid

I actually found this on mypy https://github.com/python/mypy/issues/4441

I'm an IDE user so I would ideally like for IDEs to make the experience similar to what it would be if the parameters were individually defined on the function.

Maybe I can just solve this with @typing.overload?

Well the particularly cumbersome thing with the latter is when the same behaviour as **kwargs is needed i.e. if a parameter is not passed then it's excluded from a dict. The only way I've found to solve this is to have a sentinel value as a default and have an if statement for each parameter that checks its type. This gets slightly more annoying if using a type checker, since the annotations have to included the sentinel value and that makes readability poorer.

Well it has on mypy

And as we were discussing I thought of using an overload

Yeah I will have to play with that

mypy doesn't like it when I only have a single overload :)

I can cheat it by defining an overload that just takes kwargs

An overload seems to work fine besides that quirk. I also cannot avoid the union with the sentinel type unfortunately

[paper echo]

The python type system basically doesn't account for missing attributes

That Sentinel PEP should help at least make better missingness flags

[charred wagon]

For the most part I do not mind manually creating a sentinel. The annoying bit is that it has to be included in the type annotations.

And I guess that's fair but it does make things uglier

[paper echo]

It'd be interesting if you could mark attributes as truly "optional", as in might be missing entirely

[charred wagon]

I think that would require introducing new syntax since it has to be applied on a per-parameter basis

[paper echo]

Or just an annotation like ClassVar

[charred wagon]

Well okay, it could just take the current sentinel idea and use that, but that seems unintuitive

[paper echo]

Too bad Optional is taken, should have been Nullable

x: Nullable[int] would be Union[int. None], and x: Optional[int] would only be valid in class definitions , and would indicate that the attribute might be missing

[deft pagoda]

oh yeah, i started a sentinel library yesterday

[charred wagon]

Oh you're talking about attributes rather than func params?

[paper echo]

You'd refine the type with hasattr or getattr

Oh yeah sorry

But tou could use that to annotate a typeddict which I agree should be valid for kwargs

[charred wagon]

TypedDict already has total=false, though it doesn't offer fine grained control over which ones are optional

Now that I think about this, didn't I see a PEP for this 🤔

Might have been thinking of https://www.python.org/dev/peps/pep-0645/

[deft pagoda]

i don't really care about typing that much, but this looks pretty clean

[gleaming rover]

I find it a bit weird that None doubles up as the unit type

[deft pagoda]

you can use name flags in dicts to do weird stuff so that this would work:

class KeywordArgs(TypedDict):
  ___required___
  a: int
  b: str
  __optional__
  c: float

[charred wagon]

If you look at the mypy issue they were considering **kwargs: Expand[KeywordArgs]

[deft pagoda]

i've done this before with dicts i've made with __prepare__

i dunno if that's more readable or more esoteric

[charred wagon]

It's probably not something anyone is used to seeing

[deft pagoda]

yeah, it's pretty alien

it's easy for me to parse though, personally

[charred wagon]

I don't know how I feel about it. it's not terrible...

[livid mason]

if only python had static typing, it will be as fast as C. Right now, we have solid support for typehints, but Python discards them to still stay a dynamic language.
look at the code example below

def my_static_typed_loop():
    my_int: int = 12
    i: int
    for i in range(my_int):
        # do something

If Python could infer static types here, this for loop can be compiled to pure C code and be as fast.
This is done with Cython, and it disappoints me that the CPython compiler doesn't.

maybe Python should be dynamic by default, but when we type hint our code, it should infer static types.

[grave jolt]

That would be a big breaking change to the entire language. Python is dynamically typed. If you want a fast, statically typed language, there are other options.

Even with this simple example, you can't optimize it to C-level for (int i=0; i<12; i++) because you don't know that range hasn't been redefined in the global scope.

For simple examples like this, you can use numba/cython/other tools. But for more complex scenarios it's going to be tricky. If a function accepts a parameter, there's no way to ensure that the argument passed to it will be of the correct type. What do you do if it's not -- silently segfault?

[spice pecan]

Another reason it would be difficult to optimize to C level is because of things like ints having unbounded range

[livid mason]

then what are the advantages of dynamic typing? what is python trading for speed?

[grave jolt]

I suppose:

1. Implementation and the language itself are simpler
2. Some invariants are very hard to describe in a static way, and in many languages it's impossible or very verbose.
3. Reflection is much easier

class LoggingProxy:
    def __init__(self, obj):
        self.__obj = obj

    def __getattr__(self, name):
        value = getattr(self.__obj, name)
        print("{}.{} -> {}".format(self.__obj, name, value))
        return value

    def __setattr__(self, name, value):
        print("{}.{} := {}".format(self.__obj, name, value))
        setattr(self.__obj, name, value)

    def __repr__(self):
        return "LoggingProxy({!r})".format(self.__obj)

You can look at libraries like pydantic, dataclasses, attrs, or just at collections.namedtuple -- they are all possible without adding anything to the language
4. For quick and dirty prototypes you can make the code work without pleasing the type checkers

[livid mason]

@grave jolt could you please explain "reflection"? I didn't get you, thanks for your detailed reply!
I've seen reflect-metadata in typescript, is it something similar

also, I read somewhere that from Python 3.7+, adding typehints and annotations makes Python faster. is this True?

[grave jolt]

"reflection" is when you inspect meta-parameters at runtime. In this example I'm getting an attribute by name, but the name of the attribute is only know as runtime

[grave jolt]

No, type annotations on their own don't impact runtime at all

[livid mason]

ohh but startup times were slow before 3.7

[livid mason]

so, the typing module is only present in Python for a better IDE experience (intellisense). Am I right?

[elder blade]

Yes because importing typing was slow afaik

[grave jolt]

Yes, and for finding bugs before you run the program using a type checker.

You can also inspect the annotations (which is what pydantic does) and do something with them

[livid mason]

I think that I'm getting addicted to statically typed languages ;)

[grave jolt]

which ones do you like?

[livid mason]

absolutely love TypeScript, have been learning C++

[elder blade]

Pretty sure that Guido has said that annotations will only ever provide hints and never be used for something else in CPython.

[grave jolt]

TypeScript is gradually typed, just like Python 🙂

You can still have any and such

[livid mason]

oops, but it really feels like typescript is statically typed

[grave jolt]

it's just that its type system is better than the one in Python

[gleaming rover]

the difference is that the TS compiler will yell @ you, whereas type checking in Python is delegated to 3rd party programs

[grave jolt]

IMO

[gleaming rover]

and this

[elder blade]

I think we all agree

[gleaming rover]

there's also compile-time reflection

[grave jolt]

yeah, Rust and other languages with hygienic macros have this sort of stuff

but in 1991 this was very cool

and, well, Rust is pretty complicated compared to Python

[livid mason]

having experience with languages like elixir, I would like to know if Python supports DSLs and macros (I know a little bit of metaprogramming with metaclasses)

[grave jolt]

there are no macros

although I think there are some libraries that allow them

unless you have a very specific definition of DSLs, you can make a DSL in pretty much any language

[livid mason]

hmm, something like what sinatra does

or ecto, from elixir when u define database schemas

[gleaming rover]

some languages are better @ making DSLs than others

of course, some go even further and give you symbol soup (sCaLaSkeLl)

[grave jolt]

Python has various ORMs that do the same

[livid mason]

also, I have this doubt - why not make PyPy the default interpreter instead of CPython, if it's faster?

[visual shadow]

It's not blanket faster, since cpython has a whole ecosystem built around it that interfaces with C

[spice pecan]

And neither is it up to date with the modern versions

[grave jolt]

It's not always faster (especially when using C-extensions), and it doesn't support all popular libraries with C extensions

[visual shadow]

That layer of interaction has to be filled in for pypy and this has overhead ,and does not have full support

[gleaming rover]

like people often say...PyPy is good @ making the fast stuff faster and the slow stuff slower

😔

[visual shadow]

Also, I'd say that decisions that break things for others like this often don't to down well, even if it was better overall

So it won't even be on the table as far as I can see

[livid mason]

hmm thanks for clarifying, guys!

[livid mason]

I meant the dsl? You have special tokens to define schemas

[grave jolt]

you probably have a very specific definition of a DSL then

[livid mason]

forgive for writing elixir here, but this is what I meant

defmodule User do
  use Ecto.Schema
  # special syntax here?
  schema "users" do
    field :name, :string
    field :age, :integer, default: 0
    field :password, :string, redact: true
    has_many :posts, Post
  end
end

[elder blade]

It's generally not that easy haha. CPython is the default because it's the one that Guido worked on pretty much

[grave jolt]

I understand "DSL" to mean a sub-language embedded into the host language, that's possible in any programming language, with different degrees of awkwardness

[elder blade]

from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int = 0
    password: str  # There is no refsct thing, but you can define another model
    posts: List[Post]  # Pretend it's defined

Though you'll most likely want to define a SQLAlchemy model, which is very similar but a bit more specific to the SQL language.

[grave jolt]

in Python that would look something like

class User(Table):
    name = StringField()
    age = IntField(default=0)
    password = StringField(redact=True)
    posts = ManyToMany(Post)

or ```py
class User(Table):
name: str
age: int = Default(0) & NonNegative
password: str = Redact
posts: list[Post] = ManyToMany

[elder blade]

Ah, that's not SQLAlchemy right? Just a pretend ORM you could make?

[grave jolt]

yeah

[livid mason]

my debate is not whether Python has an ORM like elixir, this was what I meant, by "DSL".
https://hexdocs.pm/ecto/Ecto.Query.html
the framework declares special keywords to work with.
I can sort of figure out that Python doesn't support this, am I right?

[elder blade]

Yeah you can't do that, though there's some cool stuff you can do by overloading the operators

[grave jolt]

if they're definable dynamically, they're not keywords, are they

[elder blade]

Take NumPy as an example

[grave jolt]

it's just a syntax for calling a macro, correct?

[livid mason]

yeah, ig

[grave jolt]

Python doesn't have macros, so if by DSLs you mean macros, then Python doesn't support DSLs

[livid mason]

I should have rephrased my question lol, I was stupid. Thanks for the reply!

[grave jolt]

There are tools to implement macros (like https://github.com/lihaoyi/macropy), they're just not widely used

(and macropy doesn't allow adding new statements/expressions)

[quiet crane]

What's the way to correctly memoize class methods?

When we lru_cache class methods we will use self as part of the cache key, right? This adds a reference to the instance that will likely live for the entire script/program. Any program that creates and destroys lots of objects with lru_cache methods will "leak" a lot of memory this way.

How could I profile this?

[livid mason]

hmm, sorry if I'm just askin a load of questions, but I think that this one belongs here.
Python has this standard library called ConfigParser. I am thinking of using it for my framework. Essentially, I want the end users of the framework to configure their application via a configuration file
(like django users do with settings.py).

is an .ini configuration file good enough for the case (they must be able to reference env variables inside it). If not, does Python support other formats?

I find that ini files have a few limitations - no support for arrays or a few other commonly used datatypes

[gleaming rover]

YAML or JSON?

actually, why not .py

[deft pagoda]

cache might use weak_refs, i don't know, if it doesn't you can always make your own that uses weak_refs

[livid mason]

maybe because configuration files are made for the job and there should be a clean split between config and the app?

ruby on rails uses yaml files mostly

[flat gazelle]

I would go with a .py, since it gives you a consistent way to find it (python imports), allows all the flexibility the user needs, as well as allowing them to write their own .env file and handle it as they want. Just using env vars as config also works reasonably well, though you have to make sure you allow a way to load a .env file or a similar tool.

json is definitely the wrong thing, but if you want to reimplement envsubst for yaml, feel free to do so.

just hooking into pyproject.toml is also an excellent option, but afaik doesn't allow env vars inside it

[livid mason]

what I have in mind right now is to make the framework flexible. At some point, users may call a method to load configurations values from an object - it might be a python file, a dict or whatever they want.
This way, the users can decide how they handle configuration, like Flask

[flat gazelle]

ye, that's probably the simplest, though if your framework needs a lot of config, it could lead to boilerplate

[livid mason]

default_settings = {
    "server": {"debug": "False", "allowed_hosts": [], "secret_key": None},
    "schema": {
        "schema_url": "/graphql",
        "root_value": None,
        "auto_camel_case": True,
        "error_formatter": "graphql.error.format_error",
        "response_encoder": "canopus.core.encoders.default_encoder",
        "max_errors": 20,
        "middleware": [],
        "validation_rules": [],
        "batch_enabled": False,
        "batch_url": "/graphql/batch",
    },
    "graphiql": {
        "enabled": False,
        "version": "1.4.2",
        "template": "graphiql_jinja2_template",
        "title": "GraphiQL Explorer",
        "pretty": True,
        "default_query": None,
    },
}

This is all the settings we need roughly now

[flat gazelle]

yeah, this merits a yaml file with a dsl for environment substitution, that is more than anyone wants to write by hand.

[livid mason]

do you mean that a yaml file is more suitable?

[flat gazelle]

ye. Toml could also work quite well, but if you will want more nesting than this, it isn't great

[gleaming rover]

homoiconicity 🙏

[quiet crane]

oh ok, thats cool, thanks! I guess https://docs.python.org/3/library/weakref.html is a good resource to read up on those.

[paper echo]

S expressions!

[dry finch]

are there any mock interviews based on python going on in the server?

[cyan scarab]

Guys i wanted to do freelancing on Fiverr but i am afraid i going to make gig then i return i can't understand what to do my fear is what if i can't do my client work ?😭😭

[jovial flame]

!rule 9 first off, but also if you don't feel confident that you can deliver what you are being asked (and paid) to do, then maybe don't do that

[fallen slate]

Rules

9. Do not offer or ask for paid work of any kind.

[uneven fable]

That doesn't look like a rule 9 violation to me..

Just a "wrong channel" violation and a "what are you even asking" violation

[main lynx]

a stop on my quest for safe and performant python is validating freedom from garbage collection overhead, humbly seeking feedback on this tool
https://github.com/mvollrath/pytest-nogarbage

[nova iris]

starred_item ::=  assignment_expression | "*" or_expr
```lmao i'm finding another fun consequence of python's grammar... so this is completely valid (since it's an `or_expr`):```py
[*(1, 2, 3, 4) + (5, 6, 7, 8)]
```but this is not (`comparison` has lower priority than `or_expr`):```py
[*Foo() < None]
```where `Foo` defines `__lt__` to return `(1, 2, 3)`

[verbal escarp]

nobody writes code like that :p

[nova iris]

basically, the asterisk unpacking in container displays "binds less tight" than all arithmetic and bitwise operations

but it's not really binding, it's not even an operator lmao

[nova iris]

you never know #esoteric-python ™️

[verbal escarp]

that's #esoteric-python, not #internals-and-peps :p

[nova iris]

true lol, i just wanted to share my discovery since it's quite odd this is valid

[verbal escarp]

with code like [*Foo() < None] you can't expect anything but odd

it's not at all clear what this should be doing

[nova iris]

oh, of course, but lambda expressions and conditional expressions are also up there, unbindable by *

[verbal escarp]

in most cases it should fail horribly

[nova iris]

which is odd because in a function call, that's permitted

positional_item ::=  assignment_expression | "*" expression
``` @verbal escarp

[verbal escarp]

so?

[nova iris]

>>> (lambda*args:"yeet")(*Foo() < None)
'yeet'

[halcyon trail]

oof

[nova iris]

but that's not the case with other uses of *, which is weird

idk, i'm just finding these subtle details interesting

[verbal escarp]

it's a function that takes *args, you give it *some_args and it always returns "yeet"

what's odd about that?

[nova iris]

here, * can bind Foo() < None without the help of parentheses

but in a container display, it can't

[verbal escarp]

show your Foo class

[nova iris]

class Foo:
    def __lt__(self, other):
        return (1, 2, 3)
```a very simple one lol

[grave jolt]

re #mailing-lists
I think I agree with the premise of https://mail.python.org/archives/list/[e-mail]/thread/JWLKBT2YYDGFS76Z37FZJNZPEDVXOLCW/.
I don't think Python would even lose anything without the is operator.

[verbal escarp]

i'd rather get rid of id if anything

[halcyon trail]

the premise of the article isn't about getting rid of is, the premise is just that it shouldn't be a PEP8 requirement basically to do x is None

[grave jolt]

Yes, I understand

[halcyon trail]

I know, I was telling amogorkon

[grave jolt]

that was more of a comment

[halcyon trail]

sorry

[nova iris]

why? it's so useful to demonstrate identity vs equality and teach it to other people

[grave jolt]

move it to sys

[verbal escarp]

yeah, exactly

if you need the address, just import sys.mem_id or somesuch

[halcyon trail]

Okay, "get rid of" is different from "move it into a module"

[verbal escarp]

i mean "get rid of" as a keyword

[halcyon trail]

it's not a keyword, it's a built-in

[verbal escarp]

sorry, that

[grave jolt]

ctypes.addressof already exists, so nvm, it has a different interface

[halcyon trail]

I agree that id doesn't merit being a built-in overall

but that's pretty much unrelated to the point of the article. Which is mostly that there's no real reason to force beginners to use is

[nova iris]

isn't is used for convention on singletons to emphasize that they're singletons?

[grave jolt]

I haven't seen any other language "emphasize" the fact that singletons are singletons

[verbal escarp]

it has nothing to do with singleton or not :p

[peak spoke]

it's nice to ensure some random object doesn't mess with your check, but it's not like that's necessary and otherwise it's the same thing as == for the cases where it's usually used

[nova iris]

then why does it exist as a pep 8 rule? i'm actually confused now-

[nova iris]

i always thought that we use is because there can only be one object of NoneType, so identity is quicker

[verbal escarp]

that's not why you check None with is

if you check implicitely like if not foo and assume it only can be None, you're missing other empty cases like [] or ""

which can easily backfire

[grave jolt]

I don't think anyone is proposing that here

[nova iris]

oh, we use is to bypass the __eq__, which can be overloaded? is that why?

[verbal escarp]

you could say that

[nova iris]

but then you can always just do if foo == None instead of if foo is None

[verbal escarp]

that won't work on things that don't have __eq__

[charred pilot]

it would default to an identity check then

[raven ridge]

Everything has __eq__ - it's inherited from object

[nova iris]

and iirc __ne__ is inherited from object as well, might be wrong though

[verbal escarp]

true, but not necessarily defined for None

well defined*

[nova iris]

del Foo.__eq__ \s

actually, i believe the actual way to suppress automatic dunder support from object is to set it to None, e.g. __eq__ = None in class body

yeah, i read it in the docs somewhere

[verbal escarp]

i think is is as fundamental in the language as and and or

[peak spoke]

The only place I know of where that properly suppresses something is hash, otherwise it's just an error from the attempted call

[main lynx]

is is correct for comparing to None, otherwise:
!e

import dis

def truthiness(a):
    return not a

def equality(a):
    return a == None

def identity(a):
    return a is None

dis.dis(truthiness)
dis.dis(equality)
dis.dis(identity)

[verbal escarp]

on the other hand, afaik is relies on id, so question is whether it's possible to import sys.mem_address or somesuch without id in the first place

[elder blade]

!e ```py
import dis

def truthiness(a):
return not a

def equality(a):
return a == None

def identity(a):
return a is None

dis.dis(truthiness)
dis.dis(equality)
dis.dis(identity)

[fallen slate]

@elder blade :white_check_mark: Your eval job has completed with return code 0.

001 |   4           0 LOAD_FAST                0 (a)
002 |               2 UNARY_NOT
003 |               4 RETURN_VALUE
004 |   7           0 LOAD_FAST                0 (a)
005 |               2 LOAD_CONST               0 (None)
006 |               4 COMPARE_OP               2 (==)
007 |               6 RETURN_VALUE
008 |  10           0 LOAD_FAST                0 (a)
009 |               2 LOAD_CONST               0 (None)
010 |               4 IS_OP                    0
011 |               6 RETURN_VALUE

[elder blade]

There ^

[nova iris]

hm, true, suppressing __hash__ actually gives the correct TypeError when hash() is attempted on it or if it's given as a key to a dict/a member to a set

[unkempt rock]

the only singleton objects are True False and None right?

[nova iris]

look the same to me lmao

it's just that IS_OP is used instead

[verbal escarp]

no

there's also ... and others

[nova iris]

also NotImplemented and Ellipsis

[grave jolt]

Define "singleton"

[peak spoke]

is is same as comparing ids, but at least in cpython id is not used directly by the operator

[verbal escarp]

what's used instead?

[grave jolt]

If it means a type that only has one value, then there will be as many as you wish

[unkempt rock]

i noticed None only has one id even when assigned to other things

[nova iris]

well, they probably meant built-in types lol

sure, you can make as many singleton types as you want with a bit of metaclassing

[peak spoke]

it just compares the pointers https://github.com/python/cpython/blob/9827710a400848c9430ed364ed5d2d54f0926701/Python/ceval.c#L3017-L3021

[fallen slate]

Python/ceval.c lines 3017 to 3021

case TARGET(IS_OP): {
    PyObject *right = POP();
    PyObject *left = TOP();
    int res = (left == right)^oparg;
    PyObject *b = res ? Py_True : Py_False;```

[halcyon trail]

Just for some comparison, in Kotlin when you have a nullable type you still compare to null with ==

[verbal escarp]

i think i've seen id used in internal caching of imports etc, but i'm pretty sure that could be circumvented

[halcyon trail]

I tend to agree with the article overall, there isn't much reason for beginners to use is. In fact, outside of the comparison to None case, I don't use is much at all.

[verbal escarp]

me neither, yeah

[main ginkgo]

id corresponding to memory address is just an implementation detail of cpython.

this might not always be the case in all implementations

[peak spoke]

There are uses for it, but if it were the other way around and the operator was being proposed as a new thing I don't think the uses would warrant it over comparing identity through id

[verbal escarp]

now i'm really wondering if foo == None would be such a bad thing, actually

[halcyon trail]

@main ginkgo no, but I expect that id is required to match up with is

[main ginkgo]

no to what exactly?

[halcyon trail]

that is, id(a) == id(b) iff a is b

[raven ridge]

Yep.

[halcyon trail]

No; agreeing with you that id doesn't have to be the memory address

I agree with you, it doesn't have to be the address in other implementations

[acoustic crater]

what's the time complexity of locals() in a module/instance vs somewhere without a __dict__?

[halcyon trail]

but it still, I believe, has the constraint I posted above

[acoustic crater]

I'm guessing O(1) vs O(n)?

[halcyon trail]

Python has a place without a __dict__ ?

[verbal escarp]

slots

[acoustic crater]

new style classes have proxies instead tho I guess that's still O(1), also comprehensions

[halcyon trail]

but those cannot be locals

[nova iris]

LMFAO WAIT SO THIS IS THE ORIGIN

[main ginkgo]

dont think this is up for debate. pretty sure language specs say is is an identity check so by default the IDs would have to match

[nova iris]

__peg_parser__ for the win

[verbal escarp]

sounds about right ^^

[halcyon trail]

Now I'm confused, I don't know if you are agreeing or disagreeing 🙂

[nova iris]

NOOOOOOOOOOOOOOOOO

[main ginkgo]

you're saying in your opinion a is b should mean that id(a) == id(b) right? im just saying its not a matter of opinion, this is already decided by the language spec

[dim orbit]

Why does the console tell me this? (I just started learning python xd)

[acoustic crater]

is behavior can't be effected by a dunder right?

[raven ridge]

You've both been talking about the language spec, and have been agreeing with each other.

[acoustic crater]

so using is with None ensures __eq__ isn't messing with your results

[verbal escarp]

yeah

[peak spoke]

yeah but if that matters then you can do it through id like above

[verbal escarp]

that pretty much sums up my reluctance to rely on == solely

[dim orbit]

Help pls I am a noob in python

[acoustic crater]

you can also write a for loop with a range instead of just iterating through a list, why manually use indices/ids if you don't have to?

[verbal escarp]

how to give people the "how to get help" thing?

[peak spoke]

because it's one more operator in the language and causes a bit of confusion to beginners

[acoustic crater]

it's too late to remove operators though and I'm not in the "anything confusing to beginners is bad" camp

[verbal escarp]

i'm all for consolidating superfluous fluff myself, but it's hard to tell in this case whether it's fundamental or not, required for bootstrapping the rest of the language

[peak spoke]

I'd say that at least in 90% of cases a direct == achieves the same thing in a more roundabout way, with the only concern being types that overload == and don't return NotImplemented/False for other types and things like True == 1

[acoustic crater]

I use is enough that I think it's warranted, id is the weird one I never use and often override

I only use id if I want to do stupid ctypes tricks

[verbal escarp]

i really hate shadowing builtins 😦

[acoustic crater]

id is the only one I'll shadow cuz it's generally useless

[verbal escarp]

although i'm often tempted

[peak spoke]

meh if it's in a normally sized scope who cares on a not so often used builtin

[verbal escarp]

not so long ago i accidentally shadowed a builtin (can't recall which) locally but it leaked out of scope and i ended up with very confusing bug

[acoustic crater]

yeah I would only shadow id because I have never used it when I'm not doing something completely inadvisable

like bypassing a dict proxy or messing with built-ins' structs

[halcyon trail]

I was saying I thought the spec required that. Also, no, I wrote "iff"

not "if"

So that also means, more significantly, that id(a) == id(b) implies a is b

that's the direction that's less obvious

That's definitely true in cpython, I don't know that python has a formal spec which says that's required, but it seems like it should be

[hot token]

sort of

[verbal escarp]

sadly there's no formal spec

[halcyon trail]

Right, that's the other thing

[hot token]

The docs say

Return the “identity” of an object. This is an integer which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value

notable is that the id is guaranteed to be unique, which would mostly imply that id(a) == id(b) ==> a is b

[halcyon trail]

Right

[hot token]

but its only unique during the lifetime of the object

[halcyon trail]

Right, though we were talking about comparing a and b so I think we can take that as given that a and b were overlapping

[hot token]

yeah

[acoustic crater]

Return the “identity” of an object. This is an integer which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value

Doesn't this mean that in id(a) == id(b) a's return value's lifetime could end after its id is taken but before the id of b is taken and, if the object returned by b is created when it's accessed, it could end up the same as id returned by a?

idk if is is guaranteed to have different behavior

[hot token]

well, in order to do a is b both a and b are live objects

simultaneously

[acoustic crater]

but if you are comparing id ints then the place those ints came from could be GCed

[hot token]

yep

[nova iris]

evil unicode hack

[verbal escarp]

meh.

abusing unicode to obfuscate code is just meh

[nova iris]

i'm not obfuscating, i saw that python uses nfkc on idents before parsing so i thought it'd be cool to experiment with some stuff

[raven ridge]

If you do id(a) == id(b), it's guaranteed to behave the same as a is b, as long as no one redefined id. But the same is not true for id(a()) == id(b()). In the former case, both objects are guaranteed to live for the entire expression. In the latter case, there are temporary objects involved that can die in the middle.

[acoustic crater]

also dotted access can be a call of a property right?

[verbal escarp]

yup

[acoustic crater]

but yeah is seems justified if only for comparing returns of calls

[halcyon trail]

well, the thing is that comparing identity is very rarely justified in the first place

[acoustic crater]

I use it for None and enum members usually

pretty often checking if the return of a call is None

[halcyon trail]

in both those cases equality either works as well or could be made to work as well

[acoustic crater]

but an object can pretend to be those things with equality

[halcyon trail]

It can, but if people do horrific things, they do horrific things.

"we're all consenting adults" etc

[acoustic crater]

but we still use non-public methods and other conventions that make it easier to avoid breaking things

[halcyon trail]

Using non-public methods is a way to very easily tell people this isn't part of the API, if you don't do that then sane, reasonable, good programmers can easily assume it's part of the API and use it

sane reasonable good programmers are not typically going to be writing == for their class so that an instance of their class is equal to None

[verbal escarp]

actually

[halcyon trail]

realistically it's been like this in python for some time, I dont' expect it to change, and is is concise enough anyway. Mostly I just think that identity comparison isn't important enough or useful enough to use a keyword as awesome as is on

[verbal escarp]

how would you implement __eq__to None without is?

[halcyon trail]

I mean a lot of objects would check the type of the other argument, and if it's not the same, immediately return False

you don't need to implement eq to None specifically

[acoustic crater]

they might just make it return True if compared to falsy values for some reason

for example

[flat gazelle]

There are some realworld cases where it matters, for example a potentially None numpy array

But yeah, not a fan of it being a keyword

[halcyon trail]

that's not how truthiness/falseiness should work in python though

I mean yes, they coudl do that, but in 99.9% of cases, that would be bad code. I'm open to the existence of this remaining 0.1%

[acoustic crater]

I think is helps beginners with quickly demonstrating things like copying and deep copying mutable objects

having to check the id of objects adds another step to experimenting

I think I most often use is in tests and experiments

[main lynx]

we still talking about comparing to None? not only is is correct, it's also faster than ==

[flat gazelle]

So why not make it a function in some module

Performance is irrelevant

[main lynx]

i've yet to hear a reason to use ==

[flat gazelle]

The issue is that is isn't better enough to warrant a separate confusing keyword

Especially considering how Cpython Caches and reuses instances

[acoustic crater]

there are a lot of built in functions that are purely for testing and investigation I don't think importing basic testing tools is pythonic

why not make assert a function in some module?

[main lynx]

what is the alternative to having is in the language?

[flat gazelle]

Not having is and getting sys.same or sth

[halcyon trail]

is and id could easily be buried in some module

[acoustic crater]

chilaxan just showed me this in #esoteric-python

!e

print(id([0]) == id([1]))

[fallen slate]

@acoustic crater :white_check_mark: Your eval job has completed with return code 0.

True

[acoustic crater]

hehe

[halcyon trail]

Kotlin for instance uses is for something more like python's isinstance, except that the is operator also does smart casting and such

[acoustic crater]

I thought so

[halcyon trail]

I think that's the best use of is

Something like isinstance/pattern matching

[flat gazelle]

@acoustic crater assert will not evaluate it's right operand if it passes, so it must be a keyword

[verbal escarp]

because assert shouldn't be a function, it shouldn't be considered part of the code-flow

[unkempt rock]

is this because it's the id of the list object?

[acoustic crater]

yeah

[halcyon trail]

Outside of is None, I write isinstance at least 10x as much as is

[flat gazelle]

It's because Cpython reuses small lists

[acoustic crater]

yepyep

[halcyon trail]

The reason to use == is just obviously for consistency

and freeing up a language keyword

well, obviously that can't happen in python per se, but in a hypothetical language

[acoustic crater]

is as it currently is makes sense and already has modular behavior with is not so if is is to be used in another context it makes sense to add another modular add-on to it

why would is mean anything else alone than... is?

[halcyon trail]

what do you mean by "modular behavior" ?

[acoustic crater]

is not is more than the sum of its parts

[verbal escarp]

it could happen, it already happened with the functional stuff 😦

[main lynx]

here's why assert should be a keyword

[halcyon trail]

and people have to learn that is and is not are a thing, I definitely wrote not a is None until I was corrected

how is it more than the sum of its parts?

[main lynx]

!e

import dis

def make_assert():
    assert False

dis.dis(make_assert)

[fallen slate]

@main lynx :white_check_mark: Your eval job has completed with return code 0.

001 |   4           0 LOAD_CONST               1 (False)
002 |               2 POP_JUMP_IF_TRUE         8
003 |               4 LOAD_ASSERTION_ERROR
004 |               6 RAISE_VARARGS            1
005 |         >>    8 LOAD_CONST               0 (None)
006 |              10 RETURN_VALUE

[main lynx]

^ compare to output with python -O

[acoustic crater]

because not object normally is just False

[halcyon trail]

when you use ==, everyone already also knows !=. So they don't need to learn that is not is also valid.

[acoustic crater]

but is not object is different from is False

[verbal escarp]

exactly, assert is special enough to warrant its own keyword

[halcyon trail]

okay, in that sense, sure, but all you're proving is that is adds yet more behavior for people to memorize 🙂

it's not terrible or anything. It just doesn't provide any value, it's a bit more to learn, and it wastes a keyword.

[charred pilot]

and is not is unintuitive for non-english speakers

[halcyon trail]

None of those are the end of the world but they're definitely not ideal.

[acoustic crater]

I imagine all of Python is pretty unintuitive to non English speakers since it tends to follow English pretty closely

[spark magnet]

it'd be cool if we could import keywords: if mything sys.is None:

[halcyon trail]

yeah, for better and worse most languages use english keywords and english sensibilities are a factor

[verbal escarp]

sys.is xD

[charred pilot]

like kotlin's infix functions, would be interesting

[halcyon trail]

Yeah, infix functions are nice IMHO.

[verbal escarp]

how would that work in python?

infix functions?

[halcyon trail]

in python it probably wouldn't

[acoustic crater]

infix functions would be a million times more confusing to beginners than is haha

[halcyon trail]

yeah, I agree, they are something that's used much more sparingly

[charred pilot]

not any more confusing than realizing that + calls a function though

[spark magnet]

python's special rule here is that the code has to compile without reading any info from the imported modules. so it's kind of a non-starter.

[acoustic crater]

there's a clear seperation between characters used for operators and characters used for names

in python

...except keywords

[halcyon trail]

sure, infix isn't a good fit for python. similarly it's probably not ever going to change that people write x is None in python.

But I don't think you'll see many new languages using is that way. It's just rather a waste.

=== seems to be a common choice

[charred pilot]

where's === used outside of js?

[spark magnet]

but we can all agree that id() should be sys.id(), right!? 🙂

[halcyon trail]

@charred pilot kotlin, swift

[raven ridge]

I think it shouldn't exist at all. It's onerous for alternative implementations that don't just return a pointer to the object, AFAIU

[halcyon trail]

that said, in Kotlin I've practically never seen it

it's really just almost never a good idea to comapre object identities. I can't remember the last time I did it.

All the more reason to make comparison to null use the common syntax, and keep the syntax for identity out of your way 99% of the time

[nova iris]

i feel like it could be a cpython-specific thing, then

(or any other implementations that support it)

[visual shadow]

i mean, i like id and it has come in clutch once or twice, so i won't go as far as to say that it shouldn't exist, but i've literally only used it mostly just for experimentation.

so im with ned that it shouldn't be in the global namespace, but i'd like it to stay

[raven ridge]

It's basically useless. It's harder to use correctly than is, and most of the legitimate uses for it were subsumed by weak references once they were introduced

[visual shadow]

what's a weak reference?

[raven ridge]

!d weakref

[fallen slate]

weakref

Source code: Lib/weakref.py

The weakref module allows the Python programmer to create weak references to objects.

In the following, the term referent means the object which is referred to by a weak reference.

A weak reference to an object is not enough to keep the object alive: when the only remaining references to a referent are weak references, garbage collection is free to destroy the referent and reuse its memory for something else. However, until the object is actually destroyed the weak reference may return the object even if there are no strong references to it.

[raven ridge]

It's a way to reference an object without keeping it alive just because you're referencing it. When you later want to do something with it, you need to check if it's still alive, and then do stuff with it only if so.

[visual shadow]

i see, in that case could you elaborate on what the usecase was for id before this existed? im curious

[verbal escarp]

well, with id in sys.id, we could get rid of is by just comparing sys.id(self) == sys.id(other) in __eq__ if necessary

[verbal escarp]

i think the primary use is for caching

[raven ridge]

Right, saving ancillary data about some object in a separate structure without keeping that object alive. You could use its id() as the lookup key

[verbal escarp]

weakref is pretty difficult to work with, easy to fall in refcounting traps

[raven ridge]

True, but using id() where you're trying to weakly reference something is strictly harder to get right 🙂

[verbal escarp]

agreed

[visual shadow]

in my case, i used id to make a mapping, with this "ancillary data" as you phrased it, but my need was that i was working with the api of a library that had these custom unhashable objects but i needed this information fed to them in one of their functions that was essentially like a single arg mapper.

perhaps there was a different way to solve it, i am not sure, but id was the most straight forward way to work around that library's limitations

[elder blade]

I'd say that would be an edge-case where you'd simply do fine importing the function to do this

[verbal escarp]

i used uuid for that purpose once

[elder blade]

The worst part is that id() means I can't name my variables id (which is very common so I do have to shadow the built-in sometimes) ;-;

[main lynx]

yes you can >:D

[elder blade]

It's kind of like file, does anybody know what that actually is?

I just see the syntax highlighting and change my variable named just because it's an abnormal color

Lmao

[visual shadow]

yeah, press f to pay respects for file 😉

[visual shadow]

yep, that would have done the trick too, i should keep uuid in mind

[visual shadow]

wait... i just checked this, i get nada for file

[halcyon trail]

file doesn't seem to be a built-in

it's a "magic" function in ipython

[verbal escarp]

file
Traceback (most recent call last):
File "<pyshell>", line 1, in <module>
NameError: name 'file' is not defined

[elder blade]

Oooh

[halcyon trail]

but not anything in python itself. Though I found an SO asking about this, so maybe it used to be?

[charred pilot]

what editor?

[elder blade]

Visual Studio Code

[charred pilot]

that's your problem

[halcyon trail]

I win!

file() was deprecated in 2.x then flat out removed in 3.x, so IMHO it's fair game –

[verbal escarp]

vscode doesn't highlight "file" for me

[charred pilot]

is that just like open but renamed?

[halcyon trail]

Now somebody with more patience than me can google to figure out what file did in python 2.*

seems like it was indeed the same as open

[elder blade]

Beuh

[halcyon trail]

I think that open may have returned an instanc eo f aclass called file, so file was like the constructor for it perhaps?

idk

[visual shadow]

in Python 2, open and file are mostly equivalent. file is the type and open is a function with a slightly friendlier name; both take the same arguments and do the same thing when called, but calling file to create files is discouraged and trying to do type checks with isinstance(thing, open) doesn't work.

In Python 3, the file implementation in the io module is the default, and the file type in the builtin namespace is gone. open still works, and is what you should use.
Credit link

[verbal escarp]

another case of removing superfluous keywords or builtins..

[charred pilot]

To check if an object is a file, as in isinstance(f, file).

[verbal escarp]

we should do that more often!

😄

[elder blade]

I believe it gave me a weird blue name

[elder blade]

All hail removing built-in stuff

[visual shadow]

... i have no idea how to link hyperlinks in discord 😦

[halcyon trail]

are you using python 2

[elder blade]

Can't, only webhooks and embed can

[halcyon trail]

we don't take kindly to python-2-nistas 'round these here parts

(kidding kidding, if it wasn't already obvious)

[visual shadow]

oh.

[verbal escarp]

wait, what? i can in my win app

[elder blade]

No, but I believe the like parser/syntax understandable-thingy-majick-ish has some legacy support

[verbal escarp]

#internals-and-peps message

[visual shadow]

no bluenix understood me. (i was looking for markdown style hyperlinks with <text here> that can be clicked)

[verbal escarp]

ohh

i see

[elder blade]

Right, but you can't hyperlink and hide the link under text

[charred pilot]

lul

[visual shadow]

safe.

[verbal escarp]

thanks 😉

[elder blade]

!decorators

[fallen slate]

Decorators

A decorator is a function that modifies another function.

Consider the following example of a timer decorator:

>>> import time
>>> def timer(f):
...     def inner(*args, **kwargs):
...         start = time.time()
...         result = f(*args, **kwargs)
...         print('Time elapsed:', time.time() - start)
...         return result
...     return inner
...
>>> @timer
... def slow(delay=1):
...     time.sleep(delay)
...     return 'Finished!'
...
>>> print(slow())
Time elapsed: 1.0011568069458008
Finished!
>>> print(slow(3))
Time elapsed: 3.000307321548462
Finished!

More information:
• Corey Schafer's video on decorators
• Real python article

[elder blade]

Here it's done at the bottom in embeds

We may be getting a bit carried away though

Ugh- should hyperlink become a new built-in 😳

[verbal escarp]

definitely.

better yet: url

[elder blade]

That could work like open yeah?

[verbal escarp]

on a more serious note: we should have an urllib that behaves similarly to pathlib and have a URL object that implements things like yarl

[charred pilot]

python replaces html 😳

[elder blade]

Oh man that would be nice. Pathlib does things very nicely

It could somewhat easily do things against URL injection too

Since the URL object would have control over creating the string

[verbal escarp]

i really dislike having to import a third party module for something basic like a URL object

that doesn't suck*

[halcyon trail]

what do you mean it has legacy support?

[elder blade]

Support for Python 2

[halcyon trail]

what does, exactly?

[elder blade]

https://github.com/MagicStack/MagicPython

[halcyon trail]

ah

[candid pelican]

i made this article:
https://github.com/lipe14-ops/python-tips/blob/main/README.md

hehe

give me a feedback pls

[spark magnet]

run a spell checker on it 🙂

[verbal escarp]

name = [letter for letter in 'Amanda'] meh.. i almost never use verbose vars for iteration if it's obvious what things are

if the var is only used in a single place, there's very little point

for number_of_iterations, thing in enumerate(things):...

for i, thing in enumerate(things): ...

[charred pilot]

especially when i is easily understood

[verbal escarp]

or for k, v in dictionary.items(): ... is often just as obvious

[charred pilot]

most of these are just formatting nitpicks, could be summed up as "follow pep8"

[peak spoke]

If it's not completely obvious I prefer to use something that describes the value at least a bit

I mostly don't like i as a name that describes what index it is of can be used but I agree that it's unnecessary in most situations

[verbal escarp]

if i isn't obvious, next best thing is idx

[peak spoke]

And overusing list comps is a common error of beginners leading to an unreadable mess of a comprehension, so I wouldn't bring that up as a point. (even the example should be list('Amanda'))

[charred pilot]

singleton as a recommended design pattern,

[verbal escarp]

add = lambda number_1, number_2: number_1 + number_2oh please, no!

lambdas are NOT for "small functions"

they are for anonymous functions

[charred pilot]

for tip 11, sometimes you just have to return None, sometimes you don't have anything else to return. and they don't necessarily return anything in the first place

[verbal escarp]

with all the recommendations regarding spaces and tabs - just use black 😉

_protected and __private is so wrong

feels like java

jeny = { 'name':'Jeny','age':17 } # wrong
jeny = {'name': 'Jeny',' age': 17} # right <- i don't think that space is right 😉

[candid pelican]

ic

[verbal escarp]

b = 0

a, b = b, a``` - fun tidbit, never needed it 😉

[candid pelican]

but cannot be???

[verbal escarp]

hu?

def add(a:int, b:int) -> int: return a + b

less chars even with annotations, less verbose and IDE-friendlier

i had a hard time at first to even find that : in the lambda

[cloud compass]

for number 15, i would reword it. you make it sound like return is just a different way to use global variables. instead, i recommend walking through an example of how to return values and use them in different scopes

also, you tend to say things like “use list comprehensions, use ternary operators, etc”. this implies that they should be used whenever possible, which is not actually the case. i would recommend rewording these parts

[candid pelican]

ic

[verbal escarp]

also, why just list comprehensions

there are also set, dict and generator comprehensions that aren't any less important

if you want to promote a functional style (which all those tips are hinting at) you should also mention func- and itertools

depending on the audience, of course

btw, i would also recommend using jupyter notebooks for beginners to keep a kind of field diary

[candid pelican]

bro thank you lol

rlly

helps me a lot

[charred pilot]

and 12 is just kinda weird. ternaries are rarely used, but it sounds like you're recommending to use them all the time

[candid pelican]

i reworded it.....

but i meant do not forget that it exists

[verbal escarp]

hmm.. by now i'm using ternaries quite frequently, mostly at the head, sanitizing input etc

[charred pilot]

certainly not in place of all if statements though

[candid pelican]

11 i can cut the else statement off

[main lynx]

a thing I don't like about casual ternary use: it can mask uncovered branches in coverage report

[candid pelican]

i don't like ternary to much
they are ugly

imao

[charred pilot]

you put it in your tips 🤔

[candid pelican]

bc they exists

im ugly but theres a mirror in my room

almost the same thing

[charred pilot]

no the same thing would be going to a fashion event thing and showing yourself off

[candid pelican]

ye

lol

[charred pilot]

if you don't want people to use it, just don't tell them about it

[main lynx]

quality by obscurity

[verbal escarp]

lmao

nice

same argument goes for singletons 😉

[main lynx]

don't tell them about the walrus either

[verbal escarp]

but do tell them about __methods__

[candid pelican]

i used it once lol

gosh wait

[charred pilot]

there's no reason to use it there though, it just makes things less readable

[verbal escarp]

it's not totally bad though either, imo

[charred pilot]

that line has to be like 100 chars, at least

[verbal escarp]

import pygame as pg ^^

[candid pelican]

lol

i short way that i found to make it

[verbal escarp]

i just found another potential usecase for AST fiddling

def foo(*, a, b, c):
    print(a,b,c)

a,b,c = 1,2,3
foo(a,b,c)

a function that only takes kwargs could take vars matching those args

[surreal sun]

This might be a stupid question but how do I get a class to fall back onto a metaclass for dunder methods

Like I want a class to fall back to the metaclass' __eq__ method

[dusty verge]

Just set the classes parent to the metaclass?

[pliant tusk]

the methods of a metaclass are inherited by the resulting class, not the instances of that class

[dusty verge]

Hmm, correct me if I am wrong, but I don't know if this is true

(Oh wait wait nvm, don't listen to me)

[surreal sun]

object has __eq__ defined iirc so that might be why when it removes the metaclass implementation with the default one

I just am confused as to how I have the metaclass keep its dunder methods

[pliant tusk]

!e ```py
class Meta(type):
def eq(self, arg):
print('Meta EQ')
return 'Meta'

class Foo(metaclass=Meta):pass

print(Foo == 1)
print(Foo()==1)```

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

001 | Meta EQ
002 | Meta
003 | False

[pliant tusk]

Foo()==1 is False because it calls the default Foo.__base__.__eq__

[surreal sun]

aH

so I should just do a parent class instead of a meta class

for dunder methods

hm

[pliant tusk]

yes

[raven ridge]

Or have the metaclass patch methods into the class after creating it

[surreal sun]

like have the result of the __new__ be stored as a class variable that represents the class instance, then set it like if class defines the class created:
class.__dict__['__eq__'] = lambda self, other: self.value == other.value or something like that

[raven ridge]

You don't need __dict__, you should just be able to do cls.__eq__ = some_func. But yeah

[elder blade]

JS has that with their objects. I find it unintuitive in Python

[elder blade]

Pretty sure you can't, you need to use the metaclass to define an __eq__ on the classes

[visual shadow]

In this case I'd probably say inheritance with a normal parent class makes more sense .

[fringe sundial]

Is 1/x=x^-1?

[visual shadow]

Yes, also you can use the normal chat channels for those questions.

[fringe sundial]

Ah thx

[surreal sun]

Yeah, that’s what I’m doing right now

[swift imp]

The class will fall back to what you define in metaclass. The instances of the class however will not and will fall back onto what's defined on the class

[elder blade]

That's interesting, nice

[tribal dirge]

Hi

Guys, what're we talking about pls ?

[boreal umbra]

This channel is for talking about Python itself.

For example, during the code jam, I reviewed a game where they used custom exceptions to broadcast the completion of a level (like raise LevelComplete to go all the way back to whatever part of the program handles directing the user between parts of the game). Are there other times when exceptions are used to broadcast a message that isn't "something's not right"?

[main lynx]

that's not how you should use exceptions

[boreal umbra]

why not?

[peak spoke]

Python itself uses StopIteration, but it's not really best design to use exceptions for flow control like that

[main lynx]

because it's a hilariously expensive way to return values

and that's really what you're doing, returning a value up the stack

[grave jolt]

it's more of a goto

[grave jolt]

I assume what the game really needed was an event system

[main lynx]

correction: if you have to ask, that's not how you should use exceptions 😉

game jam is a perfectly good place to be clever

[boreal umbra]

I've never worked on a project that's constantly reacting to the user's behavior, so I wouldn't know what atypical Python usage is considered acceptable in that space.

Why are exceptions more expensive than returning?

[main lynx]

imagine returning a value, but before you return it you pack a bunch of debugging information into it, which you then discard because you didn't catch the exception to debug it

the time overhead is really not terrible

def wrap_return():
    def do_work():
        return
    do_work()

def wrap_except():
    def do_work():
        raise RuntimeError("done")
    try:
        do_work()
    except RuntimeError:
        pass

$ python3 -m timeit -s 'from exceptflow import wrap_return as f' -- 'f()'
2000000 loops, best of 5: 171 nsec per loop
$ python3 -m timeit -s 'from exceptflow import wrap_except as f' -- 'f()'
500000 loops, best of 5: 404 nsec per loop

!e

import dis

def wrap_return():
    def do_work():
        return
    do_work()

def wrap_except():
    def do_work():
        raise RuntimeError("done")
    try:
        do_work()
    except RuntimeError:
        pass

dis.dis(wrap_return)
dis.dis(wrap_except)

[fallen slate]

@main lynx :white_check_mark: Your eval job has completed with return code 0.

001 |   4           0 LOAD_CONST               1 (<code object do_work at 0x7f2784cf27c0, file "<string>", line 4>)
002 |               2 LOAD_CONST               2 ('wrap_return.<locals>.do_work')
003 |               4 MAKE_FUNCTION            0
004 |               6 STORE_FAST               0 (do_work)
005 | 
006 |   6           8 LOAD_FAST                0 (do_work)
007 |              10 CALL_FUNCTION            0
008 |              12 POP_TOP
009 |              14 LOAD_CONST               0 (None)
010 |              16 RETURN_VALUE
011 | 
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/upocekatis.txt?noredirect

[peak spoke]

it does get a bit worse when it has to propagate the exception further

[main lynx]

makes sense, have to potentially catch it at every turn

deeper stack is when it looks more attractive too, so the more you want it the more it costs

another reason not to use exceptions in this way: everything on the stack in between is interrupted and may not have been expecting a non-fatal exception

normally every time you're in a function you know where the exits are, you can only raise or return

letting fatal exceptions bubble up is typical especially in prototypes, fail early and often

but what if the exception wasn't fatal and you were in the middle of updating some state?

[raven ridge]

Remember that Python itself uses exceptions for flow control - StopIteration, GeneratorExit, etc. The "don't use exceptions for flow control" advice from other languages doesn't really apply to Python.

And, because every for loop relies on exceptions for flow control, that path is relatively optimized.

[deft pagoda]

our favorite asyncio.CancelledError

[elder blade]

But we're talking about winning the game?

It doesn't happen so it really doesn't matter at all.

I think this is great, and for Python this is generally you'll find every here and there

[paper echo]

Just because python does it internally as a hack doesn't make it a good idea in new applications

That said, I think the advice is stated in a misleading way. The real advice should be "don't use non-local control flow (unless you absolutely are very sure that you need to)"

If anything, non-local control flow would be less scary if it didn't involve automatically unwinding the stack and crashing

The Lisp condition system is really interesting in that regard

[nova iris]

"When function decorators were originally debated for inclusion in Python 2.4, class decorators were seen as obscure and unnecessary [1] thanks to metaclasses." -- PEP 3129

THE FKING IRONY

[nova iris]

lol, our code jam entry literally uses an entire exceptions.py as signaling events

there's LevelSelector exception and NextLevel exception and stuff like that, iirc

[elder blade]

I think you should try your best not to use it

Because sometimes it should be as simple as setting a bool, or just returning an Enum.

But sometimes it does really make sense, so I don't think you should tell people it's bad

[nova iris]

yeah, i agree, i believe in our case it's actually the optimal solution due to our code structure, but then again, that code structure is likely not ideal 😂

[elder blade]

If the code is deeply nested, it is the easiest way without having to do Go-style error handling with always checking if a specific value was returned

[main lynx]

i consider it an "if you have to ask" thing

[nova iris]

i see, thanks!

[raven ridge]

That's different advice than the advice I was referring to. The original comments here were mentioning the cost of exceptions relative to "normal" returns as a reason to not use exceptions for control flow. That's common advice in C++ and Java, which doesn't apply to Python. Limiting non-local control flow is different advice, which I don't disagree with.

[paper echo]

Ah

Yeah I don't know if it's worth caring about "cost" in that sense for most programming that people do with python

The only time it matters is when you're doing something in a tight loop

For example if you have some string processing routine that you need to apply to a 1000000-row dataframe, you might not want to use exceptions for control flow

But for a basic web server handling cat photos or whatever it's not really worth worrying about

The better argument against using exceptions in python is that try/except is verbose and requires some abstraction/helpers to use in a single expression

[nova iris]

functools.wraps is basically functools.update_wrapper but it helps it accepts arguments. it does this by using functools.partial
so i ran some tests, to see which one is faster: decorators accepting arguments using partial, or decorators accepting arguments using triple definition (very nested and ugly, which i don't like).
apparently, triple def is faster! i used timeit at 100,000 reps, so i'm really confused as to why functools opted for partial instead

for those interested, here's the code: https://paste.pythondiscord.com/uxijaxizun.py

sorry for broken indents, that's idle :P

[paper echo]

Maybe just "dogfooding" with partial

[main lynx]

how much faster?

[peak spoke]

It's a decorator that's ran with definitions, run time is not a huge concern

[main lynx]

oh, right time spent setting up the decorator not much of any concern

unless it's like a 100x difference and your project uses decorators heavily

[nova iris]

usually 100 milliseconds, so definitely not a huge concern lol

[nova iris]

what does that mean?

[main lynx]

difference between the two implementations is what i'm curious about, what is the relative cost of the two approaches

[nova iris]

i told you, 100 ms is the difference

[main lynx]

100 milliseconds is a year

[nova iris]

here are the precise runs, i ran each 5 times

[main lynx]

ok so about 1 microsecond