internals-and-peps

[spark magnet]

@static bluff not sure if this is helpful to you, but this is a pretty involved tokenizer I wrote to lex JavaScript: https://github.com/django/django/blob/main/django/utils/jslex.py

[static bluff]

Your code is gorgeous ❤️

What did you base this off of

Its fully functional?

[spark magnet]

I don't think it was based off of anything, and it's functional, it's shipped as part of Django.

I'm not sure what "fully" means here.

[halcyon trail]

I'm curious what functional actually means in a context like this

skimmed the code and saw for example:

        self.regexes = {}
        self.toks = {}

        for state, rules in states.items():
            parts = []
            for tok in rules:
                groupid = "t%d" % tok.id
                self.toks[groupid] = tok
                parts.append("(?P<%s>%s)" % (groupid, tok.regex))
            self.regexes[state] = re.compile("|".join(parts), re.MULTILINE | re.VERBOSE)

surely in a functional style this would be a couple of comprehensions instead of creating a data structure and mutating it

(I'm not saying the code would be nicer or better that way, but it's closer to a functional paradigm)

this code is actually a great example of where a functional solution seems much more painful than one which does mutation. I can do this more functionally, but it's both less nice and typically less efficient

trying to see if there's a nicer solution I missed. but the imperative solution is easy, you don't have to think to make it good.

[gleaming rover]

"functional" as in "working"

[halcyon trail]

ah lol

[gleaming rover]

anyway

that would be...

[halcyon trail]

def groupid(tok): return "t%d" % tok.id
self.regexes = {groupid(tok): tok for rules in states.values() for tok in rules}
self.toks = {state: re.compile("|".join("(?P<%s>%s)" % (groupid(tok), tok.regex) for tok in rules), re.MULTILINE | re.VERBOSE)} for state, rules in states.items()}

[gleaming rover]

yeah, something along those lines

[halcyon trail]

not beautiful but the best I could do

even here you have the inefficiency of doing the string formatting operation twice as many times as necessary

[gleaming rover]

part of that is because

of the instance

[halcyon trail]

when you are producing multiple data structures from one data structure, this seems hard to do in a functional paradigm

(hard to do efficiently)

[gleaming rover]

hm

it's because Python doesn't have for-comprehensions

that said

[halcyon trail]

what are for comprehensions?

[gleaming rover]

a good way to do this

would be with zip, I think

[halcyon trail]

how would zip help?

[gleaming rover]

https://docs.scala-lang.org/tour/for-comprehensions.html

another way to express monadic computation

[gleaming rover]

you have a function

mapped over rules

hold up, let me think

[halcyon trail]

I'm not sure how for comprehensions are different from generator comprehensions

val twentySomethings =
  for (user <- userBase if user.age >=20 && user.age < 30)
  yield user.name  // i.e. add this to a list

[gleaming rover]

the existence of flatMap

[halcyon trail]

you can express that in python

syntactically it's very awkward but it's basically just map (or a comprehension) + itertools.chain

[gleaming rover]

in the case of list-equivalents, yes

[halcyon trail]

The basic problem here isn't the iteration aspects. It's that you need to populate two data structures, and you have some kind of intermediate state that's useful in computing both elements

[gleaming rover]

but there are other monads for which flatMap does not operate the same way

[halcyon trail]

Laziness doesn't really help

[gleaming rover]

anyway

[halcyon trail]

you can write out some kind of lazy expression that compujtes the intermediate state, but you still need a way to create both data structures from one lazy expression

basically you need some kind of functional abstraction that creates more than one data structure in a single pass

I'm sure it exists somewhere but it's not so super common

[gleaming rover]

which is zip, in this case

you create a tuple each iteration, then you zip to effectively transpose

then destructure

[halcyon trail]

what would that look like?

those zip transposition tricks I always find unintuitive

[gleaming rover]

probably super ugly

hm

I think

state monad would be helpful here

zip would not be appropriate actually

yeah I think state monad?

hm

def reducer(reducer_state, next_value):
    tokens, regexes = reducer_state
    state, rules = next_value
    new_tokens = {f"t{token.id}": token for token in rules}
    parts = [f"(?P<{token_id}>{token_regex})" for token_id, token_regex in new_tokens.items()]
    new_regex = re.compile("|".join(parts), re.MULTILINE | re.VERBOSE)
    return ({**tokens, **new_tokens}, {**regexes, state: new_regex})

reduce(reducer, states.items(), ({}, {}))

this is my guess

I think the middle part can be improved but I am too sleepy for this

I take your point though

[paper echo]

part of the problem with python is that there's no compiler optimizing any of this stuff away

so immutable data usually means you are actually making lots of (shallow) copies and maybe re-doing operations

[gleaming rover]

that's a data structure problem IMO

[paper echo]

but is not trivial

pandas and numpy both go through a lot of effort to avoid making copies for example

[gleaming rover]

care to elaborate

[paper echo]

copy on write semantics, returning "views" (sometimes), pandas has the nice functions like .assign and .pipe that let you kind of pretend dataframes are immutable

etc.

[gleaming rover]

if anything

I would say they go out of their way to make copies

[paper echo]

making copies of the container while delaying copying the underlying data as long as possible

something like that i guess

def groupid(tok):
    return "t%d" % tok.id


def state_pattern(rules):
    pattern_str = "|".join(
        "(?P<%s>%s)" % (groupid(tok), tok.regex)
        for tok in rules
    )
    return re.compile(pattern_str, re.MULTILINE | re.VERBOSE)


class A:
    def something(self):
        self.regexes = {
            groupid(tok): tok
            for rules in states.values()
            for tok in rules
        }

        self.toks = {
            state: state_pattern(rules)
            for state, rules in states.items()
        }

this is perfectly inoffensive python to me

well i messed it up but you get the idea

just lay everything out pretty, write lots of helper functions

but yeah either you: 1) cache the results somewhere and end up iterating twice, 2) compute everything twice, or 3) save the results in a list of tuples and unzip them later thereby iterating twice

im not sure how youd even do this in haskell without making two passes over the data... arrows?

[halcyon trail]

Yeah, the problem is that reduce operations on collections are horrible

I mean at that point you can just leave the original for loop

[gleaming rover]

ye

[halcyon trail]

and do x = x + [....] instead of appending

[gleaming rover]

without an immutable data structure

[halcyon trail]

and say look, I'm not mutating 🙂

[gleaming rover]

there's not much point

but the theory...

[halcyon trail]

Yeah. The thing is that proper immutable data structures have a lot of their own trade-offs.

[gleaming rover]

actually I think some sort of nested state would be appropriate

[gleaming rover]

ye

it depends a lot on the usecase, too

but I suppose this is one of those "spend performance for (supposed) code clarity" things

[halcyon trail]

It's kidn of debatable whether the code is clearer, is the problem 🙂

[gleaming rover]

hence "supposed"

[halcyon trail]

One thing I like is the notion of immutable data structures that have some kind of "builder" API that lets you use mutation once at the point of creation

languages that try to mix functional imperative to some degree like Kotlin have some nice APIs around this

[gleaming rover]

if your mutation never escapes the function, is it still functional? 🤔

[halcyon trail]

Although I'm not sure how it would work to create two at the same time

[gleaming rover]

anyway

this is my thought process

[halcyon trail]

I think probably for most people an approach where you can locally scope mutation probably gives you 90% of the benefits of both approaches

local mutation just isn't a big deal. It's the crazy action at a distance stuff you can get later that's a problem

And I've definitely been bitten by that in python quite a few times

[paper echo]

this is a great question. the D language says that a function is pure even if you do mutation on local variables, because it's pure from the perspective of the caller

i believe languages with algebraic effects take a similar approach, where if an effect is handled locally then the function has no effect from the perspective of anything outside

[static bluff]

God I hate regex

I've really got to sit down and just learn the bastard, balls to bones

[tacit hawk]

how super() with no arguments knows what's the current class and next one in hierarchy?

[halcyon trail]

magic

[undone hare]

!e
It is calculating what we call the MRO (Method Resolution Order) which basically says in which order you should look up super class. Here is an example:

class A:
    pass

class B(A):
     pass
```Since B will inherit from A, the first object of its MRO will be A, right?
```py
print(B.__mro__)

So now, when you call super() inside B it will look at the first object of its MRO, A. This is also used, as the name says, for resolving methods.
As you may guess this MRO can be a bit more complicated to compute with multiple layers and many parents, but it will still end up in the same flat list.

[fallen slate]

@undone hare :white_check_mark: Your eval job has completed with return code 0.

(<class '__main__.B'>, <class '__main__.A'>, <class 'object'>)

[tacit hawk]

that part I know, just don't know the magic done under the hood, probably the interpreter handles that

it searches mro from beginning for each level of the hierarchy?

doing N² loops?

[peak spoke]

https://www.python.org/dev/peps/pep-3135/#specification the argumentless super is equivalent to super(__class__, self) where __class__ is the method's class

then it just goes through the MRO starting with the passed in class as usual

[tacit hawk]

ok thanks

[grave jolt]

Philosophical question: in Haskell, there's the IO type ||(it's a type constructor, not a type, not important)||. It is required for I/O because of the Haskell semantics (referential transparency), but it also separates your application logic (the pure functions) from the nasty I/O (terminal, databases, queues, network, etc.).

Doesn't explicit async/await syntax sort of do the same? You either have to make every single bit of logic async, or separate the nasty I/O from your domain logic.

[static bluff]

There are ways to get around it

It depends on your usage. It is possible to run async within sync. What's the context

[raven ridge]

Doesn't explicit async/await syntax sorry of do the same?
Sort of, in that it enforces that async functions can't be directly called from sync functions and must be handled specially. But it doesn't enforce that blocking methods can't be directly called from async methods, so it really doesn't enforce that you've separated things

[static bluff]

Calling async from sync is possible, I think, if you either spin up an event loop within your sync function and block until the loop exits, or run the async method in another process, blocking with a while loop that checks for a return value or completion flag passed through an IPC manager

You could probably wrap both processes in a context manager if its a repetitive task

That said, its usually a sign of questionable design to have to deal with async from sync anyway- async is for event driven programs which almost universally run on an event loop by their nature

[grave jolt]

yeah, you can still call I/O in non-async functions, but if you're doing a lot of it, it's going to cause problems

that said, pathlib.Path does sneaky I/O, and I'm not sure I really like it 👀

perhaps I should use PurePath

[visual shadow]

if you don't want IO, you can use PurePath, for sure. Path's not to blame because it's purpose is to support IO calls after all

[paper echo]

in an abstract sense, yes. i've seen it argued that monads are a generalization of "function colors" like async vs non-async.

[grave jolt]

That sort of makes sense. If a function returns an Optional or a list, it returns values of a particular type, but you can't treat it like a normal value.

although... in EdgeDB applying a function to a set returns a new set with the function applied to each element, a 'missing' value is the empty set, and a single value is a singleton set.

[static bluff]

How are my favorite python people doing?

[boreal umbra]

I'm just fabulous! This isn't a social channel, though. Try one of the off-topic channels.

[static bluff]

Just thought it was a little too quite in here

😛 My bad

[acoustic crater]

does object.__init__ do anything?

[boreal umbra]

I don't believe so.

[acoustic crater]

kk thanks

wish it accepted args lol

[boreal umbra]

!e object(arbitrary='data')

[fallen slate]

@boreal umbra :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | TypeError: object() takes no arguments

[acoustic crater]

just so it wouldn't be annoying

[boreal umbra]

What would it do if it did accept args?

[acoustic crater]

nothing haha

it's just annoying that tuples call it for some reason

or I guess

when I try to get it from a tuple the tuple gives me object init

rather

[boreal umbra]

you're upset that tuples call object.__init__? object is the class that all classes inherit from.

[acoustic crater]

I'm upset that I need to check to make sure I'm not getting object init when I automatically call a class's init because object init gets mad if I pass it args

[boreal umbra]

example?

[acoustic crater]

def _method_closure(method, cls, supercls, super_new, super_init):
    """Closure for operation methods which use superclass method but return subclass."""
    _new_ = supercls.__new__ if super_new else cls.__new__
    _init_ = supercls.__init__ if super_init else cls.__init__

    # object __init__ does not take any args, check for it (tuple uses it)
    init_class = getattr(_init_, '__objclass__', None)
    if init_class is object:
        _init_ = lambda new, value: object.__init__(new)

    super_method = getattr(supercls, method)

    @wraps(getattr(cls, method))
    def method_wrapper(self, *args, **kwargs):
        """Wraps method to return current class instead of supercls using supercls methods."""
        value = super_method(self, *args, **kwargs)
        new = _new_(cls, value)
        _init_(new, value)
        return new


    return method_wrapper

tho since it does nothing I can make that lambda just do nothing

[boreal umbra]

admittedly I can't tell what this does. nice work.

(I'm also mentally exhausted)

[acoustic crater]

lol that's metaprogramming for ya

[boreal umbra]

a year ago or so, I started working on a decorator for __init__ methods that assigns each argument you pass to it to an attribute with the same name as the parameter

[acoustic crater]

nice

[boreal umbra]

I gave up when writing the tests, though I've learned a lot in the last year, so I might try it again. who knows.

[acoustic crater]

that's my kind of decorator, though any IDE and most ppl who aren't like me will get mad about lack of explicit attribute assignment

[boreal umbra]

or I can just use attrs.

[acoustic crater]

I have 100% test coverage for this module and it's helping a loooot when adapting it like adding tuples which gave me this object init problem

my tests are very ugly though, lots of copy pasting and changing a few things, should I refactor tests to re-use code?

[static bluff]

This lexer is finally starting to move, I think

Could I get someone to take a look at this regex?

        stringLiteral =  r'(?:([rR]?[bBfFuU]?)|[bBfFuU]?[rR]?)("""|\'\'\'|"|\')';
        stringLiteral += r'((?:\\?[\n]|.+)*?){/0}';

It's supposed to match a valid python string

[grave jolt]

the u prefix can't be combined with f or r

but bf is legal

[static bluff]

Oh, thank you

[acoustic crater]

doesn't look like it accounts for triple quoted single strings

like '''

nobody uses them but they are valid I think

[grave jolt]

I can't seem to match anything with it

[acoustic crater]

!e print(test)

[fallen slate]

@acoustic crater :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | NameError: name 'test' is not defined

[acoustic crater]

lol

whoops

used ticks

!e print( '''test''')

[fallen slate]

@acoustic crater :white_check_mark: Your eval job has completed with return code 0.

test

[static bluff]

Yeah its probably got some issues :/

[grave jolt]

yeah, to be more specific, it doesn't work 😄

[static bluff]

I mean- it looks promising for sure but I've got to work on it definitely

[acoustic crater]

I again am gonna recommend haskell for this haha

regex seems like it's gonna be totally unreadable even if it works with a bit of logic in between

[static bluff]

        self.createEmblem( 'INITIAL_WHITESPACE', 'WHITESPACE', '[ \t\f]*', self.whitespace );

        self.createEmblem( 'CONTINUATION', 'CONTINUATION', r'\\', self.continuation );

        self.createEmblem( 'NEWLINE', 'NEWLINE', r'[\n]|[\r][\n]|[\r]', self.newline );

        self.createEmblem( 'COMMENT', 'COMMENT', r'(\#.*)', self.comment );

        self.createEmblem( 'DOUBLE_HEADLESS', 'DOUBLE', r'[+-]?\.[0-9]+', self.double );
        self.createEmblem( 'DOUBLE_TAILLESS', 'DOUBLE', r'[+-]?[0-9]+\.', self.double );
        self.createEmblem( 'DOUBLE_BASIC', 'DOUBLE', r'[+-]?[0-9]+\.[0-9]', self.double );

        self.createEmblem( 'COMPLEX', 'COMPLEX', r'[+-]?[0-9][jJ][0-9]+', self.complex );

        self.createEmblem( 'BINARY', 'INTEGER', r'0[bB](_[01])+', self.integer );
        self.createEmblem( 'OCTAL', 'INTEGER', r'0[oO](_[0-7])+', self.integer );
        self.createEmblem( 'DECIMAL', 'INTEGER', r'[1-9](_[0-9]*)', self.integer );
        self.createEmblem( 'HEX', 'INTEGER', r'0[xX](_[0-9A-Fa-f])+', self.integer );

        prefix = r'([rR]?[bBuU]?)|([bBuU]?[rR]?))|[bBfFuU]?'
        stringStart =  r'(?:([rR]?[bBfFuU]?)|[bBfFuU]?[rR]?)("""|\'\'\'|"|\')';
        stringLiteral = stringStart + r'((?:\\?[\n]|.+)*?){/0}';

        self.createEmblem( 'STRING', 'STRING', stringLiteral, self.string );
        self.createEmblem( 'UNTERMINATED', 'UNTERMINATED', stringStart, self.unterminated );

        operators = r"""\*\*=|//=|<<=|>>=|\~\->|\.\.\.|!=|%=|\&=|\*\*|\*=|\+=|\-=|//|/=|<<|
        <=|==|>=|>>|\^=|\|=|@=|%|\&|\(|\)|\*|\+|,|\-|\.|/|:|;|<|=|>|@|\[|\]|\^|\{|\||\}|@""";

        self.createEmblem( 'OPERATOR', 'OPERATOR', operators, self.operator );

        keywords = r"""continue|nonlocal|finally|assert|except|global|import|lambda|return|
        False|break|class|raise|while|yield|elif|else|from|pass|with|True|None|and|def|del|
        for|not|try|as|if|in|is|or""";

        self.createEmblem( 'KEYWORD', 'KEYWORD', keywords, self.keyword );

Its not that bad

I think I've managed to break it up into readable chunks

[acoustic crater]

you can tell where stuff is but the cognitive load of figuring out if those are gonna work is enormous

[static bluff]

(riddled with errors, probably- its a work in progress)

[rich cradle]

That looks well divided, but taking the time to understand that RegEx seems not fun

[static bluff]

Well, Sly/Ply use regex the same as this does- it just uses a bunch of introspection to handle the actual matching and token generation

There's also lark- but that looks like a rather deep rabbit hole that I'd rather avoid

[quasi hound]

#this works
def multiple_return_vals():
  return 11, 12, 13

_, _, thirteen = multiple_return_vals()
print(thirteen) #13

# [pyflakes] duplicate argument '_' in function definition
def f(_, a, _):
  pass```
can't find a bug channel so i'm putting this here, do you guys think this is a bug or what

[static bluff]

Which leaves the tried and true method- the method that python, pypy, and parso all use- pure regex

[acoustic crater]

#❓｜how-to-get-help

[quasi hound]

i don't need help

i'm just saying

doesn't work on any version i tried

[grave jolt]

!e
It's not a bug in pyflakes 😄

def f(_, a, _):
    pass

[fallen slate]

@grave jolt :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 | SyntaxError: duplicate argument '_' in function definition

[grave jolt]

or what are you saying?

[acoustic crater]

oh not hat's not a bug _ is just a name in python

[grave jolt]

Yeah, _ has no special meaning.

[quasi hound]

right i know that

[acoustic crater]

_ does not have any special significance in python outside of patterns in 3.10

[quasi hound]

ok hear me out

outside a function right

_ can be put twice in a row

like i did in the first example

but not inside apparently

[acoustic crater]

that's because it just gets assigned twice

[quasi hound]

oh right

and assignments have no effect

but what if you wanted to ignore the first and third function arguments? what would you do then?

idk this is just kinda weird to me

[acoustic crater]

you can just do *args and only use args[1]

[quasi hound]

i guess but you are saving the whole list to memory

that's why this looked like a bug to me

[acoustic crater]

lol python does not work very hard to save memory

[grave jolt]

creating a tuple isn't really that expensive

[quasi hound]

yeah you're right

damn he typing

[grave jolt]

If it's your own original function, you can just not put the unused parameters in there.

If you're expected to pass that function as an argument elsewhere (or are ignoring arguments in a subclass method (which probably breaks Liskov's substitution principle (so you probably shouldn't to it))), the function usually has canonical argument names. You can flag them as unused to the reader
by prefixing with underscores:

from some_web_framework import SomeWebFramework

app = SomeWebFramework()

@app.get("/user/{id}")
def get_user(req):
    user = database.get_user(req.params["id"])
    return app.json({"user": user.to_json})

@app.on_error(404)
def handle_not_found(_req):
    return app.json({"error": "we searched far and wide, I promise"})

[quasi hound]

makes sense

[grave jolt]

although to be honest, it does seem quite inconsistent when you put it together with _, _, x = 42
on the other hand, it's already quite a common pattern, and I haven't seen ignored args really often

It would also be a bit inconsistent to disallow assignment targets being the same name, because there are attribute and item assignments (like foo["bar"], bar.baz = 4, 5), which can actually have side effects, so it's not redundant

you could also do

def f(*args):
    _, _, x = args
```,
but then if you want to provide a type signature, you'd need to put an overload
```py
from typing import overload, Any

@overload
def f(_0: Any, _1: Any, x: int) -> str:  ...

def f(*args):
    _, _, x = args

and now it's way more verbose

[quasi hound]

what does overload do

[static bluff]

What about

*null, x = *args

Its a tiny bit sexier

[quasi hound]

you mean *_, x = args? yeah i guess

but the inconsistency is all i was really asking about

[static bluff]

Underscore, null, no dif

[quasi hound]

oh you can use null the same way?

and it "discards" whatever was there?

[grave jolt]

Are you familiar with typing annotations in Python?

[quasi hound]

only the basic ones

[grave jolt]

null is just a variable name, just like _

[static bluff]

Underscore is just a variable name.

You can use whatever you like. What matters is the star expression

[quasi hound]

oh i get it

[native flame]

even if you use _ as a name it doesn't get discarded

[quasi hound]

wait really

[static bluff]

I soooooooo wish Python had named unpacking like javascript

[quasi hound]

ok that's new

[static bluff]

{ a, b, c } = { 'a': True, 'b': False, 'c': None }
print( a, b, c )

>>> True, False, None

• The above is not valid python code

[native flame]

!e

_ = 5
__ = _ * _
print(__)

[fallen slate]

@native flame :white_check_mark: Your eval job has completed with return code 0.

25

[quasi hound]

ok interesting

[grave jolt]

overload is a way of telling that a function has several ways of calling it.

@overload
def add3(x: int) -> int: ...
@overload
def add3(x: int, y: int) -> int: ...
@overload
def add3(x: int, y: int, z: int) -> int: ...

def add3(x, y=None, z=None):
    return add3(x + (y or 0) + (z or 0))

[static bluff]

❤️ overload

[quasi hound]

oh interesting

[static bluff]

I had no idea it was so easy though!

[grave jolt]

well, it does absolutely nothing at runtime

for that there's functools.singledispatch, but not sure it can handle that pattern

[grave jolt]

it is, it will just assign the keys to the variables 🙂

[static bluff]

I think I'll try to implement overloading into my custom dialect

Overloading, optional static typing, privacy

A few new operators and keywords

[grave jolt]

That will be somewhat available in 3.10:

[static bluff]

Oh helllllllz yeah

[grave jolt]

I agree that it's nowhere near as nice as in JS, though 🙂

[static bluff]

They're putting in a switch statement too? Ballzey

[grave jolt]

It's not a switch statement

it's pattern matching

[static bluff]

Oh

[quasi hound]

it's better than a switch

[raven ridge]

s/better/fundamentally different

it's like regexes but for arbitrary objects instead of strings.

[grave jolt]

Optional static typing is already there.
Overloading is present with functools.dispatch, or you can make a library for more complicated cases (I actually did that a while back, fun exercise).
Private methods/variables are done with underscores or with __ to make them private even for subclasses (which is mainly there to prevent name collision).

[static bluff]

For a variety of reasons, not least of which is simply a deep fascination for the subject, I'm writing my own dialect of python

[quasi hound]

in what

[grave jolt]

I agree
A switch statement checks a value with a list of values, and executes some code when there's an equality somewhere.
Pattern matching checks a value against a list of patterns, and can do destructuring (binding names)

[quasi hound]

C?

[static bluff]

My coding style is- ahem- unique (monstrous is the word you guys might use) and I want a language that takes the best of python, but tweaks a few things

[grave jolt]

Not saying you shouldn't be doing that 🙂

but optional static typing really is there.

[static bluff]

But I'm a kid in a candy store! The possibilities are endless 😄 😄 😄

[static bluff]

No, type annotation is there. But not type enforcement

The annotation helps with linters and such, but it wont throw an error

[quasi hound]

yeah doesn't js have a strict mode thing, like a string you put at the start of the program and it enforces types

do that

"use strict"; as the first line

is what it is

[grave jolt]

Enforcing the type annotations can be done with linters.

The whole point of static typing is that you don't check types at runtime. You do that statically, that's the whole point.

[static bluff]

❤️ I love you guys

[rich cradle]

It does, but if you're going to do that, just use TypeScript

[static bluff]

Finally, people to talk to

[raven ridge]

type enforcement means you can't have gradual typing. The two are mutually exclusive.

[quasi hound]

i guess yeah

[static bluff]

gradual typing?

[grave jolt]

JS doesn't have type annotations, use strict just makes it a little bit less loose https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Strict_mode

[quasi hound]

oh ok

[raven ridge]

some variables are typed, some are not, variables without types are assumed to be valid for any operation (type checks are disabled on them)

[static bluff]

And in any case- the at-runtime static typing would be completely optional with its own syntax. Don't want it? Don't use it

[quasi hound]

and here's a bit of syntax i kinda hate: a: int = 5
should be a = 5: int
so maybe change that around

(imo)

and i feel like you could do something with labels

[raven ridge]

if you statically type a function parameter as a list-of-lists-of-ints, and you call the function with ```py
foo([[1,2,3],[4,5,6],[7,8,'9'])

what happens?

[grave jolt]

You don't usually annotate literals like that.
and consider this:

a: Optional[User] = very_long(quite_complicated, expression + indeed == "yes")
a = very_long(quite_complicated, expression + indeed == "yes"): Optional[User]

in the first variant, you don't have to read the entire expression to know the type.

[quasi hound]

yeah i thought about that

int a = ... would just be better then i guess

[grave jolt]

@static bluff

1. You will still get a type error at runtime. It's a bit more descriptive, sure. But it doesn't make the code more safe.
2. For many values, you can't check the types at runtime.
   You get a list[list[int]]. Are you going to loop over each list, loop over each list, and then check if each item is an int?
   How do you implement a type-safe binary search function then? Loop over the input on every call? That kind of defeats the purpose.
   Similarly, every time you use a dict its lookup performance benefits disappear.
   How do you check the type of a function?

[quasi hound]

but maybe hard to parse idk

and by labels i mean the ones that look like this in java

outerloop:
for(int i = 0 blah blah blah){
  innerloop:
    for(int j blah blah){
      if (something) break outerloop;
      if (somethingelse) break innerloop;
    }
}

idk why these aren't in python

[grave jolt]

That has been already rejected, here's the proposal: https://www.python.org/dev/peps/pep-3136/

You can, of course, still use

!pypi goto-statement

[fallen slate]

goto-statement v1.2

A function decorator, that rewrites the bytecode, to enable goto in Python

[grave jolt]

🙂

[static bluff]

Here's what I'm after

def _innerE( var ):
  return var + " BATMAN"

def _innerD( var ):
  return innerE(var)

def _innerC( var ):
  return innerD(var)

def _innerB( var ):
  return innerC(var)

def _innerA( var ):
  return innerB(var)

def exposedFunction( var ):
  return innerA(var)

exposedFunction(1)

In this scenario, a user accidentally enters a number instead of a string. The variable propagates all the way up through the internals of the project- which may be arbitrarily large- and an error is produced. A big fat stacktrace is output to the console.

Now, to us in the advanced channel, debugging via stacktrace is simply a way of life. But if you're building a project designed for beginners, or else with such a broad focus in its exposed scope already that to have to contend with the internals would be too cumbersome, recieving a runtime error such as this

ArgumentError: could not perform operation 'exposedFunction()': required positional argument 0 'var' must be of type 'str'

saves a lot of time

[raven ridge]

OK, but you didn't answer my question, or answer Fix

[quasi hound]

nah i've heard bad things about goto lmao i just meant break

or whatever

[static bluff]

Sorry, took a while to type that. Lemme get caught up

[grave jolt]

def exposedFunction(var: str):
  return innerA(var)

exposedFunction(1) 

If you use a type checker, it will yell at you even if you never run that line.

[static bluff]

Hmmmmm

These are all fair points

[grave jolt]

Now, sometimes you might want to validate the arguments at runtime. Perhaps at the I/O boundary, or just at the boundary of a library. That's a good idea, and you can do that in plain Python

@runtime_check
def exposed_function(var: str):
    ...

and then inspect the annotations in runtime_check

[raven ridge]

if you limit your types to just int or str, it seems simple - but when you allow types like "list of lists of ints" or "function accepting an integer and a string and returning a float", it's not reasonable to check that at the time of the call.

[grave jolt]

yes

[static bluff]

What I'll say is that I'll be wanting to revisit this issue in great detail once I get to the part where I can actually be modifying the syntax

[grave jolt]

also that

[grave jolt]

I can see it being useful at the I/O boundary. pydantic does just that, sort of

but it's not accepting calls from the language itself

[static bluff]

I seem to be on the losing side of the argument- and I'm willing to accept that if it comes to it 😛

That said- I'd really rather not have my project do any precompiling at all

[raven ridge]

I don't think it's an argument, really - we're just pointing out that "I'll statically type things" is much, much harder than it seems

[static bluff]

My friend, I completely agree

[grave jolt]

or rather, that static typing isn't (dynamic typing + runtime checking)

[raven ridge]

it means that either everything needs to be typed - in which case the language stops looking like Python - or that you need to check the types of everything - which can be expensive (nested datastructures) or impossible (knowing what type a function will return before calling it is, I think, equivalent to the halting problem)

[static bluff]

Not true

Functions know what their annotations are. You'd simply have to iterate through those whenever the function gets called

[grave jolt]

def apply_to_five(f: Callable[[int], int]):
    return f(5)

how do you check if f is true to int -> int?

[static bluff]

With a little bit of eval magic, you could write a function whose signature is identical to the original and thus unpacks the arguments and injects them into the functions scope the same way, and then perform instance checks on all of those arguments. If it passes muster, move on to the original function

[grave jolt]

perhaps f is

def f(x):
    if x == 5:
        return 42
    return ["not", ["five"]]

[static bluff]

Gimme a min

[grave jolt]

The point about looping over a nested structure is somewhat invalidated is you can enable the checks in debug mode but disable in production mode.
(although this won't help users of the library who pass in an incorrect type, but then you might have some global debug mode switch (which the beginners should be aware of))

But then you might just want use a contract-checking library like deal https://github.com/life4/deal
Apparently, it also does some static analysis, but I haven't used the library: https://deal.readthedocs.io/basic/linter.html

[raven ridge]

sure, but what if the function doesn't have annotations?

[grave jolt]

and what if the annotation lie? 👀

[static bluff]

class Static(  ):

  def __init__( self, method ):

    self.method = method;

    validator = f"def {method.__name__}{inspect.signature(method)}: \n"

    for argname, typing in inspect.getfullargspec(method).annotations.items():

        validator += f"\tif not isinstance{argname, typing}\n";
        validator += f"\t\tself.raiseTypeError(method, argname, typing)\n"

    self.validator = exec( validator, {}, {} ) #can't quite remember how exec works at the moment

  def __call__( self, *positional, **encyclopedic ):

     self.validator( *positional, **encyclopedic )

     return self.method( *positional, **encyclopedic )

This is a super generalized, on the fly rendition of a type checker I've built in the past

[raven ridge]

well, that doesn't work in real Python. You can't use isinstance on str to check if it's a Callable[[Any], str]

[grave jolt]

yeah

even

[raven ridge]

that's an example of something that works only so long as it's constrained to simple types.

[static bluff]

It doesn't have to be isinistance, and there can be plenty more logic involved

[raven ridge]

well, let's go back to my first argument, then: what if the function isn't annotated at all?

[static bluff]

But thats just an example of how you can unpack the arguments naturally and only deal with the arguments that are in fact annotated

If it's not annotated, the function fires normally

[raven ridge]

and then you've lost type safety in the function that called it

def math(op1: float, op2: float, func: Callable[[float, float], float) -> float:
    return func(op1, op2)

def unannotated_function(op1, op2):
    return "oops"

value: float = math(1.0, 2.0, unannotated_function)

[static bluff]

Sorry, what's this 'Callable[[float, float]'?

[raven ridge]

we've got an unannotated function that takes two floats and returns a str. We pass it to a function that expects a function that takes two floats and returns a float. Because it's not annotated, the mistake isn't caught. Because the mistake isn't caught, it propagates, and now value holds a value that its type hint says it cannot.

[static bluff]

I understand what a callable is

[grave jolt]

Callable is an annotation for a callable object. so Callable[[float, float], float] is a function that takes two floats and returns a float. (Not necessarily a function, any callable)

[raven ridge]

Callable[[int, str], float] is a callable that takes two positional parameters, first an int then a str, and returns a float.

[static bluff]

Okay cool

Gimme a min to read this and try to wrap my brain around it

[grave jolt]

If you only run the checks at runtime, you could run the function on a few example values (perhaps with a property testing tool like hypothesis).
But:

1. The function might have side effects
2. Creating good examples of arbitrary types (e.g. functions (which is actually not that hard maybe)) isn't generally possible, and you'd have to write your own generators

[raven ridge]

perhaps with a property testing tool like hypothesis
time.sleep goes brrrr

[grave jolt]

yeah lol

[static bluff]

value: float = math(1.0, 2.0, unannotated_function)

[grave jolt]

well, it will eventually finish running all examples!

[static bluff]

Your saying that the value of 'value' could be set to something other than a float?

Hence what you said- "type safety is lost"

[grave jolt]

yes, you can't check the type of unannotated_function at runtime

[raven ridge]

yes, if we run exactly that line, value has been set to a str

despite being declared as only ever holding a float

[static bluff]

I can intercept assignments no problem

[grave jolt]

then you have to annotate every single assignment

[raven ridge]

which means you can't infer types at all.

[grave jolt]

you then also can't have nested calls

because

i_expect_a_float(math(1.0, 2.0, unannotated_function))

[static bluff]

i_expect_a_float(math(1.0, 2.0, unannotated_function): float)

I don't see why that wouldn't work

[raven ridge]

it's not that it shouldn't work, it's that now you need to type hint every expression

you're littering type checks everywhere

[static bluff]

Again, I'm having trouble understanding where this 'type check everything' is coming from

You'd only annotate what you want to annotate

[raven ridge]

at the start, you only wanted to type check function parameters. Now you want to type check function parameters, and assignments, and arbitrary subexpressions of complex expressions

[static bluff]

Fair point, the scope is certainly creeping

And you're completely correct- I'm probably getting ahead of myself

But having some level of basic enforcement as sugar only would suit my purposes nicely

[raven ridge]

and what would you do about the case where I declared that I want a list-of-lists-of-ints, but was passed a list-of-lists-of-sometimes-int-and-sometimes-str?

[static bluff]

I honestly don't know. That's half the fun

[raven ridge]

you could check that at runtime, but it would be ridiculously slow

[static bluff]

So okay. Explain to me how this pre-checking thing works

Why is it better, and what are the pitfalls

I'm just gonna run out for a smoke

[raven ridge]

it's not better than a fully statically typed language, just different. It's a way to gradually add types to a language, and type check only the places that most benefit from having their types checked, while still allowing duck typing and dynamism where it's more useful

the idea is that there's a tool that reads your code and tries to apply type analysis to it. It figures out what types variables hold, either based on the literals that you've assigned to them, or the function returns that you've assigned (in the case where it knows what type the function returns).

Then, once it knows what type a variable holds, it keeps reading your code and sees if you ever do anything with that variable that expected a different type. If you do, it yells at you.

[static bluff]

Hmmmm

So its a filter, not perfect, but generally a happy medium between static and duck typing

[raven ridge]

yeah - it's a way to have type checking in a dynamic language, without paying the costs of type checking at runtime, and without giving up dynamic typing entirely

[grave jolt]

There is a way to 'type-protect' a function argument, actually. When expecting a function, you can wrap it to check the return value every time you run it. (in debugging mode of course, otherwise it's silly)
(and, of course, it's still dynamic, not static)

[raven ridge]

taking my example from above, and adding the type hints to the function that was missing them:

(venv) $ cat testing.py 
from typing import Callable

def math(op1: float, op2: float, func: Callable[[float, float], float]) -> float:
    return func(op1, op2)

def wrong_function_type(op1: float, op2: float) -> str:
    return "oops"

value = math(1.0, 2.0, wrong_function_type)

(venv) $ mypy testing.py
testing.py:9: error: Argument 3 to "math" has incompatible type "Callable[[float, float], str]"; expected "Callable[[float, float], float]"
Found 1 error in 1 file (checked 1 source file)

[static bluff]

So if I wanted to integrate this into my dialect- what, run the check on each module on import?

Or would I have to check the entire project

[raven ridge]

this sort of check is usually performed at static analysis time, though I suppose it could be done at compilation time instead

though that would make it harder to work around things like library functions whose types are mis-declared (which happens all. the. time. when types are complex enough)

[static bluff]

I mean- all that I'm after is basic type checking so as to avoid nasty stacktraces which occur from bad variables passed to internal methods

I could use a decorator for this I'm sure. I like the idea of baking it right into the language, but you all make some good points

[raven ridge]

as soon as your types are any more complicated than just int/str/float, you'll find that there's all sorts of reasons why that's more complicated than it sounds.

I'd really suggest reading up on the prior art, at least. reading through the mypy docs may open your eyes to just how complex typing gets

There are type hints to express things like "a function that accepts two arguments of the same type, and returns a value of that type" or "an object that has a method named close"

[static bluff]

I mean, take a toolbox library for example. A hundred and fifty different functions all taking various combinations of arguments, and you want these tools to unambiguously tell you you passed them a bad variable. Doing that by hand would suck

So automating the process would be lovely

And please understand- in no way do I consider this a trivial matter

[raven ridge]

well, with Python as it exists today, you can check that in your editor as you're writing the file, but the interpreter won't check it itself as the file runs

at least, not at the point when the call is made, though if you passed a type that the function won't work on, obviously you'll get an error later.

[grave jolt]

...and you can do runtime checking with a library like deal

because, well, you could also argue that not everyone is using a type checker

and there are also beginners who will use your library

[raven ridge]

yeah. it's not impossible to imagine that a future version of the language might ship with a way to opt into enforcement of some of the simpler type checks

ooh, another interesting case for type checking is type narrowing. If you declare that a function takes int-or-None, and then you check whether the parameter was None and return if so, then for the rest of the function the parameter is automatically known to be of type int, and can be passed to functions that don't accept a None

if you reorder one of those calls to a function that doesn't accept a None so that it comes before the return, then your type checker will complain

[grave jolt]

I would love to see type checking standardized, but that means that

• it will be quite simple (no mapping/index/conditional types like in typescript)
• will probably come from mypy and inherit its stateful code, quirks and bugs

what I would enjoy is something like... TypeScript but in Python

[raven ridge]

I'm thinking a simple form of it may get standardized 3 or 4 years down the line, when the typing ecosystem has stabilized

because by that point there may be a clear winner, and it may or may not be mypy

[grave jolt]

doesn't aim to be Agda, but provides enough tools for both basic tasks (like writing a web server) and for ivory tower library writers

I think that one of the biggest issues with Python's typing system is that it works fine for most application code, but breaks down for more complex cases

and in the end, the type quality in libraries does affect simple application code

[raven ridge]

Ya know, I remember making an ass of myself at a bar during PyCon a few years back, complaining about asyncio amongst a group of core devs, one of whom happened to be Yuri Selivanov. I said that it felt like it had been added to the language prematurely, it was half baked, trio was clearly better than asyncio, etc. And he said something that stuck with me: he said that yeah, maybe asyncio wasn't as good as trio, but that if asyncio had never been added to the standard library, trio probably wouldn't exist either.

I find it memorable from the point of view of letting the perfect be the enemy of the good - maybe if we waited for a perfect type system we'd never get anything, and maybe getting the bad type system into the language is a necessary precondition for something better to eventually exist

[grave jolt]

yes, I think it's a good thought

javascript also had different competing type systems, IIRC

[static bluff]

It's almost as if typing needs to have it's own utility language, like regex

It almost already does, but there's room for improvement

[grave jolt]

Absolutely, in a type system the types form a separate language.

[raven ridge]

In C++'s case, the type system's mini language turned out to be accidentally Turing complete

[static bluff]

I mean, I'd be curious to see what one could do by applying some sort of symbol method to classes and when checking, stringifying the symbols of the arguments provided and doing a regex check against what's expected

[grave jolt]

TypeScript's type system is turing complete as well, but it's not as horrible 🙂

[static bluff]

It'd be a fun experiment if nothing else 😛

I'd be curious to see C's typing mini language

[raven ridge]

C++'s, not C's.

[grave jolt]

describing any part of C++ as mini is not accurate

[static bluff]

XD

Though my dudes, could I bug you to take one last look at this regex?

[raven ridge]

https://www.infoworld.com/article/3257727/introduction-to-metaprogramming-in-c-plus-plus.html is a "gentle" introduction

[static bluff]

I promise, I won't bug you guys with it again

[raven ridge]

though there's only so "gentle" an introduction can be when it's to a language that was not designed, but discovered

[grave jolt]

You can always open a help channel, see #❓｜how-to-get-help 🙂

[static bluff]

I have, no bites so far

[grave jolt]

can you link it?

[raven ridge]

C++ template metaprogramming may be the only example of a programming language with actual production usage that was discovered rather than deliberately created 😄

[grave jolt]

I love regex

[static bluff]

import re as regex;

prefixes = r'(([rR]?[bBfF]?)|([bBfF]?[rR]?)|[bBfFuUrR]?)';
quotes = r'("""|\'\'\'|"|\')'

stringLiteral = prefixes + quotes + r'(.*){1}';

print( stringLiteral )
print( regex.match( stringLiteral, "'test" ) )

[grave jolt]

[static bluff]

It's not supposed to match (but it does) because the test string is missing the closing single quote

[grave jolt]

@static bluff can you link the help channel? it's kinda off-topic here

[static bluff]

Oh sorry

It's chocolate. Not sure how to link to a specific channel

[grave jolt]

You put an # and start typing, it will autocomplete

[static bluff]

#help-chocolate

Coool!

[eager trail]

We all know that using the names of inbuilt functions as variable names is bad practise, but does it actually have any memory cost in python?

[acoustic crater]

afaik it just overrides it so probably not

[grave jolt]

I'd overwrite id any time

but no, there's nothing special about it

takes as much memory as any other variable

Generally, you shouldn't worry about how much memory a single variable takes up

[acoustic crater]

if I'm doing min and max I name em min_ and max_ and so on usually

ie with functions that are useful lol yeah I only use id when working with ctypes

[eager trail]

Funnily enough, that was the exact case I found, someone using Id as the name of a paginator iterable

[grave jolt]

well, that's not a very good name for a page number

but if you're naming some id, i'd say it's fine

[eager trail]

Fair do

[eager trail]

I really don't like having i, l or o anywhere near the start of my variable names

[quiet crane]

You should have a font that clearly distinguishes between I and l.

[feral cedar]

are those actually different letters...because

[quiet crane]

yes, the default discord font is a bad example a coding font

[spice pecan]

JetbrainsMono gang

[quiet crane]

Hmm I might have that font actually!

[spice pecan]

It's installed by default in all JetBrains IDEs since 2020 IIRC

[paper echo]

It uses a permissive license too so you can also download and use it in other programs

Lots of nice coding typefaces out there

[visual shadow]

Capital i and small L. 😅

Oops mistagged

[raven ridge]

I will remorselessly use id as a parameter name for a function, since parameter names are part of the public interface and I'm not making my public interface worse because someone made an unnecessary built-in

I think shadowing built-in variables is fine, as long as the scope where you do it is sufficiently limited.

[undone hare]

I often use id_ just for the sake of not having a yellow line in PyCharm

[raven ridge]

Then users need to call your function with your_func(id_=42)

Nope, I'm not making my public interface worse just because someone made id a built-in.

[undone hare]

True

[halcyon trail]

jetbrains mono with ligatures is very nice in pycharm

I enjoy the ligatures for inequalities in particular, looks much nicer than the two characters beside each other

[spice pecan]

del __builtins__.id, problem solved /s
You can disable the shadowing warning for id in particular

[paper echo]

i remember there was some very specific reason someone gave for id being a builtin

or maybe it was for is existing

itd be fun to speculate about a "python 4"

from datetime import DateTime

txt = fp.read()
    with open(filename) as fp
    except FileNotFoundError None

[swift imp]

There should totally be exception handling for context managers

It just makes sense

[paper echo]

well also note the "context expression"

and using indentation to imply line continuation 👀

[swift imp]

There was something on the mailing list about this, and some of the core devs thought it was a good idea

[paper echo]

python 4:

• indentation is valid for line continuation
• = is always an expression
• walrus operator is deprecated and synonymous with =
• inside a function call, = expressions must be wrapped in () to disambiguate from a keyword argument
• naming conventions standardized across logging, datetime, etc.
• json natively handles datetime and time objects
• first-class regex objects with // which desugars to re.compile (but they can be cached/interned when compiling to bytecode)
• with expressions
• except expressions

[swift imp]

I really don't mind the walrus op

[paper echo]

me neither tbh. itd just be nice to have fewer magic symbols

__matmul__ is deprecated and renamed to __specialop__ 😛

[deft pagoda]

pony does this weird thing where it returns the value being replaced in an assignment

a = b = a is how you swap values

[paper echo]

thats.. weirdly interesting

i kind of like the idea of having a special operator for that

= returns the previous value, := returns the newly-assigned value

[deft pagoda]

would actually make a lot of properties that dispatch nicer looking

[paper echo]

what do you mean by "properties that dispatch"

oh also: the option to disable advanced runtime introspection in order to increase interpreter performance

[deft pagoda]

if you have a property that wants to inform other methods or properties that it's value has changed, you might want to return the old value and the new value, or one or the other

[paper echo]

what would be an example (edit: which would benefit from this behavior)

[deft pagoda]

In [2]: print(root)
17
├─6 - 'python'
╰─11
  ├─7 - 'discord'
  ╰─4 - 'data'

In [3]: root.right.right = None

In [4]: print(root)
13
├─6 - 'python'
╰─7
  ├─7 - 'discord'
  ╰─EMPTY

this binary tree will dispatch the change of weight to its parent, and the parent will dispatch to its parent so that weights are updated when nodes are added or removed

[lusty kindle]

* walrus operator is deprecated and synonymous with =

ahahahhaha nice i love it since i never ended up learning how to use them

[deft pagoda]

you could just send the negative of the previous value like, self.parent.weight -= self.weight = val

[lusty kindle]

what does ```py

• with expressions
• except expressions

[paper echo]

ah i love it

[visual shadow]

it's an imagined syntax, doesnt exist

[lusty kindle]

oh, i thought salt was giving actual py4 patch notes 😞

[paper echo]

@lusty kindle my made up syntax:

print( fp with open('foo.txt') as fp )
# prints the file handle object and cleans it up

however i think GCing such a thing would get hairy quickly... you'd basically need to introduce affine types into python

[lusty kindle]

that would be nice

[paper echo]

the big question: when do you run __exit__?

right after the expression is interpreted?

[flat gazelle]

I think it would make more sense as perl adverbs

print(fp) with open('foo.txt') as fp

[paper echo]

ahh i like that idea a lot

[flat gazelle]

though the evaluation order is a bit confusing, 312 is about as bad as it gets

[paper echo]

well if it's a true expression it could be both

the interpreter could "lift" inner with's to the outside

[flat gazelle]

that would work, yeah. not sure how much I like it though. It feels very complex for what you can already do by creating a helper function

[paper echo]

or just using with:\n 😛

[barren aurora]

wait no

i love walrus

its everywhere in my code

[lusty kindle]

i just never bothered to learn it

im sure its nice

[paper echo]

yeah i love walrus too i use it a lot

[lusty kindle]

i mostly use py 3.7 still

work and all heh

[raven ridge]

There's two legitimate use cases I know of for id - one is checking if two objects are identical, and one is for storing metadata associated with an object without keeping that object alive. The former is handled by is, the latter is handled by weakref

I think that leaves 0 uses where id is the best option. Maybe for logging object identity?

[paper echo]

but why is it a builtin

as opposed to being sys.id

[flat gazelle]

what I am really missing is some nice way to fallback to an erroring expression from an except branch, something like

try:
    a = int(v)
except ValueError try:
    a = float(v)
except ValueError try:
    a = custom_identifier_object(v)
except ValueError:
    raise ValueError('input not of valid format')
else:
    unsafe_fun(a)

[deft pagoda]

i use id for creating groups of instructions in kivy

[flat gazelle]

you can do it now with patma, but that requires a whole new set of APIs

[paper echo]

instead of a chain of nested trys?

[deft pagoda]

i could use other things though, probably

[flat gazelle]

yeah, try rewriting that with current syntax, especially if you want to use else

[raven ridge]

I expect the answer, unsatisfying as it is, is that things used to get put in the built-in namespace when there wasn't an obvious module to put them in. I don't think there's any reason it has to be there.

[paper echo]

so then python 4 moves id to sys 🙂

[paper echo]

yeah 😦

[deft pagoda]

filter i use less than id

[paper echo]

@deft pagoda you also write "unusual" python code

[visual shadow]

i've used id for making dictionary mappings against unhashable complex objects

[deft pagoda]

i've also done this

[paper echo]

im not questioning the value of id. im questioning its value as a top-level builtin

also: the syntax for an except expression could be strange

print( float(input()) except ValueError as exc in exc )

[visual shadow]

i'll be perfectly fine with id being moved away to somewhere.

[paper echo]

ooh how about a syntax for only trapping exceptions raised from a specific function!

print( float(input()) except ValueError from float as exc in exc )

this is getting silly but im having fun 😆

[flat gazelle]

filter is more or less useless, id should be moved, that I concur

[paper echo]

im fine with functools.filter and functools.map

[flat gazelle]

I don't see a case in which filter beats a listcomp/generator comp

[paper echo]

when you have a pre-defined filtering function

[flat gazelle]

even then

(a for a in b if fun(a))
``` is pretty nice.

[paper echo]

def complicated_predicate(val):
    ...

filter(complicated_predicate, foo)

eh

thats a stylistic difference i guess

but thats why im ok with hiding it in functools

how about λ being a valid substitute for lambda

[deft pagoda]

i've edited my ipython to do this

[visual shadow]

i did think about that, i wonder if it's rough because it's not ascii

[flat gazelle]

I don't think that is viable since python has to run on platforms where unicode fonts are probably not smooth sailing

[visual shadow]

one strange sequence of events got me stuck in a terminal with vim in it..except the terminal couldn't deal with unicode. (windows..i do hate you sometimes)

[paper echo]

true

[visual shadow]

it was a miserable experience once i tried writing unicode there

[deft pagoda]

# ------------------------------------------
# --- `λxy.` shortcut for `lambda x, y:` ---
# ------------------------------------------

import re

__λ__ = re.compile(r'λ[a-z]+[.]')

def __λ_sub__(m):
    m = m.group()
    return f'lambda {", ".join(m[1:-1])}:'

def _transform_λ(lines):
    return [__λ__.sub(__λ_sub__, line) for line in lines]

get_ipython().input_transformers_cleanup.append(_transform_λ)

if you want to ruin your console

[flat gazelle]

unicode fonts with windows are non-trivial.

[paper echo]

python 4:

• lambda x, y: ... is deprecated. the new syntax is def (x, y): ...

[deft pagoda]

lol

i like the generator syntax for lambdas

(2 * x + y from x, y)

[paper echo]

oh yes baby

[flat gazelle]

inb4

fun = <$0 + $1>

[paper echo]

so the from suffix makes the whole expression a function definition?

[deft pagoda]

you do for too

[paper echo]

what would that be

[deft pagoda]

same, just replace from with for

[paper echo]

hmm i'd only want one

[deft pagoda]

that's what i meant, whichever one people liked more

[visual shadow]

this conversation is making me realise that i dislike any syntax where i have to mentally re-parse a line because the syntactic prompt happens later in line.

[paper echo]

x if p else y is cursed

[deft pagoda]

p implies x else y

[flat gazelle]

yeah, much prefer the F#

[for a in b do if a % 4 then yield a]

even if it is a bit more verbose

[paper echo]

so python 4 syntax changes:

• if ... elif ... else becomes an expression like lambda:, ... if ... else ... is deprecated
• def : becomes an expression like lambda:
• with : becomes an expression like lambda:
• lamba x, y: ... -> def (x, y): ...
• def f (x, y): ... desugars to f := def (x, y): ...
• with [ ... as ] suffix
• except [ ... as ] ... in suffix
• ... from x, y suffix desugars to def (x, y): ...
• indentation as line continuation

something like that 😆

oh also cpython is deprecated, python now compiles to lua

[visual shadow]

just need to remove python from that abomination and then we're set!

see?!

[paper echo]

nahh i like a lot of things about python that i dont want to give up

it's just the syntax can be sometimes really cumbersome

in reality most of my python code looks like this

async def query_entry(
        client: gql.Client,
        entry_type: ContentType,
        entry_id: EntryId
) -> AnyEntry:
    query = build_entry_query(entry_type)
    logger.debug('Query:\n%s', query)
    data, errors = await _execute_async(
        client,
        gql.gql(query),
        variable_values={'entryId': entry_id},
    )
    if (result_raw := data[entry_type]) is None:
        logger.warning('Query returned no data!')
        result = None
    else:
        result = transform_entry(result_raw)
    return result

[grave jolt]

In my latest language which is only half esoteric (it has tests!), a function is created just like in lambda calculus, but without the lambda

[paper echo]

you just use () for grouping? you're halfway to lisp

[grave jolt]

() is just parentheses, like in Python.

[peak spoke]

I like that it's reasonably hard to write completely ugly code in current python, moving more stuff into expressions wouldn't help in that regard

[paper echo]

i agree

im actually quite happy with the state of python syntax

[grave jolt]

yeah, Python is very statement-oriented

[paper echo]

patma will be nice but i think its a stretch and we really should cut it off there

id rather see continued syntactic improvements for type annotations

list[A | B] is a big improvement already

[peak spoke]

I haven't looked at it in depth yet but it looked like something that'll be easy to overuse but can reduce some of the current matching syntax while looking nicer

[grave jolt]

I have recently stumbled upon a (language-agnostic) paradox. When you extract a complex expression part into its own line, you end up with fewer lines.

// before
testPure(
  'create function values',
  '(x. x)',
  {
    ok: rt.Fun({
          fun: { arg: ast.ArgSingle('x'), expr: ast.Name('x'), capturedNames: [] },
          closure: { parent: null, names: Map() }
        })
  }
);

// after
const identity = rt.Fun({
  fun: { arg: ast.ArgSingle('x'), expr: ast.Name('x'), capturedNames: [] },
  closure: { parent: null, names: Map() }
});
testPure('create function values', '(x. x)', { ok: identity });

Couldn't find a recent python example, but it works in many languages

[granite heath]

I quite liked the approach of mapping functions to dict keys, for pseudo patma

[paper echo]

the main usage of pattern matching for me is polymorphism

and destructuring complicated objects

[flat gazelle]

you can't do patma with that, you can just make switches

[grave jolt]

yeah, dict is just a switch

you can't destructure in a dict

well, you can, with horrible hacks, but don't

[paper echo]

pattern matching works for typing.Protocol right? if you add the runtime-checkable feature

[grave jolt]

huh?

[flat gazelle]

if it gets the patma protocol, yes

[halcyon trail]

One of the benefits of having things be more expression oriented instead of statement oriented is that it tends to make things more concise in a type annotated world

when you work with expressions it's easy to infer types. when you create empty variables and mutate them later inside statements, it's more difficult

[paper echo]

import math
from typing import Protocol, Sequence, TypeVar, runtime_checkable

@runtime_checkable
class Point2D(Protocol):
    x: float
    y: float

@runtime_checkable
class Point3D(Protocol):
    x: float
    y: float
    z: float

def _dist(u: Sequence[float], v: Sequence[float]) -> float:
    return math.sqrt(sum((d1 - d2)**2 for d1, d2 in zip(u, v)))

_P = TypeVar('_P', Point2D, Point3D)

def dist(p1: _P, p2: _P) -> float:
    match p1, p2:
        case (Point2D(x=x1, y=y1), Point2D(x=x2, y=y2)):
            u = (x1, y1)
            v = (x2, y2)
        case (Point3D(x=x1, y=y1, z=z1), Point3D(x=x2, y=y2, z=z2)):
            u = (x1, y1, z1)
            v = (x2, y2, z2)
    return _dist(u, v)

@grave jolt like this ☝️

[boreal umbra]

!docs typing.runtime_checkable

[fallen slate]

typing.runtime\_checkable

@typing.runtime_checkable```
Mark a protocol class as a runtime protocol.

Such a protocol can be used with [`isinstance()`](https://docs.python.org/3/library/functions.html#isinstance "isinstance") and [`issubclass()`](https://docs.python.org/3/library/functions.html#issubclass "issubclass"). This raises [`TypeError`](https://docs.python.org/3/library/exceptions.html#TypeError "TypeError") when applied to a non-protocol class. This allows a simple-minded structural check, very similar to “one trick ponies” in [`collections.abc`](https://docs.python.org/3/library/collections.abc.html#module-collections.abc "collections.abc: Abstract base classes for containers") such as [`Iterable`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterable "collections.abc.Iterable"). For example:

```py
@runtime_checkable
class Closable(Protocol):
    def close(self): ...

assert isinstance(open('/some/file'), Closable)

[paper echo]

(not that you would write this)

[flat gazelle]

that makes a Point3D a Point2D, which doesn't seem right to me

[halcyon trail]

How do you figure that?

but also the type signature for dist seems suspicious

[flat gazelle]

well, a Point3D has x: float and y: float, which fits the Point2D protocol

[halcyon trail]

it should enforce that both arguments are of the same type

don't you need to explicitly opt into protocols?

[paper echo]

@halcyon trail yeah i fixed the signature with a typevar. could use @overload too but not important for this case

[flat gazelle]

no, protocols implement __instancecheck__

[halcyon trail]

ah, that's unfortunate

hard to avoid though

or at least, I don't immediately see how to avoid it

[paper echo]

@flat gazelle would you have to put Point3D first in the matching then?

[flat gazelle]

yes

[halcyon trail]

The ideal for this sort of thing is something that's opt in but not (necessarily) intrusive

[flat gazelle]

that is how protocols are supposed to work

[halcyon trail]

python has one thing that's intrusive, and one thing that's not opt in 🙂

[paper echo]

_P = TypeVar('_P', Point2D, Point3D)

def dist(p1: _P, p2: _P) -> float:
    match p1, p2:
        case (Point3D(x=x1, y=y1, z=z1), Point3D(x=x2, y=y2, z=z2)):
            u = (x1, y1, z1)
            v = (x2, y2, z2)
        case (Point2D(x=x1, y=y1), Point2D(x=x2, y=y2)):
            u = (x1, y1)
            v = (x2, y2)
    return _dist(u, v)

i'm ok with that, usually you wont be using "nested" protocols like this

[halcyon trail]

well, that's how swift protocols work

[paper echo]

what would "opt-in" mean here?

[flat gazelle]

like with ABCs, where you have to register your implementation

[halcyon trail]

opt-in means that a class C isn't treated as implementing a Protocol P unless someone specifically says that it does

[grave jolt]

LSP go brrrr

[paper echo]

oh. yeah no that'd defeat the point of a protocol

protocols are specifically to enable duck typing

[halcyon trail]

whether it's the owner of the class, the owner of the protocol, or somebody else (which has its own issues)

[flat gazelle]

that is what an abstract base class does

but a protocol, even outside just the typing.Protocol usage is implicitly present

for example, you don't register your descriptors, they just are of the descriptor protocol if they have the right methods

[paper echo]

same with iterables, context managers, etc

[halcyon trail]

Right, I forgot that in python ABC's can be asked to register types externally

[paper echo]

i love it. its the soul of python to me

[flat gazelle]

duck typing is very much a core of python

[paper echo]

oh also python 4 introduces algebraic effects so you have to annotate your function in order to do file i/o ha ha

[halcyon trail]

the problem is that just because someone provides the right members, doesn't mean that it has the right semantics

[grave jolt]

duck typed algebraic effects

[paper echo]

how would that be possible?

[grave jolt]

[flat gazelle]

we are a dynamically typed language, but yes, accidental subtyping is an issue

[halcyon trail]

I mean this is a perfect example

Say you wrote a norm function that operates on 2D points

it will accept your 3D point

and report a norm for it

which is not the correct norm for a 3D point

[paper echo]

i think algebraic effects are kind of inherently duck-typed? if it does i/o it's an i/o type and an ahead-of-time compiler would require you to handle the i/o effect

[halcyon trail]

now you have a norm function that thinks it can operate on a wider collection of types than it actually can

[paper echo]

@halcyon trail perhaps this is an argument for making protocols "total" in that they do not permit extra attributes?

[halcyon trail]

no

that would be a mess

[grave jolt]

yeah

[halcyon trail]

and not reasonable

[paper echo]

actually yeah i agree

nvm

then "nesting" protocols is a fundamental limitation of structural typing

[grave jolt]

duck-typed algebraic side effects are kind of an oxymoron

[flat gazelle]

protocols work as intended, they just aren't meant for describing attributes

[halcyon trail]

the actual best solution from a static typing point of view, I would say most people agree, is still to make this sort of thing explicit

but allow it to be done non intrusively

[flat gazelle]

they are more for methods and such

[halcyon trail]

that's what you see in Rust and Swift

[flat gazelle]

go is statically typed and has this system, as does pony

[paper echo]

you can have the same problem with methods, e.g. a protocol for a class that has a .foo and .bar methods and another protocol for a class that only has .foo

so does ocaml! 😆

[halcyon trail]

Yeah, Go has this but we're not going to pretend that Go is choosing the best solutions on their technical merits

[flat gazelle]

smalltalk tradition is very strong in this regard

[halcyon trail]

And pony is more irrelevant than Haskell so eh

[flat gazelle]

and keep in mind python already has protocols in place

[paper echo]

idk what happened to pony, did it just disappear

[flat gazelle]

you can just give them names now

[halcyon trail]

it never appeared

yeah, I agree that py thon being python, having protocols makes a kind of sense

It was just a language that people discussed on HN and proggit, I doubt anything was ever done with it in the real world. I came, I laughed at 0/0=0, I left

[paper echo]

actually, how do go and ocaml handle "nested" structural types?

[flat gazelle]

__iter__ and __next__ is a protocol, and it works just like typing.Protocol

pony was used by one startup which I don't think is bankrupt yet

so it is actually proven to not be a toy

[paper echo]

i do think it was kind of a research language that they were trying to shoehorn into a production language setting

[halcyon trail]

Yeah, that ivory tower 0/0 = 0, trying to make it practical

[paper echo]

yes, haskell "escaped from the lab" but that doesn't mean every other research language is going to be production ready

it took haskell like 25 years to be not laughed at as a production language

and that's with a HUGE amount of dev activity over the entire time

[halcyon trail]

btw in Go, I think you could have exactly the same issues as here

If you define Point3D and point2D, then every type that satisfies Point3D will also satisfy point2D

[paper echo]

yeah... i think it would be true in ocaml and crystal as well

[flat gazelle]

on another note

from typing import Protocol, runtime_checkable
from types import SimpleNamespace as NS

@runtime_checkable
class Safe(Protocol):
    key: str

@runtime_checkable
class Safer(Protocol):
    key: str
    salt: str

def validate(a: Safe | Safer):
    match a:
        case Safer(key=key, salt=salt):
            print(key, salt)
        case Safe(key=key):
            print(key)


validate(NS(key='a', salt='b'))
validate(NS(key='a'))
``` works as expected

ye, elm would have the same thing as well

as would purescript and smalltalk

[halcyon trail]

OCaml, really?

That's a surprise

[flat gazelle]

OCaml modules work exactly like protocols

[paper echo]

elm and purescript have structural typing?

neat

also i assumed smalltalk was entirely dynamic

never used it myself

https://journal.infinitenegativeutility.com/structural-types-and-duck-typing

[flat gazelle]

yeah, smalltalk is entirely dynamic, but you can still document a protocol

[paper echo]

iirc ruby is heavily inspired by smalltalk right

[flat gazelle]

yup

most dynamically typed OOP languages are

well, most OOP languages in general

[paper echo]

right

but ruby really has the "everything is an objecting" thing

[halcyon trail]

ah, modules

[paper echo]

i know that for example in crystal, "global" definitions are just attributes/methods on a special object Program

i assume that's based on some equivalent in ruby and it sounds like what i hear people say about smalltalk

[flat gazelle]

yeah, that is definitely from smalltalk

SmallTalk is a global in smalltalk

[paper echo]

crystal is a very fun language so far btw

i highly recommend playing with it

i was joking about algebraic effects before, but https://effekt-lang.org/ exists and it also uses ruby-like syntax, i feel like it's not a small leap to incorporate algebraic effects in this style into a language like crystal (once the research progresses further and more issues get sorted out)

interestingly, this particular implemetation of algebraic effects does not allow for first class functions, which also seems like a great fit for crystal which already does not have first class functions. something about referential transparency, idk

[flat gazelle]

yeah, ruby has a similar issue

[paper echo]

i kind of like how unapologetically imperative it is

the more functional programming i do, the more i feel like im dicking around with math and less like im bossing my computer around. i am coming to prefer the latter in at least some cases

"computer, will you pretty please map this into the functorial context"
versus
"computer, i demand that you perform file i/o right now"

theres a virgin vs chad meme in here

or the "average X enjoyer" meme with the muscley guy

[spice pecan]

Average functional programming fan vs average imperative enjoyer

[flat gazelle]

I do wonder what features will become mainstream as people discover more things. Dependent typing in a way that isn't purely a research idea would be really cool, as would an Ada like type system on a language that isn't Ada. Will pep 638 become reality and will python get macros, will python get null coalescing?

[paper echo]

ive been messing with idris 2 a lot. i dont think dependent typing will be a thing for a long time, its too reliant on the user being able to jot down proofs, and/or too reliant on every theorem in math being already embedded in the compiler

some combination of refinement types and gadt's will do for 99% of cases i think

i think idris 2 has a lot of potential for "practical dependent types" but it still feels very haskell-ey in how things get both abstract and mathematical faster than you'd expect

[grave jolt]

dynamically typed Python will do it in 99% of cases

💥 💥 💥 🔫 shots fired

[paper echo]

what is an ada-like type system? i dont know anything about ada

[grave jolt]

@flat gazelle maybe you're confusing with agda?

[paper echo]

python macros would be dope, they'd just be functions that operate on ast objects but with defmacro instead of def

[flat gazelle]

no, Ada, agda is just dependent typing

[grave jolt]

ah ok, I am confused now

[flat gazelle]

essentially, very strict rules, as well as the core feature of distinct types.

[halcyon trail]

It's definitely pretty interesting to see the newer generation of languages borrow a lot of things from functional languages

but also ignore others

[grave jolt]

like crazy custom operators?

[paper echo]

raku would like to have a word

[halcyon trail]

yes:

• really nice lambdas/lambda syntax are clearly a huge win
• pattern matching is very good
• good type systems
• more control over mutability, less mutability by default

no:

• inability to early return from functions
• currying
• purity, and/or laziness
• immutable pretty much everything

[grave jolt]

( >>=[+++Ж+++>>=()]> ) = unsafePerformIO . launchMissiles

[paper echo]

@halcyon trail personally i am VERY happy with this situation

[halcyon trail]

yeah, me too

[grave jolt]

In Haskell you can do an early return with MonadFail

[flat gazelle]

Essentially, ```ada
type A = new range 1..10;
type B = new range 1..10;
procedure Act_On_A(a: in A) is
begin end Act_On_A;
procedure Main() is
C: B;
D: A;
begin
C := 4;
D := 4;
Act_On_A(C); -- incorrect
Act_On_A(D + 4); -- fine
end Main;

[grave jolt]

well, not really

but sort of

[halcyon trail]

I'm not sure about the type system rabbit hole stuff. The thing is that it's endless, it just never stops

[paper echo]

its an active area of research. Quantitative Type Theory is what, 5 years old?

algebraic effects are still solidly "research language" territory

[halcyon trail]

and eventually the intellectual overhead in understanding the type system isn't actually a win versus having a simpler language and having something verified in testing

[flat gazelle]

the balance between a type system that doesn't stop you from writing correct code, yet is simple enough is a hard one

[halcyon trail]

Yeah, very difficult

[paper echo]

this is refinement types, no?

[halcyon trail]

You also start to hit, IMHO, very hard diminishing returns in 99% of domains

because at some point most of your bugs are that you didn't understand the spec properly

[flat gazelle]

no, the core is the new operator

range is just a type in ada

[halcyon trail]

and no type system can save you from that

[paper echo]

so A and B are distinct types? what's the point of that?

[flat gazelle]

you can't use an A in place of a B

[paper echo]

yeah but what's the point

[flat gazelle]

useful for example if you have a width unit and an angle unit

[paper echo]

ah

[flat gazelle]

both would be a float, but are completely different

[grave jolt]

Yeah, a system of axioms cannot prove its correctness. So all you might have done is prove that the implementation is internally consistent

[flat gazelle]

it is typing.NewType IIRC

[paper echo]

i was about to say

you can do that in python!

[halcyon trail]

In C++ of course you have things like boost units which go quite a step further with dimensionality

it will keep track of multiple orthogonal unit quantities, typically length, mass, time

and verify that everything exactly agrees at compile time

and correctly propagates dimensionality through multiplication, addition, etc

[flat gazelle]

julia I think also does that, but it doesn't have a static type checker, which makes it somewhat less useful

[halcyon trail]

i would be curious to see a language with a type system roughly comparable to Rust's, but GC based, and so without all the lifetime-affine stuff

but the other nice things, like pattern matching, traits

maybe Swift is roughly the closest out of popular languages

[flat gazelle]

I feel like F# is close enough

[halcyon trail]

I should try F# at some point probably

[flat gazelle]

or maybe not, seems like I misunderstood it somewhat

[spice pecan]

F#'s Measure is pretty nice

Though I do sometimes wish you could do more with it

[paper echo]

does julia have support for dimensioned/unitful numbers?

[halcyon trail]

julia is dynamically typed so it's not that exciting per se

I mean it's easy enough to support dimensioned numbers that are dynamically checked, with overhead

[paper echo]

kinda? if you annotate a function to only take certain types, it wont magically make an untyped version for you

f(x::Float64, y::Float64) = 2x + y

f(Float32(2.0), 3.0)
# ERROR: MethodError: no method matching f(::Float32, ::Float64)
# Closest candidates are:
#   f(!Matched::Float64, ::Float64) at none:1

also theres https://juliapackages.com/p/unitful

[halcyon trail]

what do you mean by "kinda"

you're saying that julia is "kinda" dynamically typed?

[flat gazelle]

it is dynamically typed with syntactic sugar for type guards and static dispatch, similar to common lisp.

[paper echo]

the "kinda" is that it's only "kinda" not exciting

[halcyon trail]

I'm only saying that "supporting" dimensioned/unitful numbers dynamically, specifically, is not exciting

you can do this in any language pretty much, including python

you just have a class that holds teh quantity + your dimension information, and define the various operators for it correctly, raising errors for any operations where the dimensions cannot work out

[paper echo]

im not sure why that isnt exciting though. if it's part of the type, and it's a subtype of Float, then you can't pass a Float there

it's the same guarantee as in a static type system. you'd get a "no method matching" error in julia, and in python with NewType the static checker would flag it (mypy, pyright, etc)

[halcyon trail]

it's not the same guarantee as a static type system, because in a static type system you get the guarantee without executing the code

[paper echo]

fair. although i think static/ahead-of-time type checking for julia is an ongoing research project

[halcyon trail]

Yeah, the point is though that what's exciting here is being able to express such things, well, within a static type system

In C++ you can express this well because you can have integer values as part of the type, and you can even do most of the operations you would expect on such integers and incorporate them into new types

[gleaming rover]

Scala?

[acoustic crater]

I feel like I recommend Haskell a lot despite only spending a couple weeks with it but ppl keep talking about stuff it does great... Haskell is statically typed af and you can easily define types and functions and functors and so on according to types and their behavior is based on either your explicit typing information or whatever the compiler determines the typing will be

and yeah you saying "such and such is part of the type and you can perform this operation on it" just makes me think of FP especially haskell cuz that's what I know

[acoustic crater]

is there a way to see what type built-in methods return? They have no annotaton when I use inspect.signature

[charred wagon]

I doubt it's possible in CPython.

[acoustic crater]

makes sense, thanks

I guess they won't ever change

so dynamic checking only helps if any are added

could throw stuff at them and see what comes out but that seems hacky

[raven ridge]

Have you seen typeshed?

It includes type stubs most built-ins

@acoustic crater

[acoustic crater]

ah nice

[swift imp]

How many of the design patterns in gang of four actually apply to python? It says to dodge a lot of common things in python, like mixins

[acoustic crater]

https://youtu.be/G5OeYHCJuv0 this talk goes into that a little bit (the title is weird)

basically, the gang of four didn't write for python

[raven ridge]

There's an argument that design patterns are indicative of language shortcomings. They're a problem commonly faced by users of a language, and are so common that it is in everyone's best interest to establish a common pattern for solving that problem, so you can quickly recognize when someone is solving that problem, and quickly tell if they're solving it right

if the language made something easy, you wouldn't call it a pattern, just a feature.

e.g. Java has the iterator design pattern specifically because Java does not have iterators as a first class concept. Python has iterators as a first class concept, and we have what we call the "iterator protocol", but we don't call it a design pattern, because there's only one (reasonable) way to do it.

likewise, the singleton pattern is, at its core, just a pattern for creating global variables in a language that is designed to not have global variables.

[acoustic crater]

I wonder though how you'd ensure a low-coupling high-cohesion pattern like bridge is enforced at a level of what's possible in a language

or at least syntactically enforced, python is the extremely permissive but the most permissive elements are clunky enough to make you think "is this the best way?" but how would you enforce low coupling and high cohesion, since those are based on what you're implementing

a lot of rly nice things are part of python instead of design patterns tho like context managers

[raven ridge]

you could maybe make the argument that mixin classes are an alternative to the bridge pattern. Maybe. 🙂

not quite, but sorta. In any event, I don't think the "design patterns as language smells" necessitates the existence of some language without those smells. It's possible all current languages have that same limitation; it's not a value judgment about a language to say that there are common problems that don't have easy solutions; it just means that the language prioritized other things

[acoustic crater]

makes sense

I've been trying to wrap my head around OOP in Python lately and I prefer to use the bridged class as a parameter and if it's many classes being bridged into instantiation I use instance factories, though I guess a factory could generate the class itself with whatever mixin

but it seems better to have a bridged class because the methods are clearly of that class and you aren't calling self.method where the method is part of the mixin, normal inheritance is unclear enough so decoupling with mixins seems not that clear at all

also class factories, like namedtuple, seem like they are best left for metaprogramming

but yeah this isn't as much of an issue in FP languages and so on

[buoyant thicket]

Is there anything Python is doing to increase its speed.
Many people are avoiding python as it known to be one of the highest level programming language. Many companies are slowly dropping python especially the backend side.
Also isit possible that Python will be able to be compiled normally and in the process exclude files that arent required In a program?

[unkempt rock]

slots

@buoyant thicket I'd love to see where you get your metrics on the whole dropping python thing.

Depending on what you're doing, speed is very relative.

[buoyant thicket]

Backend?
I recently made a backend in Python and it took 4 seconds on average to complete

[unkempt rock]

I can write an IDS in python that keeps up with "Enterprise Firewalls"

It depends on what your traffic is for that Enterprise though you know?

Python is, was and will be "A highly useful language of first choice" for many a programming problem for many a year to come.

[buoyant thicket]

highly slow as well

[unkempt rock]

Uhm

idrop is not slow =0

But this aint a pissin contest

[buoyant thicket]

Dart is coming up really fast
it's 12x faster than python
||don't quote me on the number||

[unkempt rock]

So?

This chan is about Python right?

If you don't want to use Python, don't.

Don't crap all over the lang just because shiny is shinier

[buoyant thicket]

I was asking if they were gonna do smth abit

[unkempt rock]

They did

slots

20%

OFF the rip

[buoyant thicket]

I don't this conversation is getting anywhere

[unkempt rock]

Like I said, I can write an IDS which keeps up with C for Enterprise performance. it just depends on the level of traffic before C gets the best of me.

After all, you can only spew X packets in Y seconds right?

So if I keep up with X in Y with python, then why care about C?

etc.

I'll do you one better

I'm sure this breaks some rule I didn't bother to take into account, but this perfectly sums up my answer to you

https://github.com/stryngs/flowTester

^ I wrote flowTester because I love me some 802.11

For 802.11 you have two main giants:
aircrack-ng
kismet

aircrack-ng is C or C++, I forget

Prolly C

kismet is ruby?

Either way, none are Python

So along came a scapy

Dare I say it, scapy when done right using slots and whatnot inherit to Python3 will keep up with the aforementioned

What matters is the X in Y algorithm you're trying to solve.

I don't care if you can do 8000000000000 x in Ys

Not if the throughput is only 3000 x in Ys

For that test, then all we care about is the bottom floor handles 3000 x in y

If python meets or exceeds that, then for all purposes, why the heck not use it?

I may not survive a defcon packet shootout, but I can sure survive a corporate workplace packet shootout

etc.

viva la Scapy.

[acoustic crater]

python isn't best for everything but it's a pleasure to write and can do a lot

[visual shadow]

This part, let's just say the language should be the last thing to blame. You do have to ensure you've done your optimization and the code logic is efficient.

[unkempt rock]

^

[visual shadow]

This part.. The companies and people who avoid python because python is truly a hindrance would be perfectly fine to do so. Because python can't be a one size fits all, it's got its strengths and weaknesses. Even more companies are picking python up for the same strengths python has.

[peak spoke]

Improvements are always being implemented, but for anything larger the interpreter core will have to be remade which will take a considerable amount of work from the contributors and probably won't be done without financing for it. But no matter what you do it can't be as fast as say C with all the language features in place

[unkempt rock]

^^^^^^^

It's all about the X in Y tradeoff 🙂

Flying to Mars? yeah, C is prolly in order.

Rocking a drone 1 mile away? Python that bad boy out.

[flat gazelle]

pretty sure the code running on earth for some space device was almost entirely python.

[visual shadow]

The two fundamental things people don't understand when talking about speed. 1. Speed isn't a gain in isolation, there's usually very fundamental tradeoffs involved. 2. Speed isn't the only race a program should solve. As long as something is "fast enough" to meet the needs, any extra speed is a bonus.

Chasing speed for the sake of speed is a silly thing to do, because you could always keep going down the rabbit hole

There's a good reason we moved on from writing assembly, because most use cases don't need to.

[acoustic crater]

I care more about writing code speed haha

...and a big part of that is how fun it is

[peak spoke]

For example 3.10 will bring some nice optimizations https://docs.python.org/3.10/whatsnew/3.10.html#optimizations

[flat gazelle]

very few tasks are so easy that they can be done in a language other than python, but too hard for python

[unkempt rock]

Constructors str(), bytes() and bytearray() are now faster (around 30–40% for small objects). (Contributed by Serhiy Storchaka in bpo-41334.)

Wow

So pretty much a lot of my code boosts just by an apt-get upgrade 🙂

ChooChoo

[raven ridge]

That's not for creating those objects using literals, only for calling the constructors

[unkempt rock]

Oh

f = 'h' << you're saying this isn't sped up?

more the thing that makes str() @raven ridge ?

https://www.geeksforgeeks.org/constructors-in-python/

"""
Constructors are generally used for instantiating an object.The task of constructors is to initialize(assign values) to the data members of the class when an object of class is created.In Python the init() method is called the constructor and is always called when an object is created.
"""

So yeah, f = str() just got faster no ?

[raven ridge]

If you don't call the str() function, that speedup doesn't apply to you.

x = str(1) is getting faster from it, x = "1" isn't

[unkempt rock]

o0

Okay

Interesting that str() doesn't apply to x = "1" in that sense though?

Would str() be called in the backend of it so to speak?

I'm no CS major by any means

just kinda guessing.... after all, x = 1 vs x = "1" has to define itself somewhere

[acoustic crater]

a literal is just the object to the interpreter, "casting" to the object by instantiating has overhead and that's been reduced

[flat gazelle]

x = "1"
``` will create the string at compile time, so it is not really instantiated anywhere where it makes a difference.

[unkempt rock]

Ok

[raven ridge]

It's already much faster than calling str

[acoustic crater]

cuz str cna be called on objects being presented to it at runtime

literals are always the same

[raven ridge]

What's changing is that str is getting faster, but it will still be slower than literal strings.

[unkempt rock]

f = str('a') will never be as fast as f = "a" << in a nutshell?

[acoustic crater]

ya unless the compiler cna deterministically make str calls a literal

[raven ridge]

Right. It can't be, it's defined to do strictly more work.

[acoustic crater]

which is unlikely

[unkempt rock]

Okay, thats still dope though, 40% or whatever

I use str() a decent bit, but thank you for the clarification, I'd have sounded like a jackass come monday 🙂

[acoustic crater]

haha learning the jargon for everything is tough

[unkempt rock]

^

[acoustic crater]

I'm in CS II rn and the instructor calls instance methods class methods

I cringe a lot

[unkempt rock]

I'm the worst kind of teacher, rather than define what a method is... I use reckless and loose terms. "Magic Powers" suffices sometimes for demo purposes

I'll take the bait @acoustic crater

[acoustic crater]

but he's old and about to retire I'm not gonna make his life harder

[unkempt rock]

Instance vs class method, define?

[acoustic crater]

instance method gets passed self and is instance specific

[night current]

hi guys

[acoustic crater]

class method gets passed the class and is class specific

[unkempt rock]

I code, but I'm self taught, and so the finer things in life I never learned rote memory wise

[night current]

whats up?

[acoustic crater]

__new__ is always a class method because it takes the class and returns an instance of it, __init__ is always an instance method because it initializes stuff for a specific instance

[unkempt rock]

So instance methods happen when something is instantiated and a class method is when you instantiate the class itself as a "parent" per se?

i.e.

[acoustic crater]

ya if you dont wanna explicitly pass in the self/class parameters

[unkempt rock]

  def __init__(self, bob):
    pass

f = foo()

[acoustic crater]

you can call an instance method on an uninstantiated class... but you need to pass in an instance of a class as the first argument to be self

for example

[unkempt rock]

You'd be fun to learn from =()

[acoustic crater]

lol i think I'm starting to get enough of a hang of it

[unkempt rock]

Sounds like it

[acoustic crater]

I'm self taught too but a huge nerd about the minutae of python (minutiae*, googled it)

[unkempt rock]

^ not so much the minutae myself

But love to entertain it as Python is my lang that pays the bills?

I got a masters in Google-fu, haha

I question everyday whether docstrings alone would help me understand a python lib

[acoustic crater]

same haha I help ppl with C# and C++ a lot just by googling

[unkempt rock]

splain though plz

so f is an instance method above?

or is

f.blah = 1 the method?

[acoustic crater]

no you called a class so you constructed it

which means both __new__ and __init__ are called

as part of construction

[unkempt rock]

call as in type it out but not even do anythign with it? Or do you mean by instantiating?

i.e.

class bob():

pass

[acoustic crater]

call as in ()