internals-and-peps

[rugged merlin]

it?

[unkempt rock]

Yes, that's it!

It should work now

[rugged merlin]

wtf

i don't know why it doesn't work

[unkempt rock]

Oh, and add a ":" to the end of the if.

if otvent == rand:

Sorry

[burnt girder]

Good morning everyone, I have this issue in my app Development process. So I have been able to set up signing with Apple, there is also a manual sign in with email and password. I want to be able to identify a user by their device, so that one IOS user has one account. Is there any thing unique to each apple device, that can be gotten programmatically

[unkempt rock]

@burnt girder Could you post your code?

@rugged merlin Does it work now?

[rugged merlin]

no

im on lesson and i don't know what i should do with it

[unkempt rock]

Oh, hang on, the if and else are indented to the right. Remove the indent.

[rugged merlin]

sry for my bad english im from Russia

[unkempt rock]

It should look like this:

[rugged merlin]

thx man

[unkempt rock]

otvet = int(input())

if otvet == rand:

[rugged merlin]

I will hand over the work and most likely they will give me 5

[unkempt rock]

Not

if otvet == rand:

[unkempt rock]

Yes!

[rugged merlin]

thank you so much

[unkempt rock]

You're welcome and your English is great mate!

I'm British

[rugged merlin]

Goodbye and good luck!

[burnt girder]

@unkempt rock it's not about the code, I want something unique to be able to map Apple device to the user. the Apple device id is not unique

[unkempt rock]

What information can you get from the user's phone? You can get the device id, but what else?

[gleaming rover]

this isn’t really appropriate for this channel

[unkempt rock]

in which app i can code in pc

im new

at coding

[gleaming rover]

try #python-discussion or #tools-and-devops

[unkempt rock]

ok sir tq

[jagged thunder]

You can use Sublime Text 4 or Microsoft Visual Studio Code, they both support multi-programming language

[real musk]

Hi where can I post job openings please?

[burnt girder]

@gm okay thanks

[unkempt rock]

ok sir

[jagged thunder]

your welcome

[grave linden]

"that's good my boy lol" - Programmer_alpha

[tidal marten]

Okay, so I finally come up with some sort of usable code which basically boils down to this:

@dataclass(frozen=True)
class Rule:
    ...


class ActionSet:
    __rules__: dict[str, Rule]

    @classmethod
    def prepare(cls, locals: dict[str, Any]) -> dict[str, Rule]:
        locals['__rules__'] = dict(
            (k, v) for k, v in locals.items()
            if not k.startswith('_') and isinstance(v, Rule))

    def exec(self, ...) -> ...:
        for _, rule in self.__rules__.items():
            # match with best rule here
            ...

        # maybe return best match or something afterwards
        ...


class MyActionSet(ActionSet):
    RULE1 = Rule(...)
    RULE2 = Rule(...)
    RULE3 = Rule(...)
    RULE4 = Rule(...)
    RULE5 = Rule(...)

    ...

    ActionSet.prepare(locals())

As a bonus, I can also use each Rule in MyActionSet as some kind of enum

I was going to do:

    __rules__ = ActionSet.prepare(locals())

But I added some other declaration, then deleted it. This stuck and I don't bother changing it back.

[halcyon trail]

Still so confused by this code

It looks like you're populating the dunder rules variable with the rules you declared

There's no reason to do this, you can just have an enum of rules,.and iterate it at will

[tidal marten]

I want to iterate it in the order the rules were declared though

[halcyon trail]

Enums do that already

Enumerations support iteration, in definition order:

[astral gazelle]

hello friends
could anyone explain why NoneType isnt accessible in py >2.7 and <3.10?

>>> type(None)
<class 'NoneType'>
>>> NoneType
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'NoneType' is not defined

whats the reasoning for having it in types, then removing it, then adding it back again in 3.10

[peak spoke]

They were readded because of type checkers as they're not really accessible for them without it https://bugs.python.org/issue41810

[halcyon trail]

I have wondered this too before

I like that it's being readded, currently for NoneType specifically, you don't need it because you can just use None anywhere a type is expected

But this feels silly

[peak spoke]

3.0 removed them along with types like ListType which are already easily accessible, but that left them without a way to access them except from getting the type from the objects manually

[tidal marten]

Well, that's an afternoon going to waste...

[tidal marten]

Enum cannot use the same value more than once though?

I reused some rules as aliases, e.g. RULE10 = RULE3

[halcyon trail]

you can have the same value twice, but you can't have the same name

[tidal marten]

Not sure if I want to be able to get both names, but might be useful

[halcyon trail]

afaik

https://docs.python.org/3/library/enum.html#duplicating-enum-members-and-values

[tidal marten]

So in my iteration, I might want to get both RULE3 and RULE10 as members

[halcyon trail]

This is covered explicitly in the enum documentation

and so was your last issue about in-order iteration

you should probably read the enum docs pretty carefully

it has an explicit example with duplicated values, and it explains exactly what the rules are

[tidal marten]

Oh yeah, I can use __members__

Well, that's another rewrite in order

[halcyon trail]

Yeah, although given your example with ActionSet.exec though there's no reason to iterate over aliases

[tidal marten]

So I probably should make MyActionSet subclass ActionSet and Enum, in that order?

[halcyon trail]

There's no reason for you to have two classes

Just have ActionSet which subclasses Enum, that's it

[tidal marten]

Why not? I want to have 1 base class that contains all my functions, and 1 derived class that contains all the rules declaratively-defined

[halcyon trail]

because that base class doesn't hold state, in python there's rarely a good reasonto have such a class

[tidal marten]

Maybe as a mixin?

[halcyon trail]

just have functions

There's no good reason to add complexity here

[tidal marten]

Actually, the base class does hold state since I will put my rule matching logic there, including a buffer, a buffer pointer, and some other states that I want to track

There's some stuffs outside the exec

[halcyon trail]

Okay, still not clear it should be a class and not a function

but even then, there's no reason to use inheritance

[tidal marten]

I guess so I don't have too much clutter

[halcyon trail]

i don't understand what's increasing/decreasing clutter here

[safe hedge]

This seems like you are reducing "clutter" by massively increasing complexity

Also, would you not be better off making a dict of rules as opposed to having them as attributes?

[halcyon trail]

Ideally, I would shoot for something like this

@dataclass(frozen=True)
class Rule:
    ...

class ActionRule(Enu):
    RULE1 = Rule(...)
    RULE2 = Rule(...)
    RULE3 = Rule(...)
    RULE4 = Rule(...)
    RULE5 = Rule(...)


def match(s: str) -> ActionRule:
    ...

def match_generator(buffer: str) -> Iterable[ActionRule]:
    buffer_index = 0
    while buffer_index < len(str):
        ...
        yield(match(some_substring))

[safe hedge]

Considering you effectively end up doing that with __rules__

[halcyon trail]

no inheritance, no second class

if you want to get all of the rules from some larger buffer, which is kind of what you implied with the buffer/buffer-pointer, then use a generator

If you really have more complex needs, and a generator isn't sufficient, then you can have a second class class RuleMatcher or whatever, and again, no inheritance relationship

[tidal marten]

I guess making the rules as attributes are kinda useful since it can be type-checked. I'm considering to use some sort of generator function, but I might need to have backtracking functionality. Hence, the use of class.

[halcyon trail]

@safe hedge that's an old version of his code before he realized he could just use an enum

[safe hedge]

I know, but even then why need an Enum

[halcyon trail]

Gotcha. Well, if you need a class, then use a class, but again, there's no reason for inheritance.

[safe hedge]

Seems like it could really just be a dict

[halcyon trail]

why is a dict better than an enum?

[safe hedge]

You can have a TypedDict

[halcyon trail]

If you have a fixed set of rules that you never add or remove to

[safe hedge]

It's not necessarily better I guess

But it just seems like one less overhead to me

[halcyon trail]

I don't see that at all

Enums and Dicts are two different things

[safe hedge]

But what benefit of an Enum is he using is I guess my question

[halcyon trail]

Using an Enum gives you a type that corresponds to a finite set of values

[safe hedge]

It seems like he just wants a set of rules collected together?

[halcyon trail]

if you use a Dict, then you don't have as good of a return type for match

and TypedDict's themselves should not be the first choice in new code, in the vast majority of cases

they serve a couple of specific functions, but just using them in a typical problem is weird compared to using a proper class

what the Enum lets you express is that in certain contexts, you're working with exactly a finite set of e.g. 10 rules that are possible. As opposed to all of the possible values that I can instantiate the Rule class with.

A dict or even a typedict doesn't express that anywhere near as nicely

[tidal marten]

When exec is run, the result generally will also have reference to the specific rule so I can do comparison like e.g.

if result.rule is MyActionSet.RULE1:
    ...

I tried subclassing both my base and Enum, but my base class didn't receive __member__ at __init__ so idk if this can work

[halcyon trail]

there's really really no reason to use inheritance here

sorry, except for inheritance from Enum obviously, because that's how it happens to work

[tidal marten]

I guess I can pass MyActionSet to the base class (probably renamed like ActionMatcher) as parameter

I still need that backtracking capability

[safe hedge]

Can I suggest moving this to a help thread and showing more of the code?

[halcyon trail]

why base class though...

you mean the other class?

[safe hedge]

I think he means the class with exec on it. Which as you say seems like it could be a function

[tidal marten]

The ActionSet class is the base class, yes

[halcyon trail]

there's no base class here except Enum (or rather, there shouldn't be)

[safe hedge]

Again I think we should jump to a help thread as we've deviated a bit now

[tidal marten]

It's just so I can pass MyActionSet as a paremeter to another class that accepts ActionSet

[halcyon trail]

You could have something like this:

@dataclass(frozen=True)
class Rule:
    ...

class ActionRule(Enu):
    RULE1 = Rule(...)
    RULE2 = Rule(...)
    RULE3 = Rule(...)
    RULE4 = Rule(...)
    RULE5 = Rule(...)


class ActionMatcher:
    def __init__(self, buffer, ...):
        ....
    def exec(self) -> ActionRule: # or maybe some other return type
        ...

these are two distinct concepts, the set of rules you allow, and the class responsible for doing some kind of stateful matching operation involving strings and rules

[tidal marten]

Yeah, I might need to rewrite some of the stuff here

[halcyon trail]

And yeah, welsh is right tbh, this isn't really advanced discussion at this point, probably fits in help better

[tidal marten]

My ActionSet class does not work as a mixin, so I need to break the two classes apart

[halcyon trail]

I'd really strongly recommend keeping it simpler

To be blunt, for the amount of trouble you're having using Enum, you shouldn't be worrying about things like mixins at all

just write simple functions and classes

[tidal marten]

Yeah, it's just the code that I ended up with

I was just playing around with class definition and stuffs

[tidal marten]

Huhh, after I think about it. Enum might not be a best since I need to use ._value_ to get the actual rule value

[halcyon trail]

It's just .value

[paper echo]

what in the world

In [10]: class Thing:
    ...:     def __init__(self, x):
    ...:         self.x = x
    ...:

In [11]: class Things(Thing, Enum):
    ...:     A = Thing(1)
    ...:     B = Thing(2)
    ...:

In [12]: Things.A
Out[12]: <Things.A: <__main__.Thing object at 0x10d15eee0>>

In [13]: Things.A.x
Out[13]: <__main__.Thing at 0x10d15eee0>

In [14]: Things.A.value
Out[14]: <__main__.Thing at 0x10d15eee0>

In [15]: Things.A.value.x
Out[15]: 1

[halcyon trail]

And whether you have to do .value or not doesn't really anything to do with whether you'd want to use an enum

[paper echo]

In [16]: Things.A.x.x
Out[16]: 1

uhhhh

[halcyon trail]

🤷‍♂️

do weird things, expect weird things

but it's a great illustration of why you shouldn't randomly use inheritance 🙂

[tidal marten]

The only reason I did it this way was because I want to define all the necessary state/function in the ActionSet class then set all the expected rules in the MyActionSet so I can immediately just use MyActionSet instead 😂

Changing to Enum requires me to iterate over the enum at ActionSet init (instead of MyActionSet declaration) and check whether the type of the enum value is actually Rule (this is since I use tuples to group some of the rules for use later, but I don't want to include them as part of my ruleset for matching)

So yeah, more runtime checks than necessary

But the difference is only 13 insertions, 22 deletions

[halcyon trail]

I don't understand how you're going to get more runtime checks one way than the other

If you really want to check things at top level, i.e. at module initialization instead of at instance creation, you can do that by just calling a function at top level

again, this kind of choice is totally independent of Enum

[tidal marten]

My changes basically boil down to this:

@dataclass(frozen=True)
class Rule:
    ...


class ActionSet:
    def __init__(self, rules: Type[Enum], ...):
        self._rules = {k: v.value for k, v in rules.__members__.items() if isinstance(v.value, Rule)}
        ...

    def exec(self, ...) -> ...:
        for _, rule in self._rules.items():
            # match with best rule here
            ...

        # maybe return best match or something afterwards
        ...


class MyActionSet(Enum):
    RULE1 = Rule(...)
    RULE2 = Rule(...)
    RULE3 = Rule(...)
    RULE4 = Rule(...)
    RULE5 = Rule(...)
    ...
    ALIAS1 = RULE1
    ...
    COMBINATION1 = (RULE1, RULE2)

Previously, the self.__rules__ are generated at declaration instead. Enum is a bit finicky, but at least it caught duplicate rule names so that's neat.

[paper echo]

well there's the special case of Things(str, Enum), i tried Things(Thing, Enum) as a guess by analogy

@tidal marten does _rules have to be a dict? why not a list (or set?) of Rule objects

[halcyon trail]

what's TokenRule?

Did you mean Rule?

[tidal marten]

Yeah, Rule

[halcyon trail]

Yeah, I mean I'm not sure what you're doing, this may be ok, it's a bit unusual to have combinations of rules as members of an enum

more typically you have an enum, and an enumset, and an enumset can contain any subset of the enums

[tidal marten]

It can be a list. I just haven't figured out if I will need to have the name later down the road, so I just kept it a dict.

[halcyon trail]

but if only certain combinations are possible then that could be ok

That said, I would probably have two separate enums

class MyActions(Enum):
    RULE1 = Rule(...)
    RULE2 = Rule(...)
    RULE3 = Rule(...)
    RULE4 = Rule(...)
    RULE5 = Rule(...)
    ...
    ALIAS1 = RULE1

class MyActionSets(Enum):
    COMBINATION1 = (MyActions.RULE1, MyActions.Rule2)
    COMBINATION2 = ...
    ...

the point is that now you don't need to worry with populating self._rules at all, you can always decide whether to iterate over one enum, or the other, or both

[tidal marten]

Is there any metaclass that I can use to get an ordered dict of Rule instead, since I only use MyActionSet to hold all these values anyway.

[halcyon trail]

oh boy

[tidal marten]

I literally just want to do this declaratively so that I can get type checking lol

[halcyon trail]

I just showed you how though

If all you want is to be able to iterate over a specific set of Rules, then you can just have a List

[tidal marten]

But I want to refer to the rule as a named attribute though

[halcyon trail]

okay

[tidal marten]

So exec is supposed to output a list of Result which is defined as

class Result:
    rule: Rule
    value: Any

So that later, I can iterate the list of Result and check if result.rule is some attribute like MyActionSet.RULE1

Or if result.rule is in MyActionSet.COMBINATION1

[halcyon trail]

well, that's fine, you can just annotate it

however, it does seem like mypy is pretty conservative about type checking enums

the problem basically is that enumerators don't all have to thave the same type, so mypy is very conservative when checking enum iteration

if you have a bunch of things that have the same type, that would be a collection

Do you ever actually access the rules by the names RULE1, RULE2, etc

it looks like you only ever iterate over them

oh, I guess for defining combinations

[tidal marten]

Yeah, later. Since this is for tokenization that I wrote for a parser. I can't handle raw strings anymore so I reckon giving tokenization a go. That's why I want to match for the Rule or combinations.

[halcyon trail]

and just to verify, only very specific combinations of rules are valid?

[tidal marten]

The combination is just for shorthand. For example, if I have 5 different way to represent numeric values, I'll just group them into a single combination tuple. So I can check whether a result has a rule in a combination or be more specific and check against the actual Rule attribute

[halcyon trail]

not sure I follow, and tbh it still feels like there's unnecessary complexity here somewhere, but ok, I don't know your problem well enough

the bottom line though is that if you want type checking during iteration, you'll have to define a collection, but a collection won't give you "attribute style" access

[tidal marten]

Yeah, I'm just anticipating with flexibility since I still need to somehow integrate it to the parser

[halcyon trail]

if you want to do your best to get both, you'd have to define a collection, and then define enums which can index into it

RULES = {
    "RULE1": Rule(...),
    "RULE2": Rule(...),
    "RULE3": Rule(...),
}

class RulesEnum(Enum):
    RULE1 = auto()
    RULE2 = auto()
    RULE3 = auto()
    ...
    ALIAS1 = RULE1

def get_rule_by_enum(e: RulesEnum) -> Rule:
    return RULES[e.name]

If you do this then you have to repeat the rule names, as you can see

but now, RULES is simply a Dict[str, Rule], so when you iterate over it you get full type checking

and then, for combinations you'd have:

class RuleCombinations(Enum):
    COMBINATION1 = (RulesEnum.RULE1, RulesEnum.RULE2)
    ...

And perhaps you would have some helper function:

def get_rulecombo_by_enum(e: RuleCombinations) -> Set[Rule]:
    return {RULES[x.name] for x in e.value}

And you can also have a simple helper function to iterate over combinations, with proper typing, etc

[tidal marten]

This just adds redundancy though, I need to write the same name twice.

Also that getting the value now needs additional name resolution steps compared to just putting everything in a class or module

But it does manage the rules more neatly, with different enum for atomic rule and combination rule

[halcyon trail]

I don't know what you mean by additional name resolution steps

Yeah, you have to write the same names twice

I don't see any way to get strong type checking during iteration, and also get property style access with strong checking

without that redundancy

The other thing you could do is to simply provide a helper function for iteration, and annotate that helper function properly

I guess that's probably better, in fact

@dataclass(frozen=True)
class Rule:
    ...

class ActionRule(Enum):
    RULE1 = Rule(...)
    RULE2 = Rule(...)
    RULE3 = Rule(...)
    RULE4 = Rule(...)
    RULE5 = Rule(...)

def iterate_rules() -> Iterable[Rule]:
    for x in ActionRule:
        yield x.value

man idk what mypy is doing, it seems like its drunk: https://mypy-play.net/?mypy=latest&python=3.9&gist=05175fc68e9907c9d80a697de2825f69

Why on earth doesn't mypy complain about this:

from enum import Enum
from dataclasses import dataclass
from typing import Iterable

@dataclass(frozen=True)
class Rule:
    x: float


class ActionRule(Enum):
    RULE1 = Rule(1.0)
    RULE2 = Rule(2.0)
    RULE3 = Rule(3.0)
    RULE4 = Rule(4.0)
    RULE5 = Rule(5.0)

def iterate_rules() -> Iterable[Rule]:
    for x in ActionRule:
        yield x.value
    
def foo():
    for x in iterate_rules():
        print(x.asdf)

[tidal marten]

Yeah, that's weird

I'll probably read on tokenize.py and see whatever my version of that will be, lol
https://github.com/python/cpython/blob/3.9/Lib/tokenize.py

[austere moon]

Any core devs in here? Looking for a sponsor for this PEP: https://github.com/nhumrich/peps/blob/master/pep-9999.rst

[halcyon trail]

i think the mailing list is probably a better bet for this sort of thing

[austere moon]

Thanks. Looking for a way to get this to stand out above the noise from the mailing list.

[autumn tide]

Does this make sense to do if I wanted C to use A's test method? The reason I'm trying to do this is that in our SDK code we currently have C inheriting from A but we need B's methods for C aswell.

from abc import ABC, abstractmethod

class A(ABC):

    @classmethod
    @abstractmethod
    def test(cls):
        return "A"

class B(A):
    def __init__(self) -> None:
        super().__init__()

    def test(self):
        if isinstance(self, C):
            return super().test()
        return "B"

class C(B):
    def __init__(self) -> None:
        super().__init__()

    def test(self):
        return super().test()

[halcyon trail]

it's pretty bad to have an isinstance check in B, for C

If C wants to use A's test method, can just call it directly

[autumn tide]

Well I completely over thought this, thanks 😅

[halcyon trail]

np

[paper echo]

i feel like it's weird to override a classmethod with an instance method

[halcyon trail]

I didn't even notice that

yeah, that's super weird, and mypy should really complain about that

[lean basin]

Can someone help me out here:

class Appointment():

    def __init__(self, description, date):
        self.description = description
        self.date = date

class FirstTime(Appointment):
    
    def __init__(self):
        super().__init__()

class EveryDay(Appointment):
    
    def __init__(self):
        super().__init__()

class Monthly(Appointment):
        
    def __init__(self):
        super().__init__()

ft = FirstTime("see doctor", "1/2/2021")

print(ft.date)

I am having a hard time getting my classes to inherit from appointment

[knotty tendon]

Hi all, How generally are pyrex, or .pyx files imported into python files? I meant in the sense that whether their imports results without any errors, in the same way a normal .py file would be imported?

[wide shuttle]

This question would fit a help channel better, but the short answer is that the __init__ functions in your subclasses do not take arguments. Trying to instantiate FirstTime as FirstTime("see doctor", "1/2/2021") means that you do pass two values to __init__, which will fail, as __init__ doesn't take them. A common pattern in inheritance is using * and ** to pack positional/keyword arguments to pass them on to the super method:

class FirstTime(Appointment):
    
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

[paper echo]

they're supposed to be compiled to C first, but yes the result is a module object, like what you'd get from a .py file

[lean basin]

Thank you for the help 🙂

[red solar]

>>>> a = ([],)
>>>> a[0] += [1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>>> a
([1],)

so is this a bug in the implementation?

i don't see how this could be specified behaviour

[charred pilot]

no, let me find a link to a previous discussoin that explains this

[red solar]

yes pls 🙂

[charred pilot]

#internals-and-peps message

[wide shuttle]

This is intended behavior

+= also does an assignment, which fails, since tuples are immutable

[halcyon trail]

I feel like this came up in here a few weeks ago

[red solar]

yeah, to use Python syntax,

vals[0] += ["a", "b", "c"]

is roughly the same as

temp = vals[0]
temp += ["a", "b", "c"]
vals[0] = temp

And that last assignment fails.

[red solar]

i'm sure many people have brought it up before me

i thought += had its own magic method tho?

or is it required that it assigns?

[charred pilot]

it does, __iadd__

[halcyon trail]

it has its own magic method but still assigns

[red solar]

temp = vals[0]
temp.__iadd__(["a", "b", "c"])

i don't see why it wouldn't be just this, and so not assign anything to the tuple?

[halcyon trail]

the assignment can be a trivial self assignment though

[flat gazelle]

a += b is essentially

a = a.__iadd__(b)
```(with some fallbacks)

[molten kayak]

why doesnt ansi codes work in python (running the script in CMD) is it a windows thing or python?

[halcyon trail]

yeah, I know how you feel doodspav, coming from C++ it was weird to me as well

[red solar]

i don't get why they would have an assignment external to += :/

[halcyon trail]

it's because in most languages like python, java, etc, assignment is magic

in C++ assignement itself is just another member function

[red solar]

but i did it the other way round (first language was py), i don't like feeling this way

[halcyon trail]

And they have functionality in mind where they need the magic of assignment

that functionality is allowing += to appear to work on immutable types

if += was a pure mutating method, i.e. a.__iadd__(b)

and integers are immutable

(in python)

[wide shuttle]

If an item doesn't support iadd, it falls back to add and/or radd. The current behavior is to always assign the returned value to the target, which means it even works for those non-mutating methods.

[halcyon trail]

which they are

then x += 5 where x is an int cannot possibly work

[red solar]

ah, damnit i hadn't thought of that :/

[wide shuttle]

It's also why it's called augmented assignment: it's assignment with an additional step

[red solar]

damned immutable types :/

[halcyon trail]

except it's not really augmented assignment

because it can also do all of its work via mutation and have a trivial assignment

In, I believe, C#, for example, x += y is always exactly translated to x = x + y, so += is purely just an augmented assignment operator

In C++, although they're still called augmented assignment operators, they don't actually do assignment

they're just mutators, albeit ones that you expect to conform to certain intuitive behavior

Python and Kotlin (to pick two examples) make a more complex decision where in some cases += can be a mutator and in some cases it's creating a new object and doing assignment

[red solar]

we have something called "augmented assignment operators" in c++? never seen that phrase

[halcyon trail]

it's just a common term that people use to refer to any of these operators in general

In C++ it looks like they are just called "assignment operators" which isn't very enlightening

https://en.cppreference.com/w/cpp/language/operator_assignment

but, again, assignment in C++ is a mutator as well, assignment isn't really special in C++ in the way it is in all these other languages.

In C++, immutable types are much rarer, because it's less useful since sharing ownership isn't as common, because it's not a GC language

But if you do have an immutable type in C++, you can't have a (sane) += for it.
On the flip side, in C# you can have a sane += for immutable types, but += can be very inefficient for "big" mutable types, so C# arrays/lists for example don't support +=

[molten kayak]

i believe this is true

[halcyon trail]

So, both of these approaches are simple-ish, both work well for one type of class, but not for another

Python and Kotlin take more complex approaches (and they're also different from one another), to make += work "well" for both immutable and mutable types

Python's approach is to always translate += into a few bits of code which give you the chance to do mutation and assignment

And you use whichever one of them you want, basically

In Kotlin, all variables have to be declared with either var or val, val is like final in the Java sense, i.e. you can't reseat the reference.
x += y will try to do one of two different things:

1. Call x's += member function.
2. If x is a var, it will do x = x + y (and call x's + member function).
   If both of those are possible, it's a compiler error (ambiguous). If neither are possible, it's also a compiler error. So exactly one of them needs to be available for the code to work.
   This approach also works well for both immutable and mutable types, but you need to declare your immutable types var and your mutables types val, which makes sense.

[molten kayak]

why doesnt ansi codes work in python (running the script in CMD) is it a windows thing or python?

[radiant fulcrum]

Its a windows thing

Use powershell or gitbash or change your terminal settings

Windows is very up and down in terms of colour codes

[raven ridge]

there's also colorama, which translates ANSI color escapes into calls to the Windows terminal APIs to make colors work, IIUC

[deft pagoda]

windows terminal is vt100 enabled

only win32 consoles aren't

so command before windows 10

[raven ridge]

TIL.

[sand goblet]

But it’s O(n) to add things

And dict/set might be worse until there’s a very big amount of items

I guess there just must not be any demand for it in python

[flat gazelle]

Yeah, rarely is the optimalization merited

[halcyon trail]

honestlythe only time I used bisect it was probably more out of principle/curiosity than because it was justified

i was reading in sorted lists but they were so small that it just didn' tmatter

well, I should say probably didn't matter. Who knows, maybe it would have made a difference of a second, or half a second, for some script

[knotty tendon]

Actually I never see any separate compilation process in the scripts I noticed. Mostly whatever I see is if a .pyx file is there, its functions/methods are being imported into .py files as normal functions in .py files and not even a small difference. So can you pls let me know the correct procedure?
Also in these files, the execution of the python .py files is initiated using
python filename.py
If at all the above file calls .pyx file's methods, no extra code is being written to compile that file. Is this the correct logic??

[gleaming rover]

it’s that time of the year when I complain about ugly lambdas in Python

I don’t think things will change in the near future, but still…

[nova aspen]

Anyone here know the best practices in GIT using source tree?

[terse orchid]

Hello, could someone give an idea about the discussion in #help-chili? About pytest and packages

[unkempt rock]

What is the reasoning behind collections.UserString not registering itself as an indirect subclass of str?

[paper echo]

Can you give an example? You can't just put code in a .pyx file and import it, Python doesn't recognize that extension

[halcyon trail]

The need for this class has been partially supplanted by the ability to subclass directly from str; however, this class can be easier to work with because the underlying string is accessible as an attribute.

so, the original motivation for thsi class was that you couldn't directly subclass str

so, not surprisingly, it doesn't subclass str 🙂

[paper echo]

One of two things must be happening @knotty tendon: the files are being compiled somewhere, or there is a special module being used that hooks into the import machinery in order to find and .pyx files and compile them on the fly

[halcyon trail]

registering itself; I thought that was an ABC thing, no?

[unkempt rock]

It doesn't need to subclass str to be an indirect subclass:

qwerty@MacBook-Pro ~ % cat temp.py
class Foo:
    def __init__(self, name: str):
        self.name = name

from collections import UserString
Foo(UserString('hello'))
qwerty@MacBook-Pro ~ % mypy temp.py
temp.py:6: error: Argument 1 to "Foo" has incompatible type "UserString"; expected "str"
Found 1 error in 1 file (checked 1 source file)
``` It's quite odd that mypy'll flag this as an error, along with anyone's code checking if you passed in a string

[halcyon trail]

I'm not sure if I'd say that it's odd. Really, the proper solution is that there should be a String interface

and that's what should be used typically for type annotations

[paper echo]

i agree with quicknir, this is not at all surprising or odd behavior. how would mypy ever know that Foo is meant to be a proxy/wrapper for str?

[halcyon trail]

but for whatever reason it hasn't gone that way

[paper echo]

class StringLike(Protocol):
    ...

like that?

that would be nice

i think the stdlib is gradually heading in that direction, but it probably needs a push from contributors and/or PEP authors

[unkempt rock]

Of course it's not odd behaviour that mypy is flagging it as an error, it's odd to the end user that their string class is actually not being accepted as a string

A string protocol would be nice

[paper echo]

eh, this UserString stuff kind of predates python users caring about what class something is an instance of

older python code tends to be so aggressively duck-typed that you don't have a damn clue what's going on

[halcyon trail]

How does "register as an indirect subclass" work for a non-ABC?

I'm curious; I always thought that was an ABC facility, so I don't even know what code you'd invoke

[unkempt rock]

I suppose their metaclass overriding __subclasscheck__ and __instancecheck__, but I'm just skimming the docs and hearing the term "indirect subclass" a lot

[halcyon trail]

This is fairly magical stuff to expect mypy to understand

[paper echo]

but at least it would work at runtime

and the type guards pep could help

but i agree, more protocols in the stdlib please

the side benefit for protocols is that you don't have to worry about variance issues as much

[halcyon trail]

I didn't really see much about "indirect subclass", actually with ABC's they're called "virtual subclasses", but I see little discussion of that outside of ABC's

[unkempt rock]

Those methods are essentially what abc uses

        def __instancecheck__(cls, instance):
            """Override for isinstance(instance, cls)."""
            return _abc_instancecheck(cls, instance)

        def __subclasscheck__(cls, subclass):
            """Override for issubclass(subclass, cls)."""
            return _abc_subclasscheck(cls, subclass)

[paper echo]

what if you could turn a class into a protocol? basically all its type annotations but ignore its name and implementation

[halcyon trail]

For a concrete class, it's extremely odd IMHO to register a class as its child, without it really being a child

The thing is that str in python is a "basic type" essentially, in the typing view of things. You're going to use str directly in annotations the same way that you're going to use int and float and bool

In principle you could use interfaces of some kind (either ABCs or protocols) for all of those

but these types are basic enough and rarely enough replaced that the benefit isn't seen as worth the costs

[paper echo]

is there a good reason not to subclass string in recent versions?

[halcyon trail]

Whereas, even for the built in collection types, you're generally encouraged to use interfaces more often

[paper echo]

i'd be +1 on always using interfaces for annotation anyway

more importing, though

[halcyon trail]

I guess, it seems like every language has its limit though

[paper echo]

import typing
if typing.TYPE_CHECKING:
    from typing import IString, IInt, IBool, IFloat

in every damn file

[halcyon trail]

FWIW the situation in Kotlin is extremely similar to python in this regard.
List, MutableList, Map, MutableMap, etc, are all actually interfaces, not classes

[paper echo]

basically going backwards from being able to write list[bool] to IList[IBool]

[halcyon trail]

But String, Integer, etc are classes

[paper echo]

how does kotlin do it? are those names all just in the top-level namespace by default?

[halcyon trail]

Yeah

[paper echo]

i guess from typing import * wouldn't kill anyone

[halcyon trail]

And you never work with the class directly

well, the same way that python has list as a built-in, right

python has fewer named built-ins, but then python also has collection literals, comprehensions, which are just fancy/operator built-ins

[paper echo]

right now, typing.List is a pass-through to list. maybe it should be a Protocol instead? but it isn't currently.

[halcyon trail]

I don't really see a reason for it to be a protocol, tbh

oh, it's a pass through now, hmm

what was it before, an ABC?

[paper echo]

nope, always was a passthrough i think

except the class supported __getitem__

so some kind of indirection, but i think you could always write List in "runtime code" instead of list

don't quote me on that, maybe it was totally different in 3.6

same with typing.Dict, typing.OrderedDict, typing.DefaultDict, etc.

maybe even typing.Tuple?

[halcyon trail]

https://docs.python.org/3/library/collections.abc.html#module-collections.abc

If you want to go that route, what you'd actually use is Sequence and MutableSequence

[paper echo]

right, typing.Iterable is also a passthrough to collections.abc.Iterable

the latter didn't support __getitem__ until recently

and yes, imo use MutableSequence in parameter annotations and list in return annotations

[halcyon trail]

yeah, that's one approach

I know it's one that's been recommend in python

[paper echo]

that is, return concrete types when it makes sense to do so, and accept interfaces unless you really need a specific class

is that not standard in other OO languages? what do kotlin people do?

[halcyon trail]

it really all depends on the specific case

In Kotlin you basically never work with the concrete class

if you want a mutable list, you do val x = mutableListOf(1, 2, 3)

and now x is a MutableList<Integer>

which is an interface

In practice, x is probably an ArrayList<Integer> but that's not promised

I'm not really sure there's much benefit in returning the concrete type

the user has no reason to really care, 99% of the time

In the example of Kotlin, I actually quite like this. Because, for example, one thing you can do in Kotlin, is replace some of these functions to use different list implementations. Say you want to use immutable data structures; you can replace the map that Kotlin provides to return an immutable list

And this will not break any (reasonable) code or cause surprises, because the new map is still returning a List<whatever>; if map returned an ArrayList<whatever> this would not be possible

[paper echo]

i see

so kotlin really goes the extra mile to try to avoid having to work with concrete classes by name

[halcyon trail]

I don't really think it's the extra mile; there's no extra work involved for anyone

It's literally just whether you annotate the return type with something more specific or something more general

the advice I've seen in the python community is basically that functions should be "nice" to their users; accept the most general thing and return the most specific. I think this isn't a very nuanced view.

A function that accepts something very broad and returns something very specific has a lot less flexibility; its' harder to change how it's implemented, harder to replace it with another function, etc

[paper echo]

that's fair, but how often do you really change a return type?

[halcyon trail]

Probably more often than you need to care about list vs MutableSequence

[paper echo]

if your functions always accept flexible inputs (like Sequence or MutableSequence) then the return type changing from List to SomeOtherMutableSequence shouldn't affect your code

but point taken, i'm sure you can come up with examples where this matters more

[halcyon trail]

Well, functions are relatively easy, but now say you have classes, members of classes can be both inputs and outputs, now it's trickier

[paper echo]

hmm... are instance attributes invariant in mypy? of course, they have to be

[halcyon trail]

In the example of Kotlin, what I would like to do in principle if I was working with Kotlin, is use immutable data structures. So you have ImmutableList<T> inheriting from List<T>. And most of your classes, mostly have ImmutableList<T> members rather than List<T> members, as its safer

because Kotlin's built in map only returns List<T>, I can actually replace it with my own map that returns ImmutableList<T>, this is compatible, I'm making map more specific for my own purposes and not bothering anybody else

And now I can use the result of a map directly to pass to a class with an ImmutableList<Integer> member, or whatever

Just an example

[paper echo]

https://mypy-play.net/?mypy=latest&python=3.10&gist=f17a8b4b779229c3a2df886e56a5fc54

[halcyon trail]

In python I think it's probably less valuable, because python is forever chained to exactly list, dict, and set, because the comprehensions are magic built-ins

which, IMHO, sucks

Kotlin doesn't have this problem

[paper echo]

starting tomorrow, mypy could interpret list comprehension as MutableSequence instead of list

the fact that you can't actually override the implementation is an orthogonal concern

but i see your point, as in the mypy-play link

[halcyon trail]

Well, the can is just getting kicked right, your data structure can just have MutableSequence instead of a list

yeah

It's not hard to completely ignore list in python for type annotation purposes

[paper echo]

the fact that python 3 returns a lot of things as opaque iterables is good too, because you can control exactly if/how the results are collected

[halcyon trail]

Right

What's hard in python is not avoiding the list type annotation, but avoiding list itself

you'll basically have to avoid list comprehensions a lot of the time, in favor of generator comprehensions

and some kind of explicit constructor

[paper echo]

again i think that's somewhat of an unrelated concern and not really something python users should care about

[halcyon trail]

I definitely disagree with that...

Why should I be stuck with list

[paper echo]

how often do you really need to change how list is implemented

tuple(foo(x) + 7 for x in vals if x > 10)

[halcyon trail]

I don't because python makes it hard and makes it an upstream swim

but in something like Kotlin, where it's reasonable to do, I would use an immutable list

I would use it in python too but because it's a pain I don't

[paper echo]

but that's not a MutableSequence so it'd break everything anyway

it breaks the contract that is implicitly expected by most code that consumes lists

[halcyon trail]

Well, if I were doing that I wouldn't use MutableSequence much, would I 🙂

[visual shadow]

type hinting on a return sounds like the most straight-forward thing in the whole world, no? you just..put the type of whatever you're sending.

[paper echo]

right, but overriding what a list is changes what a list means semantically, which of course you don't want to do

[halcyon trail]

Except, no, it's not that straight-forward 🙂

[visual shadow]

why should it be your concern how the consumer uses your output? they know ducktyping, they treat their cake the way they want to

[paper echo]

you wouldn't override mutableListOf to return an ImmutableSequence!

[halcyon trail]

That's part of my point

Python gives you these blunt instruments, that are very convenient, but that's what they are, blunt instruments.

In Kotlin, you have listOf, and mutableListOf, two separate functions for two separate interfaces

Python just has []

[paper echo]

doesn't kotlin have list literals?

[halcyon trail]

no

[paper echo]

well that's kotlin then

[halcyon trail]

And I hope it does not get them

[visual shadow]

i sure wouldn't want to be using kotlin in python in that case.

[halcyon trail]

Sure, that's part of the point

I think if your main concern is typing a few extra characters when you have list literals then... well, idk

[paper echo]

i just don't get the concern. in python, list means "mutable" 🤷‍♂️. [] is for lists. if you don't like list literals then don't use them. i think a frozenlist would be great, and it's probably not that hard to implement in c if you need one that doesn't have a ton of overhead.

[halcyon trail]

the concern is that the only convenient ways to do things in python are special cased into the language, and only work for the built in special data types.

[paper echo]

don't forget that python is very general-purpose, in that it needs to cater to both quick scripting as well as industrial-strength server applications

[halcyon trail]

Kotlin, and many other newer languages, are giving you tools instead to write things roughly as conveniently, without special casing certain data structures

[paper echo]

imagine these poor kids in help channels who can barely wrap their heads around if

[halcyon trail]

And I prefer that approach. that's all really.

[paper echo]

that's fine, and python gives you enough flexibility and tools that you can adopt that approach in your code

in fact i think it's a great idea

but banning list literals because you think the default list shouldn't be mutable is too far

[halcyon trail]

Darr, it's nto as simple as you're making it out to be wrt to return types, if you return something that's more concrete then you're committing to returning that specific thing forever, and changing it is a breaking change, that has ramifications on an evolving codebase. You can look at my example above if you're interested.

[paper echo]

python is python, everything is mutable and certain data types are "blessed" by the runtime

[halcyon trail]

I don't advocate banning list literals in python.

[paper echo]

overriding what [] does would be endlessly confusing

[halcyon trail]

Yes... that's why its' better not to have it 🙂

[paper echo]

now you have global state that changes special syntax

[halcyon trail]

Not in python specifically

But as a language design thing, I think the value is very minimal

[paper echo]

i disagree significantly

you really want to punish people by writing mutable_list_of(1,2,3) instead of [1,2,3]?

[visual shadow]

why shouldn't you commit to returning a specific thing? it's simple, it's easy to understand, and if you make a breaking change, so be it

[paper echo]

it just doesn't make sense for the range of uses that python has

[halcyon trail]

Darr, not sure what to tell you, this is a massively over-simplified view, there's lots of good discussion about this sort of thing, broad vs narrow contracts, etc

You can see that entire programming languages disagree with you

[paper echo]

it maybe isn't ideal language design for the specific things that kotlin is used for, but python is used for a wider range of things

[visual shadow]

i know, i think people like complicating things. it's why i use python after all. i'll put a disclaimer quicknir, i come from the stance that typing was a mistake in python. you probably won't find a lot of common ground with me in that sense.

[paper echo]

@visual shadow because if you return list[int] and one day change it to frozenlist[int] then you can break downstream code. whereas if you return Sequence[int] you won't break anything ever.

[halcyon trail]

Regardless what return type you end up deciding for a specific function you ought to understand that there a trade-off

Well, sorry, but I find no common ground with someone who doesn't understand that engineering is about trade-offs

So that part is true, yes

[visual shadow]

ouch. you're implying that im not understanding tradeoffs there, that's a low blow.

[paper echo]

yeah i can't get on board with that either. typing in python is anything but a mistake. even fucking ruby has type annotations now, and for good reason. the entire software engineering field is moving back to types.

[halcyon trail]

It's not. You're jujst saying: "just return the type. It's taht simple. Why complicate things".

I'm saying "sometimes you return the type, sometimes you may not want to. It depends"

[paper echo]

i've worked in big untyped codebases. it was literally unmaintainable until we embarked on a massive type-annotating project.

and this wasn't even messy code!

[halcyon trail]

It doesn't even need to be that big

[paper echo]

right, it was a medium-size django application. nothing really out of the ordinary.

[halcyon trail]

but anyhow the old typing vs untyped discussion probably won't lead anywhere fruitful

[visual shadow]

i suppose my perspective is simply because i don't have to work on huge code bases. i didn't mean to derail the convo

[paper echo]

the nice thing about gradual types is that you can ignore them 🙂

[halcyon trail]

Nah, you didn't derail it, we're all talking, it's all good

[paper echo]

but i think what you said about "types are a mistake" is analogous to "[] is a mistake"

[visual shadow]

i should perhaps clarify. python was not designed with types in mind when it was made

[halcyon trail]

what's the analogy?

[visual shadow]

there seems to be so many gotchas and rough edges with the typing system that it feels very off putting having this half-baked monster to deal with.

[halcyon trail]

I agree Darr, and FWIW saying that types are a mistake for python is a much milder stance than saying that statically typed languages are a mistake, generally. I disagree with both but I can understand the former much more easily.

[paper echo]

that [] makes sense in python, even "retroactively", because it's convenient, and python really ought to be convenient for basic usage, and mutable_list_of(1,2,3) is a lot less convenient than [1,2,3]

[halcyon trail]

listOf and mutableListOf are convenient enough for basic usage, relative to the number of times I actually do the thing

[paper echo]

right, which is probably a lot less often in kotlin than in python

imagine ```python
np.array(listOf(
listOf(1,2,3),
listOf(4,5,6),
))

[halcyon trail]

Err, no, I just don't have literals like that, so often in my code, usually you're iterating over variables that come from somewhere

[visual shadow]

oh yes, i think statically types languages are great, and have a genuine need to exist for sure. my comments about python are only really for python 🙂

[halcyon trail]

Most programming beyond "hello world" and introductory classes isn't swamped with collection literals

[paper echo]

no, most server app programming

[halcyon trail]

I'm looking at a script here which is nothing like server app programming

[paper echo]

lots of scripts and scientific programs have collection literals

[halcyon trail]

it's data stuff

a script and quantitative

It's 250 lines, I don't think there's a single collection literal

Maybe I missed a couple somewhere though

[visual shadow]

i have to say, we've got a lot of scripts floating around with lots of dicts of lists, weights, etc. i may even say most of my scripts tend to have collection literals somewhere

[paper echo]

🤷‍♂️ my point is that removing [] is a level of "strictness" that caters to a very specific subset of python users while punishing the rest

you are in that subset

[halcyon trail]

Sure, no doubt, a handful

[paper echo]

as you said, it's a tradeoff. python always prioritized convenience

this is one reason people hated python 3

[halcyon trail]

Your notion of "punishment" is the fact that you have to say listOf instead of [], it's just not a big deal

[paper echo]

it has drifted a bit away from convenience towards increased consistency

i think it's in a good place right now

[halcyon trail]

meanwhile, python itself is missing convenience in a lot more important situations

just the fact that I have to create empty collections and append to them so often is a lack of convenience

[paper echo]

contents = fp.read() with open('hello.txt') as fp

i'd love that

[halcyon trail]

but it doesn't work

[paper echo]

of course it doesn't, it's not valid syntax 🙂

[halcyon trail]

Right. So imagine a language which made that easily possible, without needing special syntax.

That's the braoder point in this discussion.

[paper echo]

like what, having ruby-style blocks or multiline lambdas?

[halcyon trail]

Python makes certain specific things very very convenient. But it's not good at making things that it's not targetting, at being convenient

Sure

[visual shadow]

hm, question. what is the issue with just letting [] exist and be used for a specific list type?

[halcyon trail]

the issue in practice is that it just gets used everywhere, it's super convenient compared to everything else, so there's a lot of friction in introducing alternatives

Another realistic thing is that without [], python looks much less convenient than most of its competitors

because map in python is awful

[visual shadow]

both of those seems like reasons to "keep" [], no?

[paper echo]

yeah i won't deny that it'd be convenient to have a python equivalent of

with(open('hello.txt'), function (fp)
  return fp.read()
end)

but this is why we have with blocks

[halcyon trail]

in python specifically, I'm not seriously advocating getting rid of [], or even using a codebase style without []

I'm only saying, in terms of programming language design, I don't think it's a winner.

[visual shadow]

oh. okay

[halcyon trail]

With an existing language, practical factors always outweigh theoretical factors.

[paper echo]

i'm not saying that python is perfect, all i'm saying is that i think having collection literals is not among its warts

[halcyon trail]

It's a small wart

[visual shadow]

i agree with that (that literals are fine in python). but i also understand that people look at the same things and have different opinions and takes about it

[halcyon trail]

your example with with, and the damage of not having decent lambdas/blocks is a much bigger one

[visual shadow]

it's why it's a good thing there's a lot of programming languages to choose from

[halcyon trail]

Yeah, I do think relatively speaking we've seen a lot of convergence, relatively speaking, on more features, in the last 10 years

[paper echo]

now at the other end of things you have idris, wherein [] is syntactic sugar for ::, and you actually get ambiguous-resolution errors in the repl if you don't specify the type of the variable

Main> :module Data.Vect
Imported module Data.Vect
Main> [1,2,3]
Error: Ambiguous elaboration. Possible results:
    ?x Data.Vect.:: ?xs
    ?arg Prelude.:: ?arg

(Interactive):1:1--1:2
 1 | [1,2,3]
     ^

Main> (the (List Integer) [1,2,3])
[1, 2, 3]
Main> (the (Vect 3 Integer) [1,2,3])
[1, 2, 3]

[halcyon trail]

the same way that starting with the 90's almost every mainstream language (i.e. not pure FP languages) has some notion of a class

[static bluff]

I've got a question that might be a bit more appropriate here than in a channel

[visual shadow]

i wish classes had a "from the get go" story around serializing that just..worked anywhere.

[halcyon trail]

from ~2010, many/most languages introduced tries hard to have super slick lambda notation, tries hard to provide functions like map/filter that are actually nice to use instead of something like []

[static bluff]

When and how do I use a shared (between processes) object

[paper echo]

they tried that with xml in the 90s 😬

[static bluff]

Sorry, I'll wait

[paper echo]

google has protobufs etc.

[halcyon trail]

pickle?

[visual shadow]

you're fine Sky

[halcyon trail]

Sky you don't have to wait nobody owns the channel 🙂

[paper echo]

imo don't, unless you really need to. that said you could probably ask this in #async-and-concurrency

[flat gazelle]

you can't serialize a socketserver, so it is somewhat questionable. but yes, C# can to an extent use XML for this purpose, and python's pickle sometimes works as well

[static bluff]

Yeah 😛 But Still, its polite to let one conversation play through

[halcyon trail]

JavaBeans

[flat gazelle]

the issue is that classes are of a really complex shape and so it is near impossible to have a nice way to do it that looks clean

[halcyon trail]

it really depends too, there are so many different goals in serialization, there is no one answer

[visual shadow]

yeah. especially if you also throw composition into the mix

[halcyon trail]

security, space, performance, typing, etc

[visual shadow]

my gripe/gripes with pickle is that it still needs the original code to deserialize, and can be a point of attack

[halcyon trail]

human readability

How can you deserialize a class without knowing the code for the class?

[paper echo]

pickle is basically only good for multiprocessing, afaik. for everything else you basically shouldn't use it

[visual shadow]

but i suppose it's tough to reconcile the two

[halcyon trail]

Then you'd have to pack up the code for the class, and that could even more easily be used as a point of attack

[visual shadow]

yeah, indeed

[halcyon trail]

I haven't heard about anything like that

[flat gazelle]

yeah, it is generally better to keep implicit serialization to things like pydantic etc which are designed for that

even if they are strictly speaking ambiguous

[halcyon trail]

Generally, you're either sending things that actually will be deserialized into classes, in which case the received needs to have the code

pickle, protobuff, javabeans, etc all fall into this category

protobuff is more sophisticated in that the receiver may not have the exact same code as the sender

and it can still work

but the receiver has to have some code

[peak spoke]

I like using it for local storage like caches as it's a bit speedier compared to other stdlib options in certain cases

[halcyon trail]

the only other approach really is just sending "pure" data; json, xml, msgpack, etc etc

just an example re comprehensions btw, and why I dislike the appraoch of []

you can look at the filtering example that was part of the justification for walrus operator

[f(x) for x in my_list if p(f(x))]

Like, list comprehensions were designed for filtered and mapped lists. exactly for "filtered and mapped lists". They're not even well designed for "mapped and filtered" lists.

So now we have [y for x in my_list if p((y := f(x)))] or something like that... a real eyesore. Try explaining that one to a beginner.

[visual shadow]

i tell beginners dont use the walrus. problem solved 😛

[halcyon trail]

Most languages meanwhile have just gone this route instead my_list.map { f(it) }.filter { p(it) }

[visual shadow]

theres a potential wart in there, but you're blaming the wrong thing i think 😛

[halcyon trail]

a lot more sane, IMHO, and more importantly, it's arbitrarily extensible

how so? because I'm blaming walrus instead of the list comprehension?

sorry, vice versa

[visual shadow]

oh. you are? if you are, the ncarry on

oh.

haha yes

[halcyon trail]

no, i was blaming list comprehension

How do you map and then filter with a list comprehension?

[visual shadow]

that is fair. list comps don't really extend to a variety of use cases, and it's a decent criticism of it. but at the same time, what they do, they do well

[grave jolt]

[y for y in (m(x) for x in my_list) if p(y)]
``` ![brainmon](https://cdn.discordapp.com/emojis/439516188771483658.webp?size=128 "brainmon")

[halcyon trail]

🙂

sorry where's the mapping

[grave jolt]

fixed

yeah, it's a bit terrible

[halcyon trail]

yeah

[visual shadow]

[y for y in map(m, my_list) if p(y)] 😛

[grave jolt]

well, assume that m and p aren't worth extracting

[spice pecan]

lambda

[grave jolt]

[visual shadow]

i honestly dont mind the map variant to be honest

[grave jolt]

my_list |> map m |> filter p still looks nicer

[flat gazelle]

[y for x in u for y in [f(x)] if p(y)]is also an option

[grave jolt]

yeah

[visual shadow]

this doesnt exist in python, right?

[grave jolt]

it doesn't

[visual shadow]

right, okay. yeah, i do like the way "left to right" stuff reads.

[halcyon trail]

or my_list.map(m).filter(p)

yeah. the lack of left to right in python is annoying.

[visual shadow]

i've seen some of it in rust for example, looks clean

[halcyon trail]

Rust kinda has it

[flat gazelle]

yeah, once you get to a certain complexity, you really end up needing generators and multiple lines to keep the code clean

[halcyon trail]

it doesn't have extensions though, if you want to add stuff you need to do a bit of a silly dance

I really like the new trend of extension functions, seen in both Kotlin and Swift. It helps with a lot of this functional left-to-right thing, and issues with OO, simultaneously

and helps with tooling

a lot of bang for your buck on a relatively simple language feature

[visual shadow]

i suppose left to right is fundamentally at odds with python's data model though.

[halcyon trail]

why's that?

[visual shadow]

because python data model encourages builtin calls from outside to access dunders on any class inside

[halcyon trail]

I didn't realize that was part of the "data model" per se

[visual shadow]

so, the notion that stuff like len(stuff) is inherently preferred over stuff.len

[flat gazelle]

python was made after the algol that was specifically meant to have a strict separation of statements and expressions and to be written sequentially without long expression and I am not sure there is much of a way to undo that decision now.

[halcyon trail]

len(stuff) was preferred over stuff.len because the idea was that len and other things might want to do something else than just exactly map to a specific dunder

python only has members, and free functions, when those are your only options, sure, you end up picking free functions for anything that you don't want to be a member

[visual shadow]

indeed, and so were a few other things. there's benefit to the python data model. but that also means it's fundamentally at odds with left-to-right, no?

[halcyon trail]

Again, it's not really python's "data model"

It's just the lack of extensions

[visual shadow]

okay, terminology check, this is known as the python data model, no?

[flat gazelle]

I mean, when classes are str -> object maps, you do want to avoid accidental subclassing

[visual shadow]

happy to be corrected if im mistaken

[flat gazelle]

the data model is the various protocols, like __len__

the len is just convention because writing .__len__ would be a bit silly

and python wanted to avoid accidental subclassing

[halcyon trail]

having a protocol like __len__ is part of the data model, sure

[visual shadow]

ok cool. that was how i was introduced to the topic, so just wanted to check

[halcyon trail]

the inability to easily/sanely have a function len so that x.len() which calls x's __len__ doesn't seem to be part of the data model, to me

but, who knows

[flat gazelle]

I mean, it is part of the data model in that the . operator calls the attribute access protocol which is already overloaded

[halcyon trail]

i guess that's fair enough

[flat gazelle]

I do agree that it is a nicer way to write code

but I don't think there is a nice way to get it into python

[halcyon trail]

yeah, that's probably true

constrained generics also help a lot, in practice, with extension functions

[flat gazelle]

python will just have to live with its legacy until it is replaced by something better.

[swift imp]

Has anyone seen typeclasses? Can't tell if I am fan yet. The article is compelling but the python implementation has you repeating a lot of information for a registered type. Like it gives great examples of why singledispatch isn't good enough but the solution is a bit verbose imo.

[paper echo]

got a link?

[wide shuttle]

I don't quite like languages where arbitrary methods defined on a per-type basis implement very similar behaviour, but may have different names

[paper echo]

haskell typeclasses are basically interfaces, not sure what a python equivalent would be

[flat gazelle]

the powerful thing about haskell typeclasses is the instance inferrence algorithm, which wouldn't work very well in python

[halcyon trail]

not really sure what you mean @wide shuttle

[paper echo]

like having different arrayLen and strLen functions?

[halcyon trail]

Yeah, that was what I was thinking, but that's possible in every language

[paper echo]

i never really thought about this before. but are haskell's and ocaml's type inference all that different from multiple dispatch?

i assume it's more complicated but

[wide shuttle]

I like the uniform approach of Python. There's a single len function and you can implement support for it.

[flat gazelle]

how would they be related?

[wide shuttle]

You never have to guess which method you need and the method shouldn't be different between different types.

[pseudo cradle]

It's very nice design

[paper echo]

(<*>) ((<$>) doStuff x) y

is this any different from any other multiple dispatch in the presence of type inference?

[flat gazelle]

I guess not to an extent

[paper echo]

afaik, the "type inference" part is where the magic is

maybe it's a generalization of multiple dispatch, idk

im sure theres some unifying PL theory here

[halcyon trail]

there's nothing unique about that to python

most (almost all?) languages have some standardized interfaces

idk, maybe Haskell has been lazy and hasn't bothered defining things like a Collection typeclass? If so that's rather weird.

[paper echo]

there are Foldable and Traversable, so you can always compute the length with foldr or whatever

i'm not sure if there's any abstraction for an unordered collection actually

[halcyon trail]

yeah, that's O(N) though and therefore blech

[paper echo]

yeah idk why, i've had at least 2 idris users say "you don't want to wastefully carry around the length at runtime"

that's totally backwards to me, i'd rather carry around a couple extra 64 bit integers if i know i'll need them at runtime, rather than recalculating them

even if theyre boxed how bad can it really be

[halcyon trail]

I mean.... yeah....

knowing lots of type theory doesn't translate to intuition about performance of real world programs, not terribly surprisingly

reminds me of https://www.joelonsoftware.com/2001/12/11/back-to-basics/

[swift imp]

https://sobolevn.me/2021/06/typeclasses-in-python

[paper echo]

Thanks. I find it weird and misleading that they are calling this "typeclass" and not "singledispatch but better"

Not a bad idea though

I don't often need to write multiple specialized versions of a function for data types that don't share a common protocol

As far as i can tell, the name "typeclass" has to do with the idea that the type is a member of a "class"

I don't really understand the API of this library

But I'd definitely consider using it instead of singledispatch

[static bluff]

I think I might be toooooo caffinated to keep going

[rich cradle]

Can you add custom keywords to python (for the sake of a pypi library)? If so, how, and how does it affect the way the ast/compiler works?

[static bluff]

You can, but its not easy

[rich cradle]

Does that have to do with the design of the language?

[static bluff]

@rich cradle dm'ed you

[boreal umbra]

Are there any PEPs of interest these days? I stopped keeping track after the patma PEP and the keyed indexing PEP.

[paper echo]

you can't without changing the python interpreter itself

so no, unless you expect users of your library to all start using your custom version of python

[red solar]

Where are you finding these idris users? 🤔

[paper echo]

their new discord server

[charred pilot]

there's one for better error messages, i forget what the number is. i'll try to find it

[boreal umbra]

I mean the error messages are already leaps and bounds better than "segmentation fault, core dumped, asshole!"

[charred pilot]

https://www.python.org/dev/peps/pep-0657/

[boreal umbra]

there's also a !pep command, for your reference

!pep 657

[fallen slate]

**PEP 657 - Include Fine Grained Error Locations in Tracebacks**

Link

Status

Accepted

Python-Version

3.11

Created

08-May-2021

Type

Standards Track

[charred pilot]

can you search for keywords with it? or do you just have to know the number

[neon tartan]

I think it's tied to the number

!pep column limit

[boreal umbra]

afaik you have to know the number, though you could run !help pep in #bot-commands

[charred pilot]

anyway, the pep is pretty cool, shame we have to wait 2 minor versions for it

[boreal umbra]

you say "minor versions", but you might as well say "releases"

Also I don't intend to sound critical. I'm just a bit tired.

[peak spoke]

I mostly see it as python 3 instead of python with 3 as its major ver

There's a bit going on around 563 with postponed evaluation of annotations and 649 to fix some issues with the current system which prevented it from being the default in 3.10 as planned but I'm not sure on its state now as it had some additional runtime costs

There's also a draft for sentinels which is nice

[paper echo]

type guards

!pep 647

[fallen slate]

**PEP 647 - User-Defined Type Guards**

Link

Status

Accepted

Python-Version

3.10

Created

07-Oct-2020

Type

Standards Track

[paper echo]

!pep 655

[fallen slate]

**PEP 655 - Marking individual TypedDict items as required or potentially-missing**

Link

Status

Draft

Python-Version

3.10

Created

30-Jan-2021

Type

Standards Track

[paper echo]

https://mail.python.org/archives/list/[e-mail]/thread/3JNXLYH5VFPBNIVKT6FFBVVFCZO4GFR2/

Compact syntax for Callable: (Arg1, Arg2) -> Ret

I do think we need to seriously consider how a similar notation could also
be used as an alternative for lambda (JavaScript has this, as does Scala
IIRC).
👀 👀 👀

https://mail.python.org/archives/list/[e-mail]/thread/TXL5LEHYX5ZJAZPZ7YHZU7MVFXMVUVWL/

Supporting dataclass-like semantics [in type annotations]

too much interesting stuff to post here, really. would be spam

keep an eye on https://mail.python.org/archives/list/[e-mail]/ and https://discuss.python.org/c/peps/19

ok one more: https://mail.python.org/archives/list/[e-mail]/thread/LOQFV3IIWGFDB7F5BDX746EZJG4VVBI3/

Given that much of my own excitement for variadics comes from the prospect of typing zip and gather, and that we have some time before this goes into 3.11, I'd be in favor of adding Map[] to PEP 646.

!pep 646

[fallen slate]

**PEP 646 - Variadic Generics**

Link

Status

Draft

Python-Version

3.10

Created

16-Sep-2020

Type

Standards Track

[halcyon trail]

that would be pretty cool

there's very few languages that can properly type zip

C++ can, but, ironically, it's iterator situation is a complete mess

[red solar]

Wait is python3.10 out?

[red solar]

Is it? I thought they mostly fixed it now that the categories are actual concepts and not legacy concepts

[halcyon trail]

I mean it's not even fully standardized yet, is it

ranges is still in TS as of 20 I thought

maybe it is standardized

[red solar]

Wut? No way, ranges is standardised in 20

[halcyon trail]

in any case though, ranges is still, idk, really long compile times, poor perf, really hard to write a range

[red solar]

Hmm I haven’t had the same experience with bad perf, but sure i’ll give you the compile time issue

[halcyon trail]

i've seen fairly simple examples fail to optimize as well as a for loop but I admit I haven't done it super recently

I did recently try to solve a problem with ranges and it's just overall a pretty awful experience, IME

bad compiler errors even by C++ standard

awkward pipe operator

[red solar]

Hmm was it clang? Clang doesn’t really have ranges up to scratch yet

Oooh haven’t tried pipe yet

Is it that bad? 😦 it looked cool

[halcyon trail]

It's also just hard, there's a lot of thorny issues with dangling in a ranges approach in a language like C++

GC languages just avoid it for free, Rust can have sensible APIs which would be too easy to misuse in C++, because the borrow checker can verify it

I'm not a huge fan so far, but I admit I've only dipped my toes into it

Eric Niebler's ranges talk was also, ironically, the opposite of convincing. The code got so complicated to do the relatively straightforward thing he was doing.

[red solar]

Damn ok good thing I haven’t seen it lol

[halcyon trail]

It's worth watching regardless

eric niebler and calendar stuff

[red solar]

Maybe I need more experience with them then, because from my brief foray it seemed like a simple concept - range has begin/end, and that’s mostly it

[paper echo]

@halcyon trail https://hackage.haskell.org/package/ghc-prim-0.7.0/docs/src/GHC-Tuple.html

[halcyon trail]

yep

even languages with very fancy type systems otherwise often don't have variadics

scala is the same

[red solar]

Wtf is that 😂 why does it look like someone wrote boost PP in Haskell

Manuel says: Including one more declaration gives a segmentation fault.

🤔

[paper echo]

apparently haskell can only have up to 62 elements in a tuple

who knew

[red solar]

But it’s not even a language limitation, it’s just their compiler that’s broken

[paper echo]

solid api docs https://hackage.haskell.org/package/ghc-prim-0.7.0/docs/GHC-Tuple.html#t:-40--44--41-

apparently only up to 4-tuples have an applicative implementation

[halcyon trail]

lol

@red solar it's a language limitation

haskell doesn't have variadics

so you have to do stuff like this

Rust, Scala are other examples of languages with pretty advanced tyep systems, have a tuple type, and yet, they do not have variadics

so if you want to implement a trait for tuple, you need to implement it for 1 tuples, 2 tuples, 3 tuples, etc

[red solar]

Not that they can’t have arbitrary length tuples, I meant the fact that the max size is 62 (because higher causes a segfault)

[paper echo]

i wonder how that works at the theoretical level

maybe you would need dependent types?

i think in idris tuples are like cons cells, pairs of pairs of pairs of ...

[halcyon trail]

you don't need dependent types

C++ and D both have variadics and arbitrary size tuples (within implementation limits)

neither have dependent types

[red solar]

Is what we have in C++ not dependent types?

(i.e. nttps)

[paper echo]

D seems really interesting

i keep forgetting to spend time with it... been reading about applicatives and monoids and stuff

why are qualified imports so uncommon in other languages? import and import from seems like really really good design decisions compared to having to deal with this stuff https://dlang.org/articles/hijack.html

does it have to do with the design of compilers? why isn't it always 1 module : 1 namespace?

imo you should only dump everything into one namespace if the user explicitly requests it, and you shouldn't design APIs assuming that the user requested it unless it's a really all-encompassing DSL like sympy

but again maybe there are good technical reasons why so many languages just dump everything into a single namespace upon import

obviously this hijacking thing is still a concern if you do explicitly request to dump everything into one namespace. cool to see that someone has thought about and solved (?) the problem

https://dlang.org/articles/mixin.html this is a bit funny, they let you write arbitrary strings like CPP but check them for valid syntax. weird shadow world between CPP and AST-based or homoiconic syntactic macros

[gleaming rover]

HList?

[queen mango]

Hey all! Im not sure where to go for this but, is there anyone that could help me with some basic assembly language?

[raven ridge]

That's off topic here, but try https://discord.gg/pbAx9yHu

[halcyon trail]

C and C++ in one discord server

sounds like a recipe for a brawl

In the C++ slack, the C channel goes by the tongue in cheek name cpp_the_good_parts

[raven ridge]

And that particular channel that the invite links to is about assembly.

[queen mango]

Thank you so much

[paper echo]

is there any way to annotate a mixin class such that it can only be inherited by subclasses of certain other classes?

class Base1:
    x: float

class Base2:
    x: int

class Mixin:
    def foo(self) -> float:
        return self.x // 5

# OK
class Thing(Base1, Mixin):
    ...

# Not OK
class Other(Base2, Mixin):
    ...

i suppose this gets hairy when you want to do things like extend a mixin, so it's probably not possible and probably not a good idea anyway

but i sometimes find that non-overlapping mixins really are the right level of abstraction/encapsulation for code reuse, but i get nervous at the idea that there's no real type checking of their assumptions, unless you manually re-annotate all their dependencies

e.g. i could write

class Mixin:
    x: float
    def foo(self) -> float:
        return self.x // 5

but that could get very tedious

[halcyon trail]

Hmm, does it matter if the mixin is a direct base of Thing/Other

I'm imagining if Mixin instead of being something you inherit from directly was some kind of argument to Base1 instead

[paper echo]

yeah i know that's a common suggestion for refactoring things like this

in my case it doesn't make sense, the Base classes really shouldn't have to know about the existence of any mixins

in this particular case, i have two almost entirely-unrelated RequestHandler classes that for this or that reason need to share a few common methods and attributes

[halcyon trail]

Why do you say "Base classes"

[paper echo]

and i wanted to make sure that the mixin was only used for RequestHandler subclasses and not other stuff

[halcyon trail]

I thought there was only one Base that your mixins could go with

[paper echo]

oh i meant Base1 and Base2 in the example

[halcyon trail]

ok

then why not RequestHandlerMixer

inherits from RequestHandler

receives the mixins as arguments

[paper echo]

that's an interesting idea. in my case it would involve a bunch of ugly refactors elsewhere.

however, the other and better option that just occurred to me is something like this:

# base, defined in a 3rd party library
class RequestHandler:
    def __init__(self):
        # do some internal stuff ...
        self.initialize()
        # do other internal stuff ...

# my mixin with common functionality
class CommonThing:
    def __init__(self, handler):
        self.handler = handler
    def do_something_complicated(self):
        ...

# my request handler
class Handler1(RequestHandler):
    def initialize(self):
        self._common_thing = CommonThing(self)

# my request handler
class Handler2(RequestHandler):
    def initialize(self):
        self._common_thing = CommonThing(self)

whereas my original idea was

# my mixin with common functionality
class CommonThing:
    def do_something_complicated(self):
        ...

# my request handler
class Handler1(RequestHandler, CommonThing):
    ...

# my request handler
class Handler2(RequestHandler, CommonThing):
    ...

so instead of "mixing in" methods, i have some internal thing that i can delegate to

the circular reference between self._common_thing and self doesn't bother me in this case, but you could refactor the first option to eliminate it

# my mixin with common functionality
class CommonThing:
    def do_something_complicated(self, handler):
        ...

# my request handler
class Handler1(RequestHandler):
    def initialize(self):
        self._common_thing = CommonThing()

# my request handler
class Handler2(RequestHandler):
    def initialize(self):
        self._common_thing = CommonThing()

[alpine wedge]

hey i caught a bird pokemon what should I name it

[visual shadow]

!ot Try offtopic

[fallen slate]

Off-topic channels

There are three off-topic channels:
• #ot0-psvm’s-eternal-disapproval
• #ot1-perplexing-regexing
• #ot2-never-nester’s-nightmare

Their names change randomly every 24 hours, but you can always find them under the OFF-TOPIC/GENERAL category in the channel list.

Please read our off-topic etiquette before participating in conversations.

[halcyon trail]

@paper echo it looks to me like you're rediscovered composition? 🙂

[paper echo]

lol, yes

but applying the pattern is not always as easy as knowing about it!

[halcyon trail]

usually the reason why you use mixins is because you want to directly inject public API

that's why I didn't bother suggesting something like what you have

with the pattern you have (composition), you can only inject public API by monkey patching

[paper echo]

yeah, i think it's still a valid and open question. in my particular case the only "public" API is the HTTP endpoint handled by the RequestHandler so it's not important to me right now

see e.g. scikit-learn, which uses mixins for exactly the purpose you describe

[halcyon trail]

Yeah, I mean if you don't need to add public API, you just shouldn't use mixins

Or at least, I'd need to see another specific reason to do it

I would also probably not pass CommonThing handler in its init function unless there were some really good reason. You can just pass handler in do_something_complicated

but that's probably at least partly my C++ mentality where I try fairly hard to avoid such things because of lifetime issues.

[paper echo]

right, that was the def do_something_complicated(self, handler): version

the only reason i would be tempted to attach the handler as an instance attribute in CommonThing is because some of the common functionality involves invoking i/o-ful methods on the handler

it feels kind of weird to do that from anywhere outside the handler itself, just because it's so atypical for these kinds of frameworks

self.handler.do_io() just feels less strange than handler.do_io()

but i agree the circular references are probably more awkward even if you don't have to worry about lifetime issues

[magic python]

is this a good starting point to learn what a "mixin" class is https://stackoverflow.com/questions/533631/what-is-a-mixin-and-why-are-they-useful ?

[halcyon trail]

the first answer seems reasoanble

the second answer is weird

yeah, the second answer has a lot of nonsense, I would just ignore it

[magic python]

cool, i've never used mixin before, will read the first one

[halcyon trail]

At any rate, the key point of a mixin is that it injects (provides and implements) public API into a class

In python (and many languages), the simplest way to add public API to a class is via inheritance, so it's strongly associated with inheritance

[magic python]

so if i have something like:

class MyDataFrame(pd.DataFrame): ...

is that a form of mixin?

[paper echo]

@magic python

from sklearn.base import BaseEstimator, TransformerMixin

class MyTransformer(BaseEstimator, TransformerMixin):
    ...

and no your dataframe example isn't a mixin, that's just a subclass

[magic python]

These provide fit / transform iirc

[magic python]

Ah ok

[raven ridge]

mixins are classes that are designed to be used through multiple inheritance, adding some functionality to every class that inherits from them

the thing that distinguishes a mixin from a "regular" base class is that a mixin doesn't represent an object in the domain model. It provides functionality, but there's no expectation of being able to group all things that have that functionality together and talk about them.

[magic python]

Thanks, I need to read about multiple inheritance :)

[halcyon trail]

in most cases, mixins expect the derived to provide some kind of functionality

[raven ridge]

mixins are kind of an abuse of inheritance, instead of using inheritance for its normal, intended purpose. Inheritance usually is used to create a subtype of an existing type - "OrderedDict is a type of dict", for instance. Mixins instead use inheritance to inject behavior into the derived class, without meaningfully making the derived class a "type of" the mixin class.

[halcyon trail]

well, I want to say in all cases, though maybe there are counter-examples

basically, the idea with mixins is that:
a) You have a class that has some public api, A, B, C
b) You want the class to have some more public API, X, Y, Z
c) If you inherit from the mixin with your class Foo, and Foo has A, B, C, the mixin will provide X, Y, Z (the implementation of X, Y, Z will involve using A, B, C)

[raven ridge]

In this example, MyTransformer is a type of BaseEstimator, but it isn't really a type of TransformerMixin, and it doesn't even make sense to talk about "types of TransformerMixin". TransformerMixin provides some functionality to derived classes, but is not itself a thing that you could even talk about something being a subtype of.

That is, TransformerMixin is a base class, but is not itself an object in the application's domain model.

[halcyon trail]

In python you can actually often use decorators to achieve the same thing as a mixin

it's just that inheritance is simpler than decoration

BTW, using TransformerMixin is a great example of what I was talking about:

class TransformerMixin:
    """Mixin class for all transformers in scikit-learn."""

    def fit_transform(self, X, y=None, **fit_params):

        if y is None:
            # fit method of arity 1 (unsupervised transformation)
            return self.fit(X, **fit_params).transform(X)
        else:
            # fit method of arity 2 (supervised transformation)
            return self.fit(X, y, **fit_params).transform(X)

In this case: TransformerMixin expect the derived class to provide a fit method. Given that, it will provide a fit_transform method.

[magic python]

I was just wondering if these were kinda like decorators for classes

[halcyon trail]

in some sense. Decorators can do much more though. Both of them in some sense are a way of "transforming" an existing class. It's just that mixins only do one transformation whichi s adding functions

[raven ridge]

Mixins have an advantage over decorators in that the subclass can still override something inherited from a mixin, though, whereas decorators are applied after the subclass has defined its methods.

if you use a decorator to inject a method into a class, that class doesn't get a say in the matter. If you use a mixin to inject a method into a class, that class can override it, or wrap it in its own logic, or whatever.

[swift imp]

Im not a fan of class decorators, I almost never see how they're better

[halcyon trail]

that's a good point godly

@swift imp well, they're just way way more powerful. They're "better" in that sense. If the thing you want to do can be accomplished by both, then yeah typically a mixin is a better choice

[prime estuary]

One example of mixin style classes is the collections.abc ones, though that's slightly different because it's effectively also a interface. Those provide implementations of say the get() method, or .keys().

[swift imp]

in what way are they more powerful?

[raven ridge]

they're also a bit different from typical mixin classes in that they are a noun in the domain model, and it does make sense to talk about "a type of Mapping" or whatever (specifically because they do provide an interface, and not just implementation).

[halcyon trail]

It would be interesting to look at decorators vs mixins in the context of dataclasses, dataclass uses a decorator, pydantic uses a base class, so evident its possible either way

[swift imp]

Like what can you do with a class decorator that cannot be done with __init_subclass__ or a mixin

[halcyon trail]

Uhm I mean you can return a totally unrelated class? 🙂

In decorators you are defining a new class from scratch, you can do anything you want pretty much. It's practically code generation.

[raven ridge]

so can __init_subclass__, I think

[swift imp]

no __init_subclass__ returns None

[raven ridge]

ah, my bad

[prime estuary]

It can't, but a full metaclass can.

[halcyon trail]

Sure, that's metaclasses though.

[swift imp]

If that's what you want you should be metaprogramming

Its way clearer

[halcyon trail]

what does that mean?

[prime estuary]

Decorators are a form of that...

[halcyon trail]

exactly

[swift imp]

Create your own subclass of type at that point

[halcyon trail]

so, metaclasses?

[swift imp]

Yeah

[halcyon trail]

Okay, I mean, you like metaclasses better than decorators, that's your privilege, I don't see why you should always be using mixins or metaclasses over decorators

[prime estuary]

They're all useful tools that can overlap, depending on the situation one can be better than another.

[swift imp]

I'm saying under the circumstances of you wanting to use a decorator over a metaclass, for the means of returning a different class which is the only place that __init_subclass__ and mixins fail, you should just use a metaclass. You have more freedom and its clearer imo

[prime estuary]

Metaclasses are more powerful, but can't be composed with each other without one being aware of the other.

Decorators are nice in that they can be applied outside the class, but can't interact with some parts for that reason.

[swift imp]

Ive actually never seen an application of a decorator outside of a class factory. So i'm probably biased because of that

[prime estuary]

Mixins are very much able to be built on each other and composed, but are quite limited.

There's the cheeky use of @object.__new__ to make a singleton, you could use it to do something like register the class into a dict for later lookup and instantiation.

[halcyon trail]

you've never seen dataclass?

i think one advantage of decorators generally is that they don't affect the class hierarchy at all

or rather, they don't have to, I suppose

Funnily enough, the only time that I've tried to use a metaclass recently, I immediately ran into that composition of metaclasses problem
I had a base class that was an interface and I wanted it to register its derived class. Tried to implement the latter via metaclass. But then you get ABCMeta from ABC, so it doesn't work, so now you have to write some helpers to combine stuff. Not very elegant.
Thankfully someone pointed out the init_subclass hook to me

[prime estuary]

__init_subclass__ was specifically added to give you the ability to do what some basic metaclasses do, while allowing composition.

[halcyon trail]

Yeah. I mean "just use a metaclass" is an odd attitude to me because judging from PEP's like init subclass and other things, it seems like metaclasses are more of a most powerful, last resort thing

[amber copper]

is it possible to typehint a raisable? I know you can check for specific methods (because its a subclass of type Exception), but not sure if u can typehint that

[raven ridge]

Why not just type hint it as Exception?

The set of types that you can raise is exactly the set of types that derive from BaseException

[spark magnet]

are you returning a value that then gets raised?

[amber copper]

yeah

[amber copper]

didnt really think of that to be honest. I thought there were some way of getting the 'raise (type)` effect, without subclassing Exception, maybe its not

[spark magnet]

You can only raise things derived from BaseException

[halcyon trail]

was it like that in python 2 too

[spark magnet]

at some point in the past, you could raise strings, but i don't know when that changed

[halcyon trail]

i thought you could raise anything at some point but there's a good chance I'm misremebering

[spark magnet]

well, your idea would include mine. i don't know the details.

[prime estuary]

You can raise anything, but if it's not a BaseException subclass you just get a TypeError.

[spark magnet]

your definition of "can" includes "causes an error by its very nature"? 🙂

[charred pilot]

"it works, but it fails in the process"

[halcyon trail]

but you end up raising a TypeError, which is an exception 🙂

sorry, is a child of Exception I mean

you never actually raise the original exception

[static bluff]

I figured I'd ask here instead of a help channel because you're all going to know better than anyone

What is the most accurate spell checker out there? I'm just knocking heads with a buddy about a competition he's in

A library works, but requests could also be made to a server

[grave jolt]

Maybe this is what you want: ```py
Union[Type[BaseException], BaseException]

or just use BaseException, much more simple

[cobalt agate]

Any pandas experts in the house can help me with this basic query?
#help-apple message

[magic python]

does anyone use flake9? (https://pypi.org/project/flake9/), only just heard of it.

[calm kayak]

why cant i write anything inside rich text field

[spark magnet]

It only mentions two reasons for the fork, neither of which seems like a good reason.

[magic python]

yeah, main one seems to be the toml support. I'm a bit confused with some of the things I've bumped into re toml. I'd have thought that people would want to use/support it asap, but it seems that some people don't want to / consider it for building things only rather than config settings and stuff. I'm sure there are good reasons, just seems to be more subjective than i thought it would be

[visual shadow]

https://xkcd.com/927/ explains it well, unironically.

[spark magnet]

there's two sides to that: library authors that don't want to add complexity just to duplicate configuration they already support, and library users that feel it's so important, because they want to have fewer configuration files.

I don't understand forking a library just to add toml support.

[visual shadow]

i agree. one function that parsed toml into whatever the library accepted would have been good enough, no?

[spark magnet]

But I also know when it was requested for coverage.py, I didn't want to do it, because it doesn't really add any value. But when I got a PR for it, I merged it.

[magic python]

will you phase to just toml or support both?

[spark magnet]

I would not drop the old config: that would break existing users

[magic python]

i guess it makes sense if things are gradually phased to toml only... but if there's both it could be annoying 🤔

right - but support legacy stuff, but from now on just use toml?

hrm, not sure really. tricky

[spark magnet]

the toml support in coverage.py is a class that reads a toml file and presents the same interface as configparser, so there's nothing config-specific in it. So when I add new configuration settings, I don't have to touch that file.

[halcyon trail]

If you already have both an auto formatter like black, and mypy, how much additional value add will you get from flake8? Seems like not much but maybe I'm forgetting about something

[peak spoke]

it can take care of style which doesn't really fall under formatting like docstring content, naming etc.

[worldly terrace]

i need an assistance in multi processing programming , i will pay for the service

[magic python]

How is one supposed to handle multiple ignores in a single line? For example - mypy ignore is # type: ignore[<error-code>], what if i have a # noqa: <code 1>, <code 2> on the line I want to add a mypy ignore to? Is there a convention here? Similarly I might have a pylint ignore instead of flake

[magic python]

not sure, anything that involves automated help is usually something I accept though 😅

[static bluff]

Anyone on here using a recent version of cefpython?

[spark magnet]

flake8 was an early linter, and doesn't do much. some people like it for that very reason: pylint can seem kind of psycho about the stuff it whines about. I just wish flake8 would take out "my" mccabe script, because I don't think it's worth anything.

[mild flax]

""my""?

[halcyon trail]

I guess it is something primarily contributed by Ned?

The only thing of substance I can think about flake8 doing that's not covered by mypy or black is non-white space style

Which is mostly just naming things, as Numerlor mentioned.

I guess docstring style is fair enough

[spark magnet]

The readme for flake8 says, "Ned Batchelder's McCabe script"

[mild flax]

yeah, is it not yours?

[spark magnet]

sorry, it is mine, but I don't like it, and I don't know why they don't mention the authors of the other parts.

It's something I wrote one night at PyCon, and it has limitations, but no one cares, it just lives on.

[spark magnet]

https://nedbatchelder.com/blog/200803/python_code_complexity_microtool.html

[raven ridge]

black handles style, mypy handles types, flake8 handles everything else you could want to lint. It catches things like unused imports, imports sorted wrong, variables assigned to but never used, and with plugins, things like mutable default arguments, or public functions without docstrings

[raven ridge]

I don't dislike pylint because it's psycho, I dislike pylint because it's so often wrong. A tool that has as many false positives as pylint has just isn't worth anything to me.

there's an issue that's been open for like 2 years now where pylint warns that there's unnecessary parentheses in an expression, but removing those parentheses would result in a SyntaxError

https://github.com/PyCQA/pylint/issues/3249

[spark magnet]

@raven ridge i have also written issues against pylint. The baffling one to me is that it can be non-deterministic.

[raven ridge]

heh. mypy has a failure mode where it fails every second time you run it.

it caches something on the first run that it shouldn't, and the second run fails because of the bad cache, but removes the cache in the process, so the third run succeeds (but caches something that it shouldn't)...

that's a longstanding issue that they don't seem to know how to fix, last I looked...

software's hard 😅

https://github.com/python/mypy/issues/7276

https://github.com/PyCQA/pylint/issues/4356 looks like the one you're mentioning about non-deterministic pylint results.

I personally never recommend that anyone start using pylint. It often complains not when it knows something is wrong, but when it doesn't know whether something is right. Like attempting to import something from a module that uses a module __getattr__, for instance. Or a package that imports things from its own submodules into its __init__.py using star imports.

both of those cases lead to no-name-in-module errors, and both are false positives.

[spark magnet]

yup i wrote that issue 🙂

oh, you knew that

[raven ridge]

that's how I found it, heh. I was just linking it here for the sake of the discussion 🙂

[peak spoke]

I tried using it for a bit when starting out, but ended up ignoring it because of the volume of warnings I didn't care about. In retrospect I probably could've configured it to be usable but some of them like too few methods feel silly

[cloud crypt]

hey everyone, I have a little question
suppose we want to define Default protocol, as such: ```python
from typing import Protocol, Type, TypeVar

Self = TypeVar("Self")

class Default(Protocol):
@classmethod
@abstractmethod
def default(cls: Type[Self]) -> Self:
raise NotImplementedError("Derived classes must implement default class method.")```
the question is, is there any way to implement Default for some built-in type, like int?

[sinful aspen]

!code

[fallen slate]

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

[halcyon trail]

@raven ridge I guess for me all those issues are handled by pycharm

There is value in a cli tool for ci of course

[prime estuary]

You can't no, since new methods can't be added to builtin classes. I'd suggest having a fallback Dict[type, Callable], where you store functions. Check if the default method exists, if not check the dict.

[cloud crypt]

makes sense, I guess

[grave jolt]

I suppose you want typeclasses? 🙂

[cloud crypt]

haha, that'd be nice

[static bluff]

class Application(object):

    class ApplicationInternal(object):
        ...

    def __init__(self):
      
        self.applicationInternal: Application.ApplicationInternal = None

        # OR

        self.applicationInternal = None

    def start():

        self.applicationInternal = self.ApplicationInternal()

Certain details require that the internal widget be created in a later step. So I was just wondering about the recommended way of typing the initial value. Pycharm's linter seems fine using the simpler second option- it recognizes that self.applicationInternal is eventually assigned an ApplicationInternal object. The first option is a bit more explicit but, I might just be getting carried away

What's the right way?

[signal tide]

if you're initiating them in __init__ to a null value, imo it's a bit cleaner if you put them in the class stub (? idk what it's called)

class Application(object):

  class ApllicationInternal(object):
    self.applicationInternal: Application.ApplicationInternal

    def __init__(self):
      ...```

[unkempt rock]

When I open any .py file, It closes itself. Can someone help me?

[brave badger]

Depending on the type-checker it can either be strict with the null-typing or not; I believe mypy assumes that you're initializing a null attribute later but it automatically makes any type T into Optional[T] for function parameters

[halcyon trail]

Static type systems won't accept None into a non optional type

The preferred way to do it is to refactor so you don't need to initialize something to None and then later initialize it correctly

An alternative approach is a kind of "late init" property

It's backed by an optional, but the getter returns non optional, throws if None

[acoustic mulch]

is it just closing when you run the code?

[static bluff]

Thank y'all

[quaint magnet]

Hey everyone. Apologies if this is the wrong channel for my upcoming question, do redirect me wherever if needed :
I'm currently trying to rename Python's keywords, just for fun, nothing actually useful. After much tinkering I'm at a place where I can rename keywords to anything... as long as it as the same length as the original keyword
Until I'd gotten to that point I'd only been poking around basically aimlessly, fixing problems as they came up, but this one stumped me and I decided to go look for resources on how to do such grammar changes. I've only found a couple of articles that are now quite outdated, and https://devguide.python.org/grammar/ which isn't the most descriptive of resources
Now my question is, is there anybody here that would know about what could cause my current problem with keywords needing to be the same length as the original ones, or about replacing keywords altogether ?
I can post more about exactly what I'm doing and where/how it fails if needed :)

[flat gazelle]

did you edit and rebuild the grammar file

[quaint magnet]

I did. I run a regex to replace 'keyword' with 'newkeyword' and do make regen-all whenever changing the keyword

Ah, I need to validate that but I believe after changing python.gram and regenerating, Parser/parser.c doesn't have my new keyword
I found out about this just before coming here, I was modifying parser.c directly before reading about the fact it's supposed to be generated

[flat gazelle]

hmm, odd. I don't know much beyond you can edit the grammar file to do things

[quaint magnet]

It does regenerate it. Must have bricked something last time I tested that
So I guess that's one thing less to worry about

[sharp flume]

Ok so, trust me when i say replacing all instances of a keyword doesnt work, it usually ends up breaking a lot of things

The best technique to renaming keywords is to alias them instead, which means you don't have to go through the standard library changing every instance of the keyword

then from there the tl;dr is:

• always make clean
• make regen-pegen
• make regen-ast
• make regen-keyword

[sharp flume]

as an example of this, here's making yeet an alias of del: https://github.com/Yeethon/cyeethon/commit/1a54a474a138835812b093fbf301a1d60b5929fd#

[quaint magnet]

Ha, that's all that's needed for an alias ? nice 😄
I'll go with that then. Thank you !

[forest sand]

Maybe a little off topic does anyone have a free api or method for otp sms verification codes?