type-hinting

[wicked scarab]

!d isinstance

[rough sluice]

isinstance

isinstance(object, classinfo)```
Return `True` if the *object* argument is an instance of the *classinfo* argument, or of a (direct, indirect, or [virtual](https://docs.python.org/3/glossary.html#term-abstract-base-class)) subclass thereof. If *object* is not an object of the given type, the function always returns `False`. If *classinfo* is a tuple of type objects (or recursively, other such tuples) or a [Union Type](https://docs.python.org/3/library/stdtypes.html#types-union) of multiple types, return `True` if *object* is an instance of any of the types. If *classinfo* is not a type or tuple of types and such tuples, a [`TypeError`](https://docs.python.org/3/library/exceptions.html#TypeError "TypeError") exception is raised.

Changed in version 3.10: *classinfo* can be a [Union Type](https://docs.python.org/3/library/stdtypes.html#types-union).

Similar names: 2to3fixer.isinstance

[minor nimbus]

Can ParamSpec be used for forwarding a function signature in a subclass? Lets say there's a class in the standard library that has a complicated __init__ signature, but all I want to do is store an additional attribute during initialisation. Something like:

class MySubclass(ComplicatedBase):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._my_attr = ...

I tried this, but it doesn't appear to work:

P = ParamSpec("P")

# also tried with: class MySubclass(ComplicatedBase, Generic[P]):
class MySubclass(ComplicatedBase):
    def __init__(self, *args: P.args, **kwargs: P.kwargs):
        super().__init__(*args, **kwargs)
        self._my_attr = ...

I'm reading through PEP 612 to see if this is explained, but haven't found anything relevant yet.

[mint inlet]

Yeah I don't believe ParamSpec supports that case

You might be able to extract the sig anyways with self-type madness but lol

class InitParams(Protocol[P]):
  def __init__(self, *args: P.args, **kwargs: P.kwargs) -> None: ...

class MySubclass(ComplicatedBase):
  def __init__(self: InitParams[P], *args: P.args, **kwargs: P.kwargs) -> None: ...

That's the first thing I can think of but I doubt it works (I don't think self-types refer to parent type after all).

You could probs make a decorator and @args_like(ComplicatedBase.__init__) above the init

[minor nimbus]

I see. Thank you 🙂

[hasty hull]

Typed **kwargs is coming to pyright 👀 https://github.com/microsoft/pyright/issues/3002

[trim tangle]

https://github.com/microsoft/pyright/issues/2667

[minor nimbus]

So, what I'm getting from it, is that it's not possible?

[oblique urchin]

maybe I need to bite the bullet and merge https://github.com/python/typeshed/pull/6670

[minor nimbus]

If it can break stuff, I don't think it's really worth it. I can review the mypy_primer results though, to see how big of an impact this would make. I tried checking out pandas first and found some cheeky stuff there:

[oblique urchin]

thanks, reviewing the mypy-primer output would be helpful!

I feel like every time I look at pandas code flagged by mypy-primer it is... interesting

[minor nimbus]

lol

Yeah, that name copying there makes no sense, as @wraps should already do that

but I mean, this thing is defined within another function, where the name comes from

seems like you can overwrite the name this way

[trim tangle]

seems like so

[minor nimbus]

It's quite late today and I have things to do, but I'll happily look at this tomorrow

[near kernel]

My compilation times for numba (jit, nopython=True) are very slow, meaning it can take a second to compile a dozen or so lines.

Would type hinting help?

[hearty shell]

https://numba.pydata.org/numba-doc/dev/proposals/type-inference.html

Possibly? Nothing there seems to suggest it though

[hallow flint]

try it and see? numba has a gitter and they're pretty friendly. could be worth asking on there! you could also potentially use ahead-of-time compilation https://numba.pydata.org/numba-doc/dev/user/pycc.html

[near kernel]

I don't know how python type hints work, are there any tutorials you could point me to? Numpy arrays are common in my code.

And the JIT is much better for slow compilation in general. It just is so slow that even JIT is a problem.

[trim tangle]

Type hints on their own don't do anything, it's just metadata on a function. But a tool like numba can do something meaningful with them, so numba's documentation is what I'd look at

(I don't know anything about numba)

[hearty shell]

Also the types you see on a python typing tutorial are probably going to be very different then the ones you need

Python types are too broad iirc for tools like numba

Although Mypyc uses python types I think 🤔

dont mind me

[oblique urchin]

mypyc is a static compiler. I don't know much about Numba but I think it's a JIT, so it probably doesn't need static types much

[hearty shell]

Their working theory is that the jit compilation alone is too slow and that maybe types would take the overhead of having the compiler figure out the types out.

It does use c like types int32, vector and stuff like that though

[near kernel]

Numba is in general an excellent tool, as it is like Cython with no need for C++ compiler (windows and C++ never really mixed) and can JIT with minimal code changes. So if any of you use numpy for performance and occasionally need to write a for loop numba is ideal. It's just the compiler that is so slow, and I assume that figuring out the types is the issue.

[hearty shell]

I had to use it once, it made my computation heavy test at least 10x faster but the error messages are so cryptic

maybe it has gotten better x)

[near kernel]

@empty mural: Yes the error messages take getting used to. There are a couple of things I noticed, one is to use tuples for np.ones((x,y,z)) not np.ones([x,y,z]) and it complains if the array dimensions are different for different fns.

[mortal fractal]

I should start using ParamSpec in real things now...I had put it off because mypy was blocking but i think it's fine now

[near kernel]

And make sure fns called with numba are also numba-jit-nopython'ed

*it complains if the array dimensions are different for different *calls of the same fn

Array dimensions = len(x.shape), it's OK if the sizes are different as long as the number of dims matches

[hearty shell]

Yeah my problem was just too many restriction with little support from the debugger that is all, I might give it another try another time

[near kernel]

You don't need to use it everywhere, just in those blocks of code that have for loops because numpy slices can't do it for whaterver reason.

[mortal fractal]

I've used numba (like 7 years ago) to speed up a hot path in a symplectic integrator and it didn't need any types/Cython integration then

[near kernel]

It doesn't need types, it infers them. But there seems to be somewhat of a combinatorial explosion that makes compilation really slow. So hinting a few key types may help.

Also, numba has an active Github and options for GPU. I do not know how it compares to CUDA but it is an interesting tool. Lets hope for the best going forward.

[mortal fractal]

My experience in the past with gpu acceleration has been rough. Basically the upload/download overhead to GPU was large enough that it was never feasible for me to use it to speed up a hot part of the calculation, but it was too annoying/basically a complete rewrite to try to reprogram the entire calculation to fit within whatever restrictions a GPU thing I was trying out had

It's probably fine if you start with it

[hallow flint]

numba is just compiling to cuda kernels, so they're not really separate things 🙂 if you're interested in writing gpu kernels, i'd also check out https://github.com/openai/triton

if AOT compilation using numba isn't an option, looks like numba also has a cfunc thing that might match what you're looking for? see https://numba.pydata.org/numba-doc/latest/user/cfunc.html but in general you're probably going to get better help at https://gitter.im/numba/numba

[fierce ridge]

you can also access gitter using any matrix client!

[hasty hull]

Is this a pyright bug or are there some edge cases between type and partial? ```py
from functools import partial

class Foo:
def init(self, a: int) -> None:
self.a = a

def foo(a: type) -> None:
...

foo(Foo)
foo(partial(Foo, a=1)) # Argument of type "partial[Foo]" cannot be assigned to parameter "a" of type "type" in function "foo"
# "partial[Foo]" is incompatible with "type"

[hearty shell]

Humm, why should a partial be compatible with type?

[hasty hull]

Wouldn't it have the same runtime semantics?

[oblique urchin]

no

partials just provide __call__, right?

[hearty shell]

they hide the object behind a __func__ var

But even if it did, python typing is mostly nominal

and where it isnt, it is hacked into x)

If you just pass foo(partial) then it works

[oblique urchin]

partial itself is a type

[hearty shell]

Nvm, it is just partial.func, not __func__

https://docs.python.org/3/library/functools.html#partial-objects

[hasty hull]

Ah I misunderstood how partial worked internally

[fierce ridge]

unfortunate limitation imo, it'd be very cool if partial worked like that (but would probably be very complicated to type check and not worth the effort)

[soft matrix]

has anyone noticed pyright has changed unpacking behaviour?

        mine = (1, 2, 3, 45)
        zip((1, 2, 3, 4), *mine)```apparently this is legal?

and type inference seems broken cause that should be Sequence[ParseResult[Path]]

[brisk heart]

Always was?

[ornate gale]

Is there a way to make the type hinting displaying each argument on a new line in VS?

[trim tangle]

I think no, unfrotunately

but... why do you have so many arguments? 👀

[ornate gale]

@trim tangle Ehm, i dont know how to answer that question ? 😄 I mean its a class for a question type and different question types have different opt attributes

[trim tangle]

can you show the code?

[ornate gale]

Sure but dont bash me 😄

class comment:

    def __init__(self, optCommentText:str=None, optCommentIndex:int=None):
        self.commentText = optCommentText
        self.commentIndex = optCommentIndex

    def createComment(self, XMLParent):

        #comment = myXML.Comment(commentText)
        #XMLParent.insert(index, comment) 
        for index,line in enumerate(self.commentText.splitlines()):
            
            #this is just to get the pro formatting to fit on my comments
            removeIndentsFromLine = textwrap.dedent(line)
            lineLength = len(removeIndentsFromLine) + 1
            modified_string = removeIndentsFromLine.ljust(lineLength)
            XMLParent.insert(self.commentIndex+index, myXML.Comment(modified_string)) 

        #XMLParent.insert(self.commentIndex, myXML.Comment(self.commentText))

        return

class question(comment):

    def __init__(self, questionText:str, originalQuestion:str, sequence:str, helpText:str=None, optCommentText:str = None, optCommentIndex:int = None):
 
        comment.__init__(self, optCommentText, optCommentIndex)
        self.questionText = questionText
        self.originalQuestion = originalQuestion 
        self.sequence = sequence
        self.questionNumber = 1000

        #Optional
        self.helpText = helpText

    def setQuestionNumber(self, questionNumber):
        self.questionNumber = questionNumber

class multiChoiceInput(question,extraQuestion):

    def __init__(self, questionText:str, originalQuestion:str, sequence:str, svar:list, minAnswers:str = None, maxAnswers:str = None, helpText:str=None, layout:str=None, optFeedback:feedback = None, optExtraQuestion:extraQuestion = None, optImage:newImage = None, optCommentText:str = None, optCommentIndex:int = None):

        question.__init__(self, questionText, originalQuestion, sequence, helpText, optCommentText, optCommentIndex)
        #extraQuestion.__init__(self, triggerObjectID, onAnswer, extraQuestionText)
        self.questionType = "multiChoice"
        self.svar = svar
        self.layout = layout
        self.optFeedback = optFeedback
        self.optExtraQuestion = optExtraQuestion
        self.optImage = optImage
        self.minAnswers = minAnswers = minAnswers if minAnswers is not None else "1"
        self.maxAnswers = maxAnswers = maxAnswers if maxAnswers is not None else "1"

    def createQuestion(self,XMLsection):

        global questionNumber

        if self.commentText:
            question.createComment(self, XMLsection)
            #createComment(XMLsection,question)

        XMLsection_ = addQuestionIntro(XMLsection, self.sequence)
#....deleted lines because of max limit


        # > XMLsection____ = ___component4_section_entry_organizer_question1_observation_entryRelationship_observation_value
        XMLsection____ = myXML.SubElement(XMLsection___,"value", **{'xsi:type':'IVL_INT'})
        myXML.SubElement(XMLsection____, "low", value=self.minAnswers)
        myXML.SubElement(XMLsection____, "high", value=self.maxAnswers)
  
    
        if self.helpText:
            addHelpText(XMLsection_,self)

        if self.optFeedback:
            addFeedbackText(XMLsection_,self)

        if self.optImage:
            dispayImage(XMLsection_,self)

        if self.optExtraQuestion:
            addExtraQuestionAsTextSupplement(XMLsection_,self)

something like this (this is far from being finished but that should be the structure right now and show a little more information @trim tangle

[trim tangle]

@ornate gale Can you give some examples of valid multiChoiceInput objects?

Why does question inherit from comment?

[ornate gale]

multiChoiceInput(
        questionText = "Kan du besvare et simpelt MC-spørgsmål?", 
        originalQuestion = "Kan du besvare et simpelt MC-spørgsmål?",
        sequence = "1",
        helpText = "Spørgsmålet skal besvares.",
        svar = ['Ja','Nej','Måske på en god dag','Det er ikke relevant for mig'])
    )

and another

multiChoiceInput(
        
        questionText = "Dette (horisontale)", 
        originalQuestion = "Dette",
        sequence = "1",
        svar = ['Ja','Nej','Muligvis'],
        layout = 'horisontal',
        optFeedback = feedback("Hvis du kan bestemme dig, kan du vælge Ja eller Nej","3"),
        optExtraQuestion = extraQuestion("1005", "1", "Hvad er det, du gerne vil supplere med?", """

                                         * Hvad er det, du gerne vil supplere med?                *
                                         * ASSOC.TXT-SPØRGSMÅL-1006                               *

                                         """, 5),
        optCommentText = """ * 
                             * MC-SPØRGSMÅL-1005                                                                                                                                                                                               *                                                                      * 
                             * Dette (horisontale) MC-spørgsmål har et associeret tekst-spørgsmål og en feedback-tekst udløst af den 3. svarmulighed.<br>Vil du give supplerende oplysninger med et associeret tekst-spørgsmål?       """,
        optCommentIndex = 5
        )

every question can insert optional comments inside the XML structure

[trim tangle]

What is sequence?

and what is the difference between questionText and originalQuestion?

[ornate gale]

thats an XML attribute in which order the question is handleded

the sequence

[trim tangle]

Why not just have a list of questions?

[ornate gale]

I do have it as a list, but this python code is generating XML/QFD´s and they need this sequence attribute. Once a chapter is done the sequence starts new

so you can have multiple chapters, multi sections and multi questions inside a question form

def addQuestionIntro(XMLSectionPointer, Sequence):

    # > ___component4___question1 = ___component4_section_entry_organizer_question1
    chapter_question = myXML.SubElement(XMLSectionPointer, "component", typeCode="COMP", contextConductionInd="true")
    myXML.SubElement(chapter_question, "sequenceNumber", value = Sequence)
 
    # > ___component4___question1_ = ___component4_section_entry_organizer_question1_observation
    chapter_question_ = myXML.SubElement(chapter_question, "observation", classCode="OBS", moodCode="DEF")

    return chapter_question_

example of the sequence attribute inside the XML structure

I will create a kivy UI for it, that it´s fairly simple to append new questions or elements to the list

[plucky trench]

I'm trying out type hinting and how it affects the auto suggest in vs code. I see in this screenshot that highlighting a suggestion shows a description. I'd love that! But when I do it, it doesnt show descriptions for any of the methods. What can I do to enable this?

I think I figured it out. This is a feature of a linter!

I lied. Highlighting the above suggestions still doesnt do anything despite installing pylint

[undone carbon]

how can i type hint a dict:

foo = {
    str: (float, float),
    ...
}

[trim tangle]

dict[str, tuple[float, float]]

[undone carbon]

gg tat's fast

[trim tangle]

although, depends on where you want to typehint it

[undone carbon]

function with *args

i don't know how to phrase it

[trim tangle]

If you want it as a function argument, it's better to use Mapping[str, tuple[float, float]] (where Mapping comes from collections.abc)

[undone carbon]

hmm okay thx

[trim tangle]

Wait @undone carbon, Is your dict storing the types, literally?

or does it have strings as keys and tuples of floats as values?

[undone carbon]

def foo(*args: need the type hint here) ...

n the *args would be in the form of tiz

is tat the ans u were looking for?

[trim tangle]

Can you give an example of how you'd call the function?

[undone carbon]

oh ya

result = foo(
    {
        "abc": (1.2, 3.4),
        "def": (5.6, 7.8),
    }
)

@trim tangle

[terse sky]

Mapping[str, tuple(float, float)] is good

for that example

[undone carbon]

hmm okayy thx

[undone carbon]

my func is goin to return multiple values return 123, True, how to type hint it?

[hearty shell]

then it returns a tuple

tuple[int, bool]

[undone carbon]

okay then

thxx

[rustic gull]

I have a function,

def foo(m: int):
    ...

Is there a way to typehint such that I could indicate that the int I am expecting is of unit 'seconds' ?

[hearty shell]

You would have to make a new type for that

[oblique urchin]

you could use a NewType

[trim tangle]

yeah

[hearty shell]

from typing import NewType

Seconds = NewType('Seconds', int)

def foo(s: Seconds):
    ...

a = 1440

foo(Seconds(a))

[trim tangle]

well, depends on what you mean by the units of something

[tranquil turtle]

Seconds = int
def foo(s: Seconds): ...

foo(Seconds(1)) # valid
foo(1) # also valid

[oblique urchin]

yes. that gives you more explicit names for human readers of the code but it doesn't give you type safety

[trim tangle]

I doubt the utility of aliases like these

I would prefer a newtype or: ```py
def foo(*, seconds: int): ...

foo(seconds=1)

Rust has an explicit Duration type which doesn't have the problem of passing around raw integers, I wish Python had this.

oh well, there is datetime.timedelta. But nobody uses it for sleeping

[topaz oracle]

I much prefer NewType in most cases as well @trim tangle . Especially, when I want to be sure that the input value to a function is the output of another function and not just any float.

from datetime import datetime
from typing import NewType

DateInSeconds = NewType("DateInSeconds", float)


def get_date_in_seconds() -> DateInSeconds:
    dt = datetime.today()
    seconds = dt.timestamp()
    return DateInSeconds(seconds)


def print_seconds_to_date(seconds: DateInSeconds) -> None:
    print(datetime.fromtimestamp(seconds).strftime("%A, %B %d, %Y %I:%M:%S"))


if __name__ == "__main__":
    today_in_seconds = get_date_in_seconds()
    print_seconds_to_date(today_in_seconds)

[blazing nest]

Can anyone explain to me why str doesn't have __int__()? I am using SupportsInt at a bunch of places and I have first now realized this incompatibility. ```python
from typing import SupportsInt

def a(b: SupportsInt) -> None: ...

a('123')

main.py:5: error: Argument 1 to "a" has incompatible type "str"; expected "SupportsInt"
Found 1 error in 1 file (checked 1 source file)

Locally, in my real code, Pyright is giving me the same type of issue: ```
Argument of type "Union[int, str]" cannot be assigned to parameter "id" of type "SupportsInt" in function "__init__"
  Type "Union[int, str]" cannot be assigned to type "SupportsInt"
    "str" is incompatible with protocol "SupportsInt"
      "__int__" is not present

[soft matrix]

https://github.com/python/cpython/blob/main/Objects/longobject.c#L5296 its just special cased

[rough sluice]

Objects/longobject.c line 5296

return PyLong_FromUnicodeObject(x, (int)base);```

[void panther]

The conversion is done by the int builtin

I'd check what that has in typeshed, there's a bunch of fallbacks for it if __int__ is not defined too

assuming that's what you want

[blazing nest]

Yeah str doesn't have __int__() (copied from my typeshed-fallback): https://paste.pythondiscord.com/jetayayage

[unreal bolt]

yeah it doesn't, I usually just type alias something like IntConvertable = str | bytes | SupportsInt | SupportsIndex | SupportsTrunc since int() supports all those types

[soft matrix]

i personally dont think just implicitly converting something to an int is a good idea

cause its less obvious if the user is being silly and passing something like "not an int" to your function

[blazing nest]

I'd usually agree that explicitly requiring an integer is often better by using : int although in this case I have some custom-made objects I want to accept that have __int__().

I was able to work around this, since I don't expect the user to pass around strings and the place that I found the issue I could just int() the argument directly (that'd probably be better either way, even if str had __int__())

[undone carbon]

is it possible to type hint the contents of an ndarry?

np.array(
    [
        (float, float),
        (float, float),
        ...
    ]
)

[hearty shell]

Yes, you need to import their special types

numpy added them a while ago

https://numpy.org/devdocs/reference/typing.html

[undone carbon]

which section...

[hearty shell]

Of the doc?

[undone carbon]

ya... kinda lazy 🙃

if u r busy i'll find myself

[hearty shell]

npt.NDArray[np.float64]

Look like this is for an array of floats

Just just extrapolate that to your example?

Idk, I don't use numpy, just read about them adding types a while ago lol

[undone carbon]

k thxx

[mortal fractal]

huh, I wonder how black ends up running an old version with my vim plugin

I assumed it was running my out of date system black instead of my venv, but now both my system and venv are upgraded

so it's somehow still finding an old version somewhere 😄

[solemn sapphire]

If I have a module with a lot of classes, and a list can contain instances of those classes how do I type hint that list?
I could do list[class1 | class2 | class3 | ... | classn]
But there should be a better way 🤔

[hearty shell]

Do they all share some functionality?

[solemn sapphire]

Yup

All if them inherit from one class

[hearty shell]

Oh

then just list[BaseClass]

[solemn sapphire]

Oh!

Thanks!

[hearty shell]

wait wait

no

sorry

Sequence

@solemn sapphire

!docs collections.abc.Sequence

[rough sluice]

collections.abc.Sequence

class collections.abc.Sequence``````py

class collections.abc.MutableSequence``````py

class collections.abc.ByteString```
ABCs for read-only and mutable [sequences](https://docs.python.org/3/glossary.html#term-sequence).

Implementation note: Some of the mixin methods, such as `__iter__()`, `__reversed__()` and `index()`, make repeated calls to the underlying `__getitem__()` method. Consequently, if `__getitem__()` is implemented with constant access speed, the mixin methods will have linear performance; however, if the underlying method is linear (as it would be with a linked list), the mixins will have quadratic performance and will likely need to be overridden.

Changed in version 3.5: The index() method added support for *stop* and *start* arguments.

[solemn sapphire]

Sequence[BaseClass] ?

[solemn sapphire]

Ah I'll look into this!

[hearty shell]

Yeah, I doenst work on list because for lists list[Child] is not a subtype of list[Parent]

even though Child is a subtype of Parent

[solemn sapphire]

Hmm

[hearty shell]

This is called invariance

[solemn sapphire]

Oh I was reading PEP 484

They were talking about invariance an co variance with TypeVars

I wonder if I can make a TypeVar and the do list[<That typevar>]

@hearty shell Does this make sense?

[hearty shell]

TypeVars are not really for that, also you can use lists, you might just get unwanted annoyances from mypy when you try to assign a list of some of those classes to another, which is way I would just use Sequence

[solemn sapphire]

Yup I got no compliants with Sequence!

I would like to learn more about invariance, couldn't find anything.
You have any resource that I can read up?

[hearty shell]

Yup

https://decorator-factory.github.io/typing-tips/tutorials/generics/variance/

[solemn sapphire]

Thanks again!

[hearty shell]

Np! And that resourse is made by one of the regulars here :P

[solemn sapphire]

Oh!

[fierce ridge]

this belongs in #tools-and-devops or #editors-ides , but check $PATH from inside vim

[mortal fractal]

it turns out the black vim plugin installs its own venv in a location separate from the plugin that isn't upgraded with the plugin, so you have to update it separately (and uninstalling/deleting and reinstalling the plugin won't update it)

[soft matrix]

@hearty shell I've started working again on Self in mypy but the example you've given doesn't seem to type check using the old form (I also tried using @classmethod and @property where it works slightly more but it still doesn't like it)

[hearty shell]

If it works in other examples maybe it is because of the other bug? Where it expects the protocol to use T as contravariant and K as covariant

[soft matrix]

https://mypy-play.net/?mypy=latest&python=3.10&gist=5f510a0e743dbdebe639d5363752e7e8

my implementation is a little bit buggy https://github.com/python/mypy/pull/11666#issuecomment-1044626259 judging from the number of regressions but yeah i could make it work if i made it just a classmethod or a property

[hearty shell]

That example, while it works in pyright it is a little contrived, since now you can just chain the property and classmethod descriptors. Could just make an edge case for that, since it is a common ORM pattern

[fierce ridge]

i have noticed other plugins doing this too. it probably uses python3_host_prog to create a fresh venv upon install; maybe you need to configure your plugin manager to run the update process upon updating the plugin?

[soft matrix]

i think it might be fixed by https://github.com/python/mypy/pull/11719

[mortal fractal]

I need a IF_MYPY IF_NOT_MYPY so I can use recursive type aliases without breaking mypy 😛

[oblique urchin]

I think MYPY = False; if MYPY: actually works

they introduced that before typing.TYPE_CHECKING existed and I'm pretty sure support was never removed

[mortal fractal]

ah, that would probably work then

MYPY = False
if not MYPY:
    GraphItem = Union[tuple[str, list["GraphItem"]], int]
else:
    GraphItem = Union[tuple[str, list[Any]], int]

This works, but only because pyright picks up the first alias only

so if I do if MYPY it doesn't work

also pyright will throw an error if I annotate it with TypeAlias, because it doesn't like assigning a TypeAlias more than once

not the worst though

[fierce ridge]

i'd rather have some standard for typing.CURRENT_TYPE_CHECKER

[soft matrix]

id rather everything had support for recursive type aliases :)

apart from that i think it sets a bad presidence

[oblique urchin]

if all type checkers worked exactly the same what would be the point of having multiple

[trim tangle]

so that when <colleague_name> writes his project in PyCharm, and then I open it in VSCode, I don't see the squiggly lines 😭

/hj

[hearty shell]

I thought there was just multiple of them because of a general sentiment of "not good enought, we can make it better". Ideally speaking, they should all resolve types the same way no?

[oblique urchin]

Innovation is important. One advantage of multiple type checkers is that they can innovate and introduce new features.

Though I do agree that it's useful to standardize areas where they disagree

Once assert_type() exists and I'm thinking of creating a cross-type checker test suite that will make it easier to compare behavior

[hearty shell]

But then I would it is not unreasonable for us to want (not expect) other type checkers to catch up x)

[fierce ridge]

very true

[fierce ridge]

all the more reason to have a standard way to figure out which type checker is currently running!

[trim tangle]

if MYPY:
    os.system("pip uninstall mypy")
    os.system("pip install pyright")

[hearty shell]

😂

[trim tangle]

wait, it doesn't work at runtime

nevermind

[hasty hull]

pip install pyright 👀

[tranquil turtle]

pyleft

[trim tangle]

pywrong

[oblique urchin]

yourpy

[leaden oak]

pyinTS

[trim tangle]

ourpy

[leaden oak]

sharedpy

[trim tangle]

myjavascript

[dapper trench]

Hey guys! I created a suggestion for PyCharm to improve AWS boto3 usage. Please let me know what you think and if you agree. Thank you!

https://community-support.jetbrains.com/hc/en-us/requests/3837777

[clever bridge]

If I want to dynamically add a function with a known type signature (the arguments don't change, just dynamically creating the body in a decorator), what would be the best way to go about that so that I don't get type errors whenever I call it?

A protocol class?

[hearty shell]

Could you provide an example 👀

[clever bridge]

👀

Hello there

[hearty shell]

👋🏻 😂

[clever bridge]

I'm using attrs for attributes, but I want to create a save method from the __slots__ instead of making it manually

So like

def save(obj: T) -> T:
  # adds save method
  return obj

@save
@attrs.define
class Nation:
  id: int
  ...

I'm thinking I need some sort of typevar/protocol

Pass in with one protocol, and it goes out with the typevar plus whatever the protocol adds

Problem is I have no idea what to do

Oooo, I just found Concatenate

Aw no, not even close 😦

[hearty shell]

You can't concatenate to an object, thinking about the way to go about it

[clever bridge]

Concatenate would be useful though

Oh my god TypeGuard is awesome

[hearty shell]

Why not use inheritance for this? I am not sure you can do much here apart from creating virual classes that have the methods you are adding and the methods already present on Nation

[clever bridge]

I could, but I'm trying to optimize a bit when there's a million instances of classes hanging around

[hearty shell]

I mean it would be the same thing

class SupportsSave:
    def save(self): ...

@attrs.define
class Nation(SupportsSave):
    ...

[clever bridge]

Yeah

It's probably stupid, but I'm trying to avoid lots of multi-inheritance and whatnot when I'm going to make hundreds of thousands or millions of class instances

[hearty shell]

I think this thread discuses what you would need to make adding those methods via a decorator a practical option

https://github.com/python/typing/issues/213

Otherwise, I think you will just be adding more complexity then necessary

[hearty shell]

Also generally speaking I think one would consider mixin inheritance sensible inheritance, I wouldn't avoid it in this case, it is definitely more of what you would expect then using a method that you have to chase down a decorator to see where it is coming from

[clever bridge]

I know it's sensible, I'm just micro optimizing 😛

And I'm going to use a decorator either way to create queries once

[hearty shell]

Will they add methods?

[clever bridge]

It's one decorator to add a save method

Which is just

def save(self) -> None:
  ...

[hearty shell]

https://github.com/python/typing/issues/213#issuecomment-983981492

[clever bridge]

That would be wonderful

[hearty shell]

3rd point, that is exactly what you want, but it is currently not supported unfortunately , so if you want the type checker not to complain you will have to make a mixin class

[clever bridge]

😭

I'll prolly just put

async def save(self) -> None:
  ...

In all the classes

Not worth a mixin for just that I don't think

[hearty shell]

That is the other option x)

[clever bridge]

Yeah 🙂

Thanks for all the help! Hopefully we'll see an Intersection PEP soon.

[hearty shell]

np! That would be great, new typing features are always welcome xD

[clever bridge]

Yup 🤣

[brisk heart]

error: Argument 2 to "isinstance" has incompatible type "_SpecialForm"; expected "Union[type, Tuple[Union[type, Tuple[Any, ...]], ...]]"

if isinstance(annotation, Annotated)

🤔

[soft matrix]

you shouldnt be using Annotated to check for instances of Annotated

[brisk heart]

is _AnnotatedAlias supposed to be used despite being private or smth?

[soft matrix]

AnnotatedGenericAlias = type(Annotated[str, int])
isinstance(x, AnnotatedGenericAlias)
```do this

[brisk heart]

the actual types in it don't matter right?

[soft matrix]

no

[brisk heart]

yeah looks like that solved it

thanks mate

[tacit escarp]

A whole channel on just type-hinting? wow

[trim tangle]

is it more surprising than #esoteric-python? 🙂

[tacit escarp]

Hmm not really, I see alot of channles for weird fetishes all the time

[hearty shell]

Lmao

[trim tangle]

isn't typing just another weird fetish?

??

[hearty shell]

[tacit escarp]

Ok lets just pretend this convo never happened I don't know what I am saying bye

[fierce ridge]

is "type hinting" some fetish lingo?

[buoyant swift]

"so...what's your type 😳"
"um...int?"

[trim tangle]

def psvm() -> Intersection[VisualLearner, INTJ, Pisces]:
    ...

[buoyant swift]

why not a typedef or something

[trim tangle]

is it really the question you're asking 🤔

[buoyant swift]

idk lol ¯_(ツ)_/¯

[blazing nest]

I have this code, although I find it pretty cursed. Any tips? ```python

@overload
@classmethod
def builder(cls, id: SupportsInt, type: PermissionTarget, *, create_instant_invite: Optional[bool] = ...) -> Self:
    # For simplicity there's only one kwarg here, but in reality there's 36.
    ...

@overload
@classmethod
def builder(cls, id: SupportsInt, type: PermissionTarget) -> Self:
    # This is just provided as to allow us to use @overload. Since you need
    # more than one method to use @overload, otherwise it wouldn't be an
    # overload to the static type checker.
    ...

@classmethod
def builder(cls, id: SupportsInt, type: PermissionTarget, **kwargs: Optional[bool]) -> Self:
    allow, deny = 0, 0

    for option, value in [(k, v) for (k, v) in kwargs.items() if v is not None]:
        flag = getattr(cls, option)
        if value is True:
            allow |= flag
        else:
            deny |= flag

    return cls(
        id=int(id),
        type=type,
        allow=Permissions(allow),
        deny=Permissions(deny),
    )

I want to receive the kwargs as the **kwargs dictionary, but need (well, want) full typing of all kwargs.

I guess it works, just seems super odd.

[fierce ridge]

doesnt seem that bad to me tbh

[blazing nest]

Yeah like I am somewhat happy with the hack I figured out - but it's a hack overall that may be a bit unclear. Wasn't expecting much of a response but wanted to see if there's another direction you can take that I haven't thought about.

[soft matrix]

id personally just # type: ignore on the error that says you need multiple overloads

[blazing nest]

Yeah good idea that way I don't need to waste that

[hallow flint]

if you don't care about type checking the body of your method, if TYPE_CHECKING works well

(also if you want to use the same trick, but you don't have a good no-op method, you can use something like def builder(cls, _use_previous_def_for_type_checking: int) -> NoReturn: ... )

[hasty hull]

Pyright just added support for typing **kwargs, if you don't need to target multiple type checkers that could be an option

[wicked scarab]

how to make so type checkers handle setattr to not raise "unresolved attribute"?

a = lambda: 1
setattr(a, "b", 2)
a.b  # warning

[hearty shell]

a = lambda: 1
setattr(a, "b", 2)
a.b  # type: ignore

Problem solved x)

type checkers cant really understand this sort of dynamic code

[wicked scarab]

that's the only way?

[hearty shell]

Well, I think the reverse question might be more useful, is that the only way you can express this? If you want to make a callable that has an attribute b, you can just make a callable class

The problem is that this goes against the idea of static checkers, what you are doing is only possible in a dynamic "context"

[wicked scarab]

how if I want to make like

class Foo:
  def __init__(self, thing):
    setattr(self, thing, "some value")

a = Foo("hi")
print(a.hi)

[hearty shell]

You can't, but you can hack it to make it somewhat possible, but Idk if it suits what you are trying to do

one sec

What would be the type of "some value"?

[wicked scarab]

any

[hearty shell]

What decides what that value is?

From what you have showed it just seems like a static value

[wicked scarab]

class ClassDict:
  def __init__(self: Self, d: dict):
    self.convert(d)

  def convert(self: Self, d: dict) -> Self:
    for i, j in d.items():
        if isinstance(j, dict):
            setattr(self, i, self.convert(j))
        elif isinstance(j, (tuple, list, set, frozenset)):
            setattr(self, i, type(j)(self.convert(v) if isinstance(v, dict) else v for v in j))
        else:
            setattr(self, i, j)
    return self


test = ClassDict({"a": 1, "b": 2})
print(test.a)
``` the real code is something like this

when accessing the attribute it always raises a warning which sucks

[hearty shell]

I mean, if you want to get rid of it, here is a solution, but I would make it more robust by using generics

from typing import Any

class ClassDict:
  def __init__(self: Self, d: dict):
    self.convert(d)

  def convert(self: Self, d: dict) -> Self:
    for i, j in d.items():
        if isinstance(j, dict):
            setattr(self, i, self.convert(j))
        elif isinstance(j, (tuple, list, set, frozenset)):
            setattr(self, i, type(j)(self.convert(v) if isinstance(v, dict) else v for v in j))
        else:
            setattr(self, i, j)
    return self

  def __getattr__(self, attr) -> Any:
    raise AttributeError



test = ClassDict({"a": 1, "b": 2})
print(test.a)

Although, without Generics, this is as good as not using a type checker at all for that particular class x)

[wicked scarab]

wait what does that actually do

making __getattr__ to always raise AttributeError

[hearty shell]

Nothing, your class will work the same way, you are just adding ambiguity to the type checker so it cant be sure if the attribute exists or not

[hearty shell]

accessing an attribute doesnt directly go to __getattr__ it first goes to __getattribute__ and then does a bunch of other things until at last it fallsback to __getattr__ if it is defined

So if an attribute isnt defined, this will just raise an AttributeError, just like it would before

[wicked scarab]

ahh okay thanks, that worked fine

[blazing nest]

Hmm, I'd still have 36 lines of code with all option 🤔

[soft matrix]

weird thought, isnt this not entirely true as it could be Iterable | Iterable & AsyncIterable?

maybe its worth looking into if intersections ever actually happen

[trim tangle]

oops sorry

[soft matrix]

yeah but wouldnt isinstance((Iterable & AsyncIterable)(), Iterable) pass?

[trim tangle]

Iterable & AsyncIterable is a subtype of Iterable, so Iterable | (Iterable & AsyncIterable) is just Iterable

just like Iterable | list is just Iterable

[soft matrix]

is it?

[trim tangle]

I guess pyright marks it as that, but conceptually it's the same

There aren't any elements of Iterable that are not in Iterable | list, and vice versa

[soft matrix]

ig so

[trim tangle]

pyright often avoids "normalization" because very often it's an unneeded performance hit, I think

[rustic gull]

is this a new channel??

it wasn't there last time i checked

[rustic gull]

hmm

[hearty shell]

No one notices typing x)

[acoustic thicket]

whoa didnt realize its close to 5 months old now

[median ledge]

I have a function like

async def foo(self, keywords: KeywordsConverter):

This is for a discord bot and, internally, the library converts str automatically, using KeywordsConverter.convert(), before passing the arg to the method:

Keywords = List[List[str]]

class KeywordsConverter(commands.Converter):
    async def convert(self, ctx: commands.Context, argument: str) -> Keywords:

How can I tell the type checker that keywords will actually be of type Keywords, not KeywordsConverter?

[soft matrix]

if typing.TYPE_CHECKING:
    KeyWordsConverter = KeyWords
else:
    # actual keywordsconverter impl.

thats the best way to do it at least on vanilla dpy

[median ledge]

OH, I see

thanks!

[solid kettle]

no Annotated support 😦

[mortal fractal]

Hmm, you can sorta make NewType compose but I wonder how a dimensionally correct implementation could look

As in a/b for a dist and b time gives velocity, or such, and you could hint something as velocity and have enforced by type checkers

I'm not sure this is reasonably possible with python system, but I think I can do it if generics could be over values too

Let me think about it

[dire bobcat]

how do you do t.List[t.List[int]]

i've got a list of ints inside a list

it says it received too many args or smth

[trim tangle]

Can you show the code and the error?

[dire bobcat]

sure, one mo

crap it works

i did something right

whelp

[trim tangle]

🙂

[void panther]

if using float specific methods, how do you specify a param can be only a float, and not an int?

so ```py
def f(a: float):
a.is_integer()

f(5)

wouldn't pass

[oblique urchin]

you cannot with the current type system. Though you can do it inside of containers thanks to invariance: int is compatible with float, but list[int] is not compatible with list[float]

[hallow flint]

in theory, i think you could with a constrained typevar. but it doesn't seem to work with mypy

from typing import TypeVar

T = TypeVar("T", float, float)

def foo(a: T):
    a.is_integer()
  
x: int
y: float
foo(x)
foo(y)

[hearty shell]

TypeVar still allows subtypes though

[oblique urchin]

I don't think so. The typevar gets solved to float, but that still allows you to pass subtypes of float

Constrained TypeVars are weird 🙂

[hallow flint]

jelle is correct. clearly i haven't written enough str subclasses 🙂

[hearty shell]

You could make a protocol though, and then add all the methods that float has, but that is probably cumbersome x)

[hallow flint]

oh that's a nice solution. you'd only need to add a method that int doesn't have, so just a protocol for is_integer should work

[hearty shell]

You would have to add all methods you need to make use of I think no?

Oh wait no you are right

[hallow flint]

unfortunately, doesn't seem to work 😕

the duck typing is too strong!

[hearty shell]

But why

I am very confused x)

[oblique urchin]

Mypy bug? Pyright does flag this: https://mypy-play.net/?mypy=latest&python=3.10&gist=fd4c74a0e9a4d3bd1dcb34b6b2d7614c

It's weird because mypy does yell at you when you do (1).is_integer() directly

[north palm]

why do we need type-hinting?

[hearty shell]

fromhex also passes for int in mypy

[north palm]

I use vim, so no ide goodies

[buoyant swift]

you don't use a LSP?

[north palm]

ohh LSP seems interesting(googling)

how does LSP compare to like pycharm?

[hearty shell]

Yeah, all of these type check for some reason

from typing import Protocol

class Float(Protocol):
    def fromhex(cls, str) -> float: ...
    def is_integer(self) -> bool: ...
    def hex(self) -> str: ...
    
def foo(a: Float): ...
  
x: int
y: float
foo(x)
foo(y)

Maybe it is a side effect of elevating float to be a supertype of of int by mypy, and then exceptions are made for stuff like (1).is_integer()

[oblique urchin]

yeah it's definitely related to the weird special case where int is a subtype of float but not really. I guess the Protocol code takes a different path there than the normal attribute lookup

But good luck actually figuring that out 🙂

[mortal fractal]

https://github.com/microsoft/pyright/issues/3089

We're going to get so many bugs by using functions as types instead of the rejected callable syntax

Because people don't add / to the function parameters

Maybe if everyone includes a / in all the standard teaching examples it won't become a huge problem

[rustic gull]

from __future__ import annotations
from collections import defaultdict
TrieNode = lambda: defaultdict(TrieNode)
# main.py:3: error: Need type annotation for "TrieNode"
# main.py:3: error: Cannot determine type of "TrieNode"
# Found 2 errors in 1 file (checked 1 source file)

how do I annotate this?

[blazing nest]

What's the type of the default dictionary?

[rustic gull]

Dict[str, TrieNode]

[blazing nest]

I think something like this? Hold on let me test it a bit closer

from __future__ import annotations

from collections import defaultdict
from collections.abc import Callable
from typing import Any, TypeAlias

TrieNodeType: TypeAlias = "defaultdict[str, defaultdict[str, Any]]"

TrieNode: Callable[[], TrieNodeType] = lambda: defaultdict(TrieNode)

Ah right, this is Mypy

You can't have recursive types

edited. Now you at least have a depth of 2

[acoustic thicket]

what are you trying to achieve with this?

[rustic gull]

it's a trie

from collections import defaultdict
from functools import reduce
TrieNode = lambda: defaultdict(TrieNode)
class Trie:
    def __init__(self):
        self.trie = TrieNode()
    def insert(self, word):
        reduce(dict.__getitem__, word, self.trie)['end'] = True
    def search(self, word):
        return reduce(lambda d,k: d[k] if k in d else TrieNode(), word, self.trie).get('end', False)
    def startsWith(self, word):
        return bool(reduce(lambda d,k: d[k] if k in d else TrieNode(), word, self.trie).keys())

[acoustic thicket]

i meant the TrieNode = lambda: defaultdict(TrieNode) line specifically

hmm maybe its fine, it just looks like an odd structure

[rustic gull]

TrieNodeType = Dict[str, "TrieNodeType"]
TrieNode: Callable[[], TrieNodeType]= lambda: defaultdict(TrieNode)

pyright seems happy with this, look about right? ive not annotated a recursive type before

[blazing nest]

Yes this looks right, it won't pass on Mypy though

[dire bobcat]

t.Optional[t.Tuple(int, int)]:

TypeError: Type Tuple cannot be instantiated; use tuple() instead

i tried t.Tuple[int, int] as well

does it want me to do tuple()?

seems a bit odd...

[hearty shell]

t.Tuple[int, int]

[oblique urchin]

tuple[int, int] is right

[hearty shell]

what happend when you did that?

[dire bobcat]

same error

[dire bobcat]

oh

ok

[oblique urchin]

(or t.Tuple[int, int] in older versions)

[dire bobcat]

i did that previously

[hearty shell]

Humm, you should not get this error with that TypeError: Type Tuple cannot be instantiated

are you sure?

[dire bobcat]

i think so>

tuple[] works

[hearty shell]

Do you need to target older then 3.9?

[dire bobcat]

no

[hearty shell]

alright

[foggy thicket]

What is the actual way to typehint higher order functions?
I have something that looks like this with Callable : just the outermost function would look like Callable[..., Callable[..., Callable[..., Callable[..., ...]]]]

that doesn't look very good

[chilly kindle]

Perhaps this is the type hinting trying to hint to you that this may not be the most suitable abstraction

[foggy thicket]

This is what I was trying to typehint :
https://paste.pythondiscord.com/hunisuvuno

[hearty shell]

What is commands.when_mentioned_or

[proven fog]

#discord-bots

[foggy thicket]

wait, no it's a different context here

[hearty shell]

Bruh, I am reading the paste xD

So it is from discordpy I assume

[proven fog]

?

[foggy thicket]

commands.when_mentioned_or takes a variadic of strings and returns a callable which takes two other positional arguments

example :

async def get_prefix(bot, message):
    extras = await prefixes_for(message.guild) # returns a list
    return commands.when_mentioned_or(*extras)(bot, message)

[proven fog]

I don’t understand. Does when_mentioned_orexist outside dpy? Or are we talking about it in dpy

[foggy thicket]

we were talking about typehinting when using that function

[proven fog]

Oh

[hearty shell]

def when_mentioned_or(*prefixes: str) -> Callable[[Union[Bot, AutoShardedBot], Message], List[str]]:

So, I assume yours your just pass that forword

One sec

[foggy thicket]

In my case I am guaranteed to not have an instance of AutoShardedBot so I think I could replace that union with just Bot

[hearty shell]

Humm, maybe? Not sure how that interacts with dpy internal checks

[foggy thicket]

I see, then we could keep that :p
also couldn't we use | instead of Union ?

[hearty shell]

In 3.10 yes

Also you think the type hints are a little unmanageable you can always use type aliases

[foggy thicket]

AliasWhenMentioned : TypeAlias = Callable[[BotBase, Message], List[str]]

seems better, thanks for the tip

[hearty shell]

Nice profile pic btw :)

[foggy thicket]

Thank you :D
it from here : https://en.wikipedia.org/wiki/Journey_(2012_video_game)

[hearty shell]

Yeah I know xD

[mortal fractal]

@oblique urchin huh, is this a variance thing?

# pyright: strict
TEST_MAP: dict[str | None, str] = {"4": "5"} | {"a": "b"}

surely this is just a pyright bug

[trim tangle]

Well... the type of the right hand side is dict[str, str]

which is not assignable to dict[str | None, str]

[mortal fractal]

it goes away if you remove the None, but adding None has a useful purpose

[mortal fractal]

that's not the error it gives 😄

[trim tangle]

oh

Operator "|" not supported for types "dict[str, str]" and "dict[str, str]"

lmao

[mortal fractal]

pyright is fine with TEST_MAP: dict[str | None, str] = {"4": "5"}

[trim tangle]

yeah it's kinda confusing

[mortal fractal]

I'm going to report this, there's surely a bug in one of the behaviors

[void panther]

If I want to add a builtin is editing the stub the only option?

[mortal fractal]

@trim tangle anyway, if you annotate the input as possibly being None, it lets you use it with .get(None, "default")

without type complaining

or you might add a none key later in this case

[trim tangle]

I know that it's different

but in this case you can just do TEST_MAP: dict[str | None, str] = {"4": "5", "a": "b"}

[mortal fractal]

it came up in code where the first dict was a comprehension

[trim tangle]

ahh

[mortal fractal]

and I wanted to annotate it Final so I couldn't split it

[trim tangle]

what about **?

[mortal fractal]

yes, like {**comprehension, **dictliteral}?

I didn't try it but I assume it would work

https://github.com/microsoft/pyright/issues/3098

from typing import Final
test_map: Final = {"4": "5"}
test_map |= {"a": "b"}

no error from pyright

maybe it's intentional and pyright considers it like a dict update as opposed to an assignment

I'll mention it in the bug to ask just in case

[blazing nest]

I have a dictionary like this: ```python
class Example(TypedDict):
field: str

class ExtendedExample(TypedDict):
extra: int

class Wrapper(TypedDict):
data: Union[List[Example], List[ExtendedExample]]

How would I naturally narrow down to `List[ExtendedExample]`?
At runtime, code like this should work: ```python
data: Wrapper = ...
if data['data'] and 'extra' in data['data'][0]:
    # I now expect data['data'] to be List[ExtendedExample]
else:
    # Here the type would be Union[List[Example], List[ExtendedExample]] still
    # if the list was empty and the latter comparison never ran

[mortal fractal]

ah right I think this is always the pyright beahvior

a: Final = 4
a += 1

this type checks in pyright

surely you should at least error on assignment operators for immutable data types, no?

[hearty shell]

I think type checkers might not support that? By "naturally" do you mean without using a TypeGuard?

[blazing nest]

I mean some form of logic other than an assert

Like how I tried with an in operation

[hearty shell]

Actually because TypedDicts arent generic I am not sure that is even possible with TypeGuard

[hearty shell]

Are you only concerned with the data inside the wrapper or the whole wrapper changing its type?

Otherwise I dont think it is possible to narrow from

class Wrapper(TypedDict):
    data: Union[List[Example], List[ExtendedExample]]

to

class Wrapper(TypedDict):
    data: List[ExtendedExample]

implicitly

[blazing nest]

Only data inside the wrapper

[trim tangle]

Unfortunately this doesn't work, even with @final

(you need final because https://github.com/microsoft/pyright/issues/1899)

[hearty shell]

I mean doing something like this would somewhat work

def is_extended(inner) -> TypeGuard[List[ExtendedExample]]:
    return 'extra' in inner

data: Wrapper

inner_data = data['data']

if is_extended(inner_data):
    reveal_type(inner_data)
else:
    reveal_type(inner_data)

It doesn't narrow the else branch though

[trim tangle]

class Example2(Example):
    extra: str

[hearty shell]

What do you mean?

[blazing nest]

Yeah that's fine for me. I'll look into TypeGuard though

[trim tangle]

example2s: list[Example2] = [{"field": "foo", "extra": "yeah"}]
examples: list[Example] | list[ExtendedExample] = list(example2s)

if is_extended(examples):
    plus_one = [x['extra'] + 1 for x in examples]  # TypeError: can only concatenate str (not "int") to str

[hearty shell]

I am still not sure what do you mean by this 😅

Ohh I think I know what you mean

def is_extended(inner: List[Example] | List[ExtendedExample]) -> TypeGuard[List[ExtendedExample]]:
    return 'extra' in inner

fixed x)

[trim tangle]

class Example(TypedDict):
    field: str

class ExtendedExample(TypedDict):
    extra: int

class Example2(Example):
    extra: str

If you have e.g. a Union[Example, ExtendedExample], you can't narrow it down to ExtendedExample just by checking if the "extra" key is there

[hearty shell]

You cant extend a typedict and then overwrite it though

[trim tangle]

You can inherit a TypedDict

[hearty shell]

But not overwrite a key

I mean you can at runtime

[trim tangle]

example2s: list[Example2] = [{"field": "foo", "extra": "yeah"}]
examples: list[Example] | list[ExtendedExample] = list(example2s)

if is_extended(examples):
    plus_one = [x['extra'] + 1 for x in examples]

do you understand why this will type check but fail at runtime?

[hearty shell]

Yes I do

[trim tangle]

If something is an Example, it doesn't mean that it doesn't have an extra key

[hearty shell]

But that does not satisfy the contract

[trim tangle]

class Example(TypedDict):
    field: str

class Example2(Example):
    extra: str

I'm not overwriting any existing keys

[trim tangle]

what contract?

A TypedDict is considered open (meaning that it allows any other keys) unless it's @final

[hearty shell]

Ahhhh, okay, I thought you were overwriting the keys over there

Yeah I understand the issue now

that is why you mentioned final

[trim tangle]

overwriting keys is prohibited, I think

at least at the type level

[hearty shell]

Yeah it is, I am just blind

x)

[trim tangle]

it happens 🙂

I once filed a pyright bug because I forgot to update

it was rather embarassing

[hearty shell]

I imagine what erictraut might have said xD

[trim tangle]

he was very polite

[acoustic thicket]

You dont typically annotate assignments as simple as that

Its more for function signatures, assignment to a long complicated expression, etc

[hearty shell]

Well, you dont have to. Unless you are working with them xD

That annotation is implicit anyway, but it is still there

[acoustic thicket]

Yeah

Another assignment i annotate is x: Optional[T] = None

Since just None doesnt inform you that it might become T later

[hearty shell]

Also, while Python might be a dynamic language, that doesn't mean some people dont want types to be enforced before runtime. The nice thing is that it is completely optional

JavaScript is also supposed to be a dynamic language

Yet TypeScript is well and alive

[rustic gull]

suppose i have a dictionary with structure : py { "1" : {'a': "a"}, "2" : 2, "3" : "3" } how do i typehint it ?

Dict[str, Union[Dict[str, str], Union[str, int]]] ?

Dict and Union are imported from typing ofc

[acoustic thicket]

looks fine

[trim tangle]

Union[A, Union[B, C]] = Union[A, B, C] 🙂

So you could have: ```py
Dict[str, Union[Dict[str, str], str, int]]

(if you're on 3.9+ you should do ```py
dict[str, Union[dict[str, str], str, int]]
```)

But why do you have such a structure? what does it do?

[rustic gull]

oh yea right

and the dictionary is actually a json response i get from https://pokeapi.co/ ( you could see the structure on the main page ) and im caching it in a dictionary

which is like py my_dictionary ={ "id for which the response is" : response }

[trim tangle]

Ah, that's a different situation. If you want a dictionary with fixed keys, you can use TypedDict.

dict[str, Union[dict[str, str], str, int]] doesn't really help the reader or the type checker

Another, even better option, is to keep the responses as opaque objects, and then parse them into proper objects.

Like dict[str, PokeResponse]

[rustic gull]

mm yeah that seems like a better option

thanks!

[proud brook]

If you are on Python 3.10 you can do

dict[str, dict[str, str] | str, int]

[trim tangle]

yeah ```py
dict[str, dict[str, str] | str | int]

[terse sky]

if you're writing new code, rather than use typed dict I'd strongly suggest just pulling the dict into a dataclass or something similar

typed dict is mostly for type annotating existing code, shouldn't be something to use too often in new code

[rustic gull]

dict[int, str]

this means it will get a key as an int and a str as a key?

[hearty shell]

str as a value

{1: "one"}

[proven fog]

think he made a typo

but yeah int is the type of the key and str is the type of the value

[rustic gull]

yeah

[rustic gull]

yeah i did lol

[mortal fractal]

ah this one is actually really nice I'm glad I caught it https://github.com/microsoft/pyright/commit/101e717ab63cc2ebe0c00be952e4e6bb68960231 :^)

[tranquil turtle]

!e a.b: int =0

[rough sluice]

@tranquil turtle :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | NameError: name 'a' is not defined

[tranquil turtle]

Lol, why it is allowed?

Is it 3.11 feature?

I get syntax error on my machine

[hearty shell]

What is a 3.11 feature?

[buoyant swift]

the bot runs 3.10

[hearty shell]

Not sure what you mean by allowed here

[tranquil turtle]

a: int - normal annotation
a.b: int - annotation of attribute

I thought annotation of attribute isnt allowed...

[acoustic thicket]

🤔 it is

[tranquil turtle]

🤔 TIL

[acoustic thicket]

theres no way to annotate (a, b) = foo if thats what you're thinking of

[blazing nest]

How would I mark a TypedDict() final?

I can use @final for subclasses, but I don't get how to do it when I need to use TypedDict()

[tranquil turtle]

final(TypedDict()) ?

[tranquil turtle]

Yes, ty
I remember that there are no way to annotate something, but forgot what we cant annotate

[blazing nest]

hmmm

[trim tangle]

You mean, the class?

by final you mean - cannot create subclasses?

[blazing nest]

Yeah, Pyright seemed happy with what denball sent though so I simply used that.

Haven't really verified its behaviour though 🤔

[trim tangle]

@final
class Foo(TypedDict):
    x: int
    y: str

btw, mypy disallows putting @final on a TypedDict 🥴

[blazing nest]

Are you fucking kidding me..... BRUH

wait why 😭

[trim tangle]

I think it generally disallows combinations it doesn't understand

and it doesn't let you do any narrowing on typeddicts anyway

[blazing nest]

It just joins them into one with unions yeah?

[trim tangle]

hm?

[blazing nest]

Let me test it myself hold on, there's the playground 😉

[trim tangle]

I just noticed that I can actually buy a .ru domain for cheaper than .su 😩 what a waste

now I'm gonna waste $2.48 every year!

[blazing nest]

from typing import TypedDict, Union


class One(TypedDict):
    field: None


class Two(TypedDict):
    field: str


Three = Union[One, Two]


x: Three = ...  # type: ignore

reveal_type(x['field'])

main.py:17: note: Revealed type is "Union[None, builtins.str]"

[trim tangle]

that works with normal classes as well, I think

[blazing nest]

Yeah

[trim tangle]

yeah ```py
from typing import Union

class One:
field: None
class Two:
field: str
Three = Union[One, Two]

x: Three = ... # type: ignore

reveal_type(x.field)

main.py:11: note: Revealed type is "Union[None, builtins.str]"

[blazing nest]

This is annoying though ```
main.py:15: error: Type of TypedDict is ambiguous, could be any of ("One", "Two")

...it's almost as if that was the entire point

It should be able to union them into and be quiet about it ```python
class Three(TypedDict):
field: Optional[str]

[sweet whale]

How can I indicate that a variable is a one-letter string? If that's possible, I need to typehint that it's a letter basically

[oblique urchin]

there is no built-in type for that. you could perhaps use a NewType or phantom type

[gleaming karma]

hint: @young knot it's best to not spend too much time thinking about me when you should be focusing on other things

[hasty hull]

Is this a pyright bug? ```py
def foo() -> None:
raise RuntimeError('bar')

print('foo')  # no error is raised here

[hearty shell]

Type checkers dont consider exceptions

[hasty hull]

What about NoReturn?

[hearty shell]

That is an exception xD

[oblique urchin]

Eric explicitly rejected an error for unreachable code (https://github.com/microsoft/pyright/issues/2672)

[hasty hull]

Ah I see, thanks

[oblique urchin]

though it seems like pyright does emit a "hint" about unreachable code. does it not do that in your case?

[hasty hull]

No it doesn't

pylance does but the pyright CLI does not

[hearty shell]

I thought Pyright/mypy only gave the Never type on code under conditionals based on types

[tulip hearth]

is there a way to do something like this with a generic

T = TypeVar("T")
O = ParamSpec("O")

class Foo(Generic[T, O]):
  ... # implementation here

bar: Foo[str, [some, stuff, here] = "some random string"

where bar is interpreted as str and not Foo

[hearty shell]

!docs typing.Annotated

[rough sluice]

typing.Annotated

typing.Annotated```

[hearty shell]

Something like this?

[tulip hearth]

yes ik
but thats not as flexible as i want

yeah something like that

[little hare]

what are you trying to do exactly?

[tulip hearth]

basically a generic which takes in a subclass of a particular class and some metadata
so i want the type of the variable to be interpreted as that class and not the generic

basically typing.Annotated

[hearty shell]

What are you trying to do that Annotated does not provide?

[tulip hearth]

storing the metadata in a particular manner

[little hare]

sounds like attrs

[hearty shell]

In what particular matter?

[little hare]

like dataclass.field or attr.field

[tulip hearth]

as attributes rather than a tuple

[hearty shell]

Ah, not possible (within reason) as far as I am aware

I wanted that functionality once as well, sadly the pep for it was rejected

[tulip hearth]

sadge
Seems like typing.annotated it is

[hearty shell]

Well not exactly but I assume what you wanted could be fixed by having keyword arguments

!pep 637

[rough sluice]

**PEP 637 - Support for indexing with keyword arguments**

Link

Status

Rejected

Python-Version

3.10

Created

24-Aug-2020

Type

Standards Track

[acoustic thicket]

What's a phantom type?

(also, congrats on core-dev!)

[soft matrix]

a type thats erased at runtime

[acoustic thicket]

hm

i installed django-stubs
how do i get pyright to, like, use them

the documentation of django-stubs describes how you can get mypy working with it, but not pyright

[trim tangle]

Doesn't it rely on mypy plugins anyway?

[acoustic thicket]

i dont know

[hasty hull]

Yeah but shouldn't pyright pick up that the stub package is installed and resolve types from that instead?

I can't remember the what it uses to determine the stub package though so maybe django-stubs is incompatible?

[hasty hull]

Just installing it worked for me somehow

are you using the pyright CLI?

[acoustic thicket]

hmm im having trouble with models mainly

[acoustic thicket]

no

[hasty hull]

Ah I see, it is picking up the stubs, pylance comes bundled with Django stubs

The error is that a generic class hasn't been given generic arguments

I haven't used Django so can't comment on what it should be

[hasty hull]

I think for CharField it should be CharField[str, str](max_length=1000)

It is not possible for the maintainers of django-stubs to support default generic arguments (outside of the mypy plugin) because they can't modify the signature of the actual django objects but you can, this is definitely over-engineered but heres how you can use default generic arguments ```py
from typing import TypeVar, Type, Any, overload
from django.db import models

set value type

_ST = TypeVar('_ST', contravariant=True)

get return type

_GT = TypeVar('_GT', covariant=True)

class CharField(models.CharField[_ST, _GT]):
@overload
def init(self: 'CharField[str, str]', **kwargs: Any) -> None:
...

@overload
def __init__(
    self: 'CharField[_ST, _GT]',
    g_type: Type[_GT],
    s_type: Type[_ST],
    **kwargs: Any
) -> None:
    ...

def __init__(
    self, g_type: Type[Any], s_type: Type[Any], **kwargs: Any
) -> None:
    super().__init__(**kwargs)

v = models.CharFieldstr, str # explicit generic arguments
x = models.CharField(max_length=1000) # unknown generic arguments
u = CharField() # default generic arguments

The much simpler solution is to simply make an alias ```py
CharField = models.CharField[str, str]

[soft matrix]

im in the process of writing a pep for this gonna yonk this as an example

[acoustic thicket]

i tried doing models.CharField[Any, Any] first, but that raised an error at runtime saying type object isn't subscriptable

thats when i thought i must be doing something wrong and discovered the stubs

[hasty hull]

https://github.com/typeddjango/django-stubs#i-cannot-use-queryset-or-manager-with-type-annotations

[acoustic thicket]

hmmm

interesting

[hasty hull]

wait this doesn't actually work

y = CharField(g_type=int)
reveal_type(y)  # CharField[str, str]

[soft matrix]

ill make it work :P

[hasty hull]

I took it from this example https://github.com/python/mypy/issues/10207#issuecomment-798812236

It's the **kwargs

[hasty hull]

@soft matrix This works lol ```py
from typing import TypeVar, Type, Any, TypedDict, Optional, Union, overload
from typing_extensions import Unpack

from django.db import models

set value type

_ST = TypeVar('_ST', contravariant=True)

get return type

_GT = TypeVar('_GT', covariant=True)

class CharFieldArgs(TypedDict, total=False):
verbose_name: Optional[Union[str, bytes]]
name: Optional[str]
primary_key: bool
max_length: Optional[int]
unique: bool
blank: bool
null: bool
db_index: bool
default: Any
editable: bool
auto_created: bool
serialize: bool
unique_for_date: Optional[str]
unique_for_month: Optional[str]
unique_for_year: Optional[str]
help_text: str
db_column: Optional[str]
db_tablespace: Optional[str]
db_collation: Optional[str]

# omitted fields for easier POC
# validators: Iterable[_ValidatorCallable]
# error_messages: Optional[_ErrorMessagesToOverride]
# choices: Optional[_FieldChoices]

class CharField(models.CharField[_ST, _GT]):
@overload
def init(
self: 'CharField[_ST, _GT]',
*,
g_type: Type[_GT],
s_type: Type[_ST],
**kwargs: Unpack[CharFieldArgs],
) -> None:
...

@overload
def __init__(
    self: 'CharField[str, str]', **kwargs: Unpack[CharFieldArgs]
) -> None:
    ...

def __init__(
    self,
    g_type: Optional[Type[Any]] = None,
    s_type: Optional[Type[Any]] = None,
    **kwargs: Any,
) -> None:
    super().__init__(**kwargs)

v = models.CharFieldstr, str # explicit generic arguments
x = models.CharField(max_length=1000) # unknown generic arguments
u = CharField() # default generic arguments
y = CharField(g_type=int, s_type=str)
reveal_type(y) # CharField[str, int]

[soft matrix]

cool ill see how i can use this

[hasty hull]

You could of course just remove all the extra arguments to show the actual implementation details

[fierce ridge]

$ mypy models/topic.py | wc -l
     133

what's the point of even having annotations if nobody is checking that they are correct

this is a 65 line file

(admittedly a lot of these errors are in other imported files but still)

oooh mypy --install-types is really useful

[trim tangle]

that's one scary command

[fierce ridge]

it gives you a y/n prompt at each one

[trim tangle]

ah

[fierce ridge]

and it just installs the packages from typeshed

[trim tangle]

ah, just typeshed

[fierce ridge]

it's not random stuff from pypi as far as i can tell

yeah

hmm.... are annotations fully evaluated by mypy?

or is it some kind of partial subset of python?

i assume the latter

[oblique urchin]

mypy only accepts a restricted subset of expressions in annotations. it errors on whatever it doesn't recognize

the runtime accepts any expressions

[fierce ridge]

that's what i figured

i just encountered a circumstance where someone is dynamically generating a class and then trying to use it in an annotation by calling the generating function

[oblique urchin]

yeah mypy won't like that

[fierce ridge]

(what they are doing doesn't make any sense anyway, mind you)

fair enough

i forget: is there a way to ignore specific imported packages?

oh right, you put it in a section in the config

[oblique urchin]

it's a hardcoded list in the mypy source. when it was first added it included some packages that didn't actually exist. I put up dummy packages on pypi so they wouldn't get hijacked

[fierce ridge]

i assumed it was something like that

still useful

typeshed is pretty big

[rustic gull]

can you type hint in lambda or no? and if so whats the syntax?

[hasty hull]

Depends on the form you want, if you're assigning to a variable you can type hint it like so: ```py
from typing import Callable

f: Callable[[int], int] = lambda x: x * 2

[rustic gull]

ah ic thank you

[rustic gull]

var: list[str, str] = []

you can type hint as so to something thats mutable? so right now its saying if i append something it will have the content of a string and only a string? and is that how you type hint a list? and why 2 elements? does it mean it will have 2 different types or just like a dict which if you type hint like dict[str, int] which accepts a string as a key and a value is an int?

[oblique urchin]

that's invalid, list only takes a single type argument. var: list[str] = [] means a list of strings. there is no way to provide a type for a list of a specific length

and dict[str, int] means what you think it does

[rustic gull]

ah ic i was told its only in tuples which you type hint as tuple[int, ...] which means it can take a unknown value so can i do this with lists as well? or no

[oblique urchin]

tuples are special in the type system

[rustic gull]

why so?

[oblique urchin]

tuple[int] means a tuple of exactly one int, tuple[int, ...] means a tuple of any number of ints

because it's common to have heterogeneous tuples (with a fixed number of elements of specific types)

[rustic gull]

yep

[rustic gull]

alright and can it be done with lists or no?

[trim tangle]

no

[rustic gull]

ic

[trim tangle]

For example, the type annotation of asyncio.gather says that it returns a tuple, so that you can unpack it results.

But in reality it returns a list 🤷

why do you need a fixed-shape list though?

[buoyant swift]

huh why does it have to be typed as a tuple?

[rustic gull]

im just asking really because i just want to know haha

[trim tangle]

Because you can't type a heterogenous list

list[int, int, str] is not a thing

[buoyant swift]

ohhhhh, i see

[oblique urchin]

so we can give precise return types where gather(T1, T2) returns [T1, T2]

well that with some awaitables

[trim tangle]

on top of that hack, it also need a staircase of overloads to work properly 😩

[buoyant swift]

iw as just hthinking about the unpacking part

[oblique urchin]

we have had people complaining to typeshed about this, I guess they were relying on it actually being a list

[soft matrix]

Do you have any clue when Map[] could actually be added?

[oblique urchin]

when you write the PEP

more seriously, I don't know of any plans to add it, but if someone does spend the time to write up a proposal, it has a good chance of making it

I think the PEP 646 folks might want to do that as their next step, not sure though

[soft matrix]

Are Pradeep and Steve not working on it?

[oblique urchin]

PEP 646 itself was a bit of a drag

[soft matrix]

Yeah

[buoyant swift]

what would Map[] do?

[oblique urchin]

map() at the type level

[buoyant swift]

huh, what would adding that change?

[oblique urchin]

you'd be able to type gather as something like def gather(coros: Map[Awaitable, Ts]) -> Ts:

[buoyant swift]

ah

[mortal fractal]

:l heckin pylint bugs

src/pyffstream/cli.py:1626:20: C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary) >:l I'm iterating over the dict &'d with a set, you can't & without doing keys too!!

[oblique urchin]

my experience with pylint has been that it's slow and complains about a lot of things I don't care about

[mortal fractal]

heh

[hasty hull]

Yeah it's been the same for me, I just removed it eventually

[fierce ridge]

this is like an anti-recommendation

i almost always use .keys() even just to make the code easier to read

[buoyant swift]

interesting, i hate seeing .keys anywhere

[mortal fractal]

[tool.pylint.messages_control]
max-line-length = 88
disable = [
  "fixme",
  "missing-docstring",
  "no-else-return",
  "no-else-break",
  "no-else-continue",
  "no-else-raise",
  "too-many-locals",
  "too-many-branches",
  "invalid-name",
  "global-statement",
  "too-few-public-methods",
  "too-many-statements",
  "too-many-arguments",
  "too-many-lines",
  "too-many-instance-attributes",
  "too-many-nested-blocks",
  "unsubscriptable-object",  # buggy
]

[hasty hull]

My pylintrc file was nearly 100 lines long before I deleted it lol

[brisk heart]

Bro wtf why the hell would pylint care about the amount of public methods a class has

[trim tangle]

a class with 100 methods is probably not a good thing

[brisk heart]

What the hell are people supposed to do if they have a dataclass or whatever

Oh yeah too many methods is bad

But there's never too little

[trim tangle]

oh, too-few-public-methods

lol

[blazing nest]

lmao

[terse sky]

It's really weird to me that iteration over a dict in python is just the keys

I know the reason but still weird

It also means that a dict is a Collection[K]

Which is wildly unintuitive and not how anybody would typically speak about a dict in relation to the concept of a collection

[trim tangle]

what's the reason?

[terse sky]

Consistency with in, I believe

[trim tangle]

huh, interesting

[terse sky]

They want iteration and membership check to be consistent

[acoustic thicket]

iirc iteration over the dict is a lot faster than iterating over .keys()

[acoustic thicket]

that makes sense but i still dont like it :P

[terse sky]

It's a reason, but yeah, it's not great

I would want dict to model Collection[Entry[K, V]]

[trim tangle]

yeah, one place where it's really awkward is that a bunch of places accept Mapping[K, V] | Iterable[Union[K, V]]

[terse sky]

Gross

[trim tangle]

like dict's __init__

it is particularly awkward because Mapping is an Iterable

[oblique urchin]

it's worse than that, because it really wants Iterable[Iterable[K, V]], but we can't say "an iterable of a K and V" so we just use tuples

[trim tangle]

ah

[acoustic thicket]

hmm conceptually do Mappings really need to be Iterable

like, isnt a function technically a Mapping

[trim tangle]

not in Python

a function doesn't have keys(), values(), items()

[acoustic thicket]

yeah i dont mean in python

i mean, like, in math or whatever

[trim tangle]

!e

kvs = {("a", "b"): "c", ("b", "c"): "d", (1, "x"): "y"}
print(dict(kvs))
print(dict(iter(kvs)))

[rough sluice]

@trim tangle :white_check_mark: Your eval job has completed with return code 0.

001 | {('a', 'b'): 'c', ('b', 'c'): 'd', (1, 'x'): 'y'}
002 | {'a': 'b', 'b': 'c', 1: 'x'}

[trim tangle]

😎

[acoustic thicket]

gross

[terse sky]

I don't agree with the "worse than that" part

An iterable of K, V doesn't make sense really

[trim tangle]

yeah I think that's a separate issue

session types for iterators/generators would be kinda cool tho

[terse sky]

Personally I probably wouldn't bother with the union in the type signature

I would just annotate iterablr

And users would call .items on their dicts

[trim tangle]

Maybe dicts shouldn't've been iterable in the first place?

[terse sky]

That's silly, for looping over a dict is a common use case

[trim tangle]

that's true

[trim tangle]

oh I meant still having .keys(), .value(), .items() methods

but not having the actual mapping be iterable

right now you still have to do for k, v in the_dict.items()

[boreal ingot]

!zen 1

[rough sluice]

The Zen of Python (line 1):

Explicit is better than implicit.

[trim tangle]

xactly

[boreal ingot]

so I agree that we should need to explicitly say what part we are looping over

[terse sky]

It's a way to go but not a good one IMHO

The default should just be iterating over key-values

That's what you expect from iteration on a container, iteration over everything

And this is what most languages do

[boreal ingot]

fair

it just feels a bit random that it is only keys in the first place.
I get the thing with in, and that it makes sense for py for item in something: assert item in something to pass, but it still feels a bit arbitrary

[brisk heart]

this used to be very common

if "key" in mapping:
    return mapping["key"]

but these days people should be using walrus instead

if value := mapping.get("key"):
    return mapping["key"]

it's very rare to see anything differentiate between None and missing tbh

[oblique urchin]

no they shouldn't, that doesn't cover other falsy values

[brisk heart]

yeah then (value := mapping.get("key")) is not None

[boreal ingot]

I have a lot of cases in my code where I have py self.value = SomeClass(data["key"]) if data.get("key") is not None else None
I wonder if there is a way to make that nicer

[brisk heart]

yeah I do that a lot too

probably

self.value = None
if value := data.get("key"):
    self.value = SomeClass(value)

but of course this is a bit redundant if the entire thing can be a short one-liner

[boreal ingot]

but that makes it 3 lines, and I often have like 3 of these in a row.
and imo it looks nicer when the attributes can be more grouped up like py self.value1 = ... self.value2 = ... self.value3 = ...

[brisk heart]

meh, code repetition isn't the best either

at that point either process them all as a list or make a util function for this kinda stuff

[boreal ingot]

yhe I might make a util function at this point

[terse sky]

None aware operators :-)

Also in your particular case can't you just assign the result of get

[brisk heart]

actually wait I have a funky idea

[terse sky]

Oh I see

Nm

[brisk heart]

(value := data.get("key")) and SomeClass(value)

[terse sky]

So yeah, none aware operators

In Kotlin value = data["key"]?.let { SomeClass(it) }

[brisk heart]

yeah honestly that's just walrus + and in python

[boreal ingot]

that works, but it still feels a bit to #esoteric-python for my liking

[terse sky]

Yeah

[chilly kindle]

ya I'd say that's far too clever code for it's own good

[terse sky]

In python you're just going to use a branch or ternary

[chilly kindle]

I'd much rather have blah if dict.get(key) is not None else None

[terse sky]

And it starts to feel really clumsy after a while

[chilly kindle]

much easier to read

[terse sky]

When you work with a lot of potentially absent values

[brisk heart]

is it really esoteric if you use both concepts separately on a daily basis?

[terse sky]

But that's life

Using and in that way is esoteric in python IMHO

[acoustic thicket]

why .get(...) is not None over not in?

[brisk heart]

alright but like still

[acoustic thicket]

what if the key exists and is associated with None

[brisk heart]

the concepts separately are cool

[boreal ingot]

then I would want the result to be None

[acoustic thicket]

🤔

[boreal ingot]

basically the values of the dict are NotRequired[WhatIWant | None]

[brisk heart]

value = data.get("key")
return value and SomeClass(value)

return SomeClass(value) if (value := data.get("key"]) else None

[boreal ingot]

so for example {} and {"abc": None} should both give None, but {"abc": 123} should give SomeClass(123)

[chilly kindle]

i guess those feel pretty equivalent to me. The walrus thing just feels very hard to read to me

for the typical use case .get(key) is not None and key not in dict give an identical result but not always. So actually I'd say which one you want depends on your business logic, they're not really competing

[terse sky]

probably want to use key not in dict; use the least powerful function that does what you need

i think using the walrus on get is fine, that's basically one of the idiomatic purposes of walrus. It's the use of and in that way that I'm not a big fan of.

[trim tangle]

hot take: using walrus in a list comprehension is confusing

[terse sky]

it is a bit, but you get used to it

there's no question that it's a bad solution compared to what other languages offer

the question is how it compares to alternatives in python

if you want to map, then filter

[brisk heart]

I use it in my libraries >_<

[fierce ridge]

yikes, i don't like this at all

you can still write helper functions and use a list comp, if you don't want to map+filter

since the list comp could be significantly faster on very big sequences (less function call overhead)

[blazing nest]

I have the same issue and made a helper-method for it 😔

def _get_as_snowflake(data: Optional[Mapping[str, Any]], key: str) -> Optional[Snowflake]:
    """Get a key as a snowflake.

    Returns None if `data` is None or does not have the key.

    Parameters:
        data: The optional mapping to get the key from.
        key: The key to attempt to look up.

    Returns:
        The value of the key wrapped in a Snowflake, if there was a mapping
        passed and the key could be found.
    """
    if data is None:
        return None

    value: Union[str, int, None] = data.get(key)
    return Snowflake(int(value)) if value is not None else None

[boreal ingot]

nice, I will likely make something a bit more generic as I use this pattern with multiple different classes (so being able to pass in the class to the function will be nice)

[little hare]

pylance why

it both thinks its a bool and not a bool

[oblique urchin]

maybe you have it running in <3.7 mode?

[terse sky]

how can you do it with helper functions?

x = [y for e in elements if pred((y:=transform(e)))]

how can you do this a) without walrus, b) with a single list comp, c) without calling the function twice per element ?

[boreal ingot]

x = [y for y in map(transform, elements) if pred(y)]```

[trim tangle]

what if transform is not just a function call?

like y := foo.bar.get(e + "!")

[boreal ingot]

L A M B D A

😛

[trim tangle]

🥴

[boreal ingot]

or using some hacky objects py _L[ _C[foo].bar.get(_0 + "!") ]

straight out of my #esoteric-python quest ```py
In [10]: class Foo:
...: def init(self):
...: self.bar = {1: "abc"}

In [11]: foo = Foo()

In [12]: transform = _L[_C[foo].bar.get(_0 + 1)]

In [13]: transform(0)
Out[13]: 'abc'```

(linked the wrong channel, dam you similar names and not being bothered to look closely)

[void panther]

[y for e in elements for y in [transform(e)] if pred(y)]

[boreal ingot]

or that

[fierce ridge]

maybe like apply_if(bool, my_func, x)

[fierce ridge]

ah, i see. this is when i just resort to writing a loop

in haskell you'd probably just write a custom recursive function, no? or maybe something with foldl and <+>

this is how i'd do it in python (or maybe use a generator function + list()):

def iter_stuff(xs):
    result = []
    for x in xs:
        y = transform(x)
        if pred(y):
            result.append(y)
    return result

and here's my naive equivalent in haskell, although you might want instead to use Maybe and Alternative

iterStuff acc [] = acc
iterStuff acc (x : xs) =
  let y = transform x
  in if pred y then y : acc else acc

[trim tangle]

Data.Maybe.mapMaybe :: (a -> Maybe b) -> [a] -> [b] 

[fierce ridge]

i can never remember how to do anything in haskell

too much abstraction

im too stupid for it

there we go

[trim tangle]

idk, mapMaybe seems like a completely normal function to me 🤷

[fierce ridge]

https://hackage.haskell.org/package/base-4.16.0.0/docs/src/Data.Maybe.html#mapMaybe

yeah that's easier

i was envisioning something a lot more abstract

mapMaybe is how i tend to write functional programming stuff, e.g. in ocaml

i don't have any room in my brain for advanced algebraic abstractions

[trim tangle]

filter_map in Rust https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.filter_map

[fierce ridge]

i really like ocaml for this reason, ocaml code tends to be a lot simpler to read imo

seems "cultural" more than anything

[trim tangle]

I have a much harder time reading Rust than Haskell tbh...

[fierce ridge]

i dont really know rust at all but it looks very symbol-heavy

[trim tangle]

as much as I like it, the signal-to-noise ratio is very low

[fierce ridge]

i can usually muddle through small rust snippets

lots of casting and memory safety stuff

seems better than C at any rate

[terse sky]

@fierce ridge yeah, but having to write a loop sucks because you have to create this list and mutate it, it's lame

and now it's three lines and it can't be used as an expression when it should be very short and simple

[fierce ridge]

that's why i like generator functions

is it short and simple though? even in haskell you'd either write a helper function like i wrote, or you have to use an existing one that just happens to be in the stdlib, or use some higher-order abstractions like Alternative

[terse sky]

I think generator functions are nice but in this particular case the generator function is a hack

well, haskell while it's sophisticated in many ways still looks pretty painful compared to the modern approach

val y = elements.map { transform(it) }.filter { pred(it) }

[fierce ridge]

i think what you want is something like julia where more things are "expressions" rather than "statements", with some semantics around what gets returned from those kinds of things

[terse sky]

I'd say this is pretty short and simple and you don't need any helpers

nah nothing to do with julia really

[fierce ridge]

then what's wrong with filter() and map()?

[terse sky]

the code I wrote above is kotlin but Rust and Swift are both going to look pretty similar

[fierce ridge]

this just comes back to lambda syntax & function call syntax with .

[terse sky]

it looks awful

no, it's not just lambda syntax

it's also the left-right chaining of it

[fierce ridge]

then use a language with UFCS 🙂

[terse sky]

there's literally only two such languages, and since nobody else uses them, why would I 🙂

[fierce ridge]

nim isn't literally 0 but

[terse sky]

UFCS isn't the only way to support left to right chaining

or even the most popular

or even in the top 2 🙂

[fierce ridge]

sure, i am not opposed to having map/filter methods either

its one of the good things about js for example

but it just... isnt how python is

🤷‍♂️

[terse sky]

you don't want them as members (assuming I understood "methods" right)

you want them as extensions

either that, or you have some kind of left to right composition pipeline operator, like F#

if you have them as members you'll be reimplementing map and filter for every single data type and that's just silly

(or trying to reuse implementations via inheritance, yikes!)

but yeah, python just completely missed this boat

btw though left to right is also one of people's main objections to walrus

people read the y, which only gets assigned at the end

[fierce ridge]

idk what you mean by "extensions"

[oblique urchin]

in some languages "extensions" are a mechanism to add methods to an existing type I believe

[terse sky]

functions called with member function syntax, but are not members

that is, they don't have to be defined intrusively on the class and they don't have access to private data, and they also can't be dynamically dispatched on

So, here's the definition of map in kotlin

(the signature)

inline fun <T, R> Iterable<T>.map(
    transform: (T) -> R
): List<R>

That means that every single type that implements Iterable (which is pretty much exactly what you'd expect coming from python) gets to call .map

filter is defined similarly

So, lets say you decide to implement your own type, Classroom, and Classroom is iterable (you iterate over students), then you get to use map, filter, and probably dozens and dozens of other standard library functions for free

very much like itertools but with syntax that doesn't make your eyeballs bleed if you do more than one thing on the same line

[trim tangle]

what about piping?

foo | Filter(lambda x: x % 2 == 0) | Map(abs)

[terse sky]

yeah, so "pipe" operators are another way to solve this problem

it's just kind of gross when it's not built into the language, IMHO

also runs into issues of dispatch and conflict a lot more easily

because things aren't constrained in python

[fierce ridge]

i just... don't see why this is a problem

[terse sky]

if foo defines its own overload for pipe that's what you'r egetting, it will take precedence over Filter

[fierce ridge]

believe me, i think this stuff is really nice and ergonomic

but i don't think its a problem thatp ython lacks it

[terse sky]

it just depends how you define problem

it doesn't stop you from getting stuff done in python

it just makes a lot of code in python more annoying to write than it would be otherwise

all other things considered, a language is better if it can do this nicely. And isn't language design ultimately about such decisions?

[fierce ridge]

🤷‍♂️ i guess

tradeoffs

and hindsight is 20/20

[terse sky]

trade-off implies that you're getting some benefit in exchange for it though

which I don't see

but I agree, hindsight

[fierce ridge]

the whitespace syntax

that's the tradeoff

it's python, take it or leave it imo

[terse sky]

right, and before auto formatters were good, whitespace syntax meant that the way the code read matched the way the code behaved

so it was a trade-off then, now I don't think it is. but again, it's hindsight, in 1990 I don't think it was anticipated that auto formatters would be where they are now

clang-format started a bit of a revolution in that regard