type-hinting

[hearty shell]

What actually is happening is that Item_T is resolved to DiscordGuildRelation, and Item_T is just the new name for Entity_T

That is why you are getting all those warnings

[noble pilot]

ah, I think I fixed it by doing:
ManyToOneCache(PgListenerCache[Dict[int, List[Item_T]], Entity_T, Item_T], Generic[Entity_T, Item_T], ABC):

God that is ugly heh

[hearty shell]

Yeah gotta agree xD

This is a minimal example of that btw

T = TypeVar("T")
K = TypeVar("K")
S = TypeVar("S")


class A(Generic[T, K, S]):
    t: T
    k: K
    s: S

class B(A[tuple[S, K], K, S]): ...

class C(B[int, float]): ...

reveal_type(C().t)
reveal_type(C().k)
reveal_type(C().s)

main.py:26: note: Revealed type is "Tuple[builtins.int*, builtins.float*]"
main.py:27: note: Revealed type is "builtins.float*"
main.py:28: note: Revealed type is "builtins.int*"

[noble pilot]

mypy will find it but not pycharm. boo

https://mypy-play.net/?mypy=latest&python=3.10&gist=c2df7ba617d772c5884ae8e495820b8a

[devout barn]

I added some comment to depict what on god's green earth is happening here

    @staticmethod
    def _event_lock(
        wait_for_previous: bool = True, set_on_fail: bool = True
    ) -> Callable[P, Callable[P, Awaitable]]:
        # ^^ This is wrong, linters accept this
        # Firsly we return decorator function, ie the outer callable
        # Callable[]
        # This takes anymore function (inner) ie
        # Callable[Callable]
        # This function takes any Params (Using ParamSpec here P)
        # And returns something (held to typeVar R)
        # Callable[Callable[P, R]]
        # This returns a function which takes same param and returns a coroutine
        # Callable[Callable[P, R], Callable[P, Awaitable[R]]]
        # Linters hate this.
        def decorator(func):
            async def wrapper(self: PrefixHelper, *args, **kwargs):
                if not self.__wait.is_set() and wait_for_previous:
                    await self.__wait.wait()
                self.__wait.clear()
                try:
                    r = await func(self, *args, **kwargs)
                    self.__wait.set()
                    return r
                except Exception as e:
                    if set_on_fail:
                        self.__wait.set()
                    raise e

            return wrapper

        return decorator

How do I correctly annotate this? Pyright accepts the current annotation despite it being incorrect.

btw, __wait is an instance of asyncio.Event if it matters.

[fierce ridge]

what is the type of func? the problem is probably that decorator itself isn't annotated

yet again, a use case for being able to derive a ParamSpec from an existing function...

[devout barn]

func is just a very bland Callable[..., Any]

Although we might have to consider using paramspec in place for ellipsis when specifying return type too, I think

Kind of new to typing

[fierce ridge]

decorator does not change the type of the decorated function, right?

i think you would have to start by annotating the wrapper itself like this

R = TypeVar('R')  # `func` return type
P = ParamSpec('P')  # `func` params

def decorator(func: Callable[P, R]) -> Callable[P, R]:
    async def wrapper(self: PrefixHelper, *args: P.args, **kwargs: P.kwargs) -> R:
        ...
    return wrapper

[devout barn]

it returns a callable which returns a coroutine. So it does make a difference

[fierce ridge]

ahh ok

so it takes a plain function and returns a coroutine function

or no

they are both coroutine functions

easy enough, you have to wrap the result in Awaitable i believe

typing with coroutines is already a mess, hang on

let me get that right first

[devout barn]

they are both coroutines

[fierce ridge]

coroutines, or coroutine functions?

wrapper is a coroutine function

[devout barn]

coroutine functions

[fierce ridge]

i.e. it is a function that returns a coroutine

ok. let me see if i can get this right

[devout barn]

yes, coroutine functions

thanks

[devout barn]

Mhm, Callable[Callable[P, R], Callable[P, Awaitable[R]]]
that is what I came up with

[fierce ridge]

hmm, well pyright seems confused

that or there's a variance problem

i expected this to work:

import asyncio
from collections.abc import Awaitable, Callable
from typing import ParamSpec, TypeVar

class PrefixHelper:
    __wait: asyncio.Event


R = TypeVar('R')  # `func` return type
P = ParamSpec('P')  # `func` params

def decorator(func: Callable[P, Awaitable[R]]) -> Callable[P, Awaitable[R]]:
    async def wrapper(
        self: PrefixHelper,
        *args: P.args,
        **kwargs: P.kwargs
    ) -> R:
        ...
    return wrapper

but pyright didn't like the return of wrapper

Expression of type "(self: PrefixHelper, *args: P.args, **kwargs: P.kwargs) -> Coroutine[Any, Any, R@decorator]" cannot be assigned to return type "(**P@decorator) -> Awaitable[R@decorator]"
  Type "(self: PrefixHelper, *args: P.args, **kwargs: P.kwargs) -> Coroutine[Any, Any, R@decorator]" cannot be assigned to type "(**P@decorator) -> Awaitable[R@decorator]"

at least, that's using @trim tangle 's pyright playground 🙂 maybe it's been updated and this is a bug that was fixed

it's possible that the problem is unifying self: PrefixHelper, *args: P with *args: P

let me fix that first

gah, and of course you can't mix ParamSpec with actual args

that's super annoying

[devout barn]

I didn't quite catch that, but I am closely listening

[fierce ridge]

i think that's the problem here, and might not be fixable given the limited semantics of ParamSpec

fortunately in this case you just pass *args and **kwargs to the wrapped function, so you can annotate them with object and give up 😛

for example this "workaround" doesn't fix the problem, because it thinks the type of *args is object:

import asyncio
from collections.abc import Awaitable, Callable
from typing import ParamSpec, TypeVar

class PrefixHelper:
    __wait: asyncio.Event


R = TypeVar('R')  # `func` return type
P = ParamSpec('P')  # `func` params

def decorator(func: Callable[P, Awaitable[R]]) -> Callable[P, Awaitable[R]]:
    async def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
        self: PrefixHelper = args[0]
        ...
    return wrapper

Expression of type "object" cannot be assigned to declared type "PrefixHelper"
  "object" is incompatible with "PrefixHelper"

[devout barn]

Actually I think that wouldn't matter because the func annotation will be the same, but the Awaitable[R] will have one more abstraction where the awaited content needs to be awaited.

[fierce ridge]

this is what frustrates me about python typing... we get obscure and mildly useless stuff like LiteralString but the Callable/ParamSpec stuff is still a giant mess

[devout barn]

I very much agree

[fierce ridge]

where is facebook, google, etc. putting a couple devs on this problem full time?

[devout barn]

this is pain

[fierce ridge]

they probably have 100s of millions of lines of python code between the two of them

[devout barn]

I still don't understand how pyright thinks Callable[P, Callable[P, Awaitable]] is correct.

lol

[fierce ridge]

it's correct

the type of an async def function is Callable[P, Awaitable[R]]

[devout barn]

That part is correct
P here is the callable, the async function doesn't take P (the callable) as the argument

[fierce ridge]

async def foo(x: X, y: Y) -> Z: ...

def deco(func: Callable[[X, Y], Awaitable[Z]]): ...

deco(foo)  # valid

ah

i see

yeah i am not sure why it thinks P is correct for the outermost function

unless it's unifying wait_for_previous: bool = True, set_on_fail: bool = True with P

so the type checker might believe that P ≡ (wait_for_previous: bool = True, set_on_fail: bool = True)

[devout barn]

Copilot knows

[trim tangle]

Yeah, I update it manually from time to time

I want to add some config panel but I always end up doing something else (like playing factorio for 8 hours). But anyone is free to send a PR 🙂

[upbeat wadi]

import asyncio
from collections.abc import Awaitable, Callable
from typing import TypeVar, ParamSpec, Concatenate

class PrefixHelper:
    __wait: asyncio.Event

R = TypeVar('R')  # `func` return type
P = ParamSpec('P')  # `func` params
PHT = TypeVar('PHT', bound=PrefixHelper)

def decorator(func: Callable[Concatenate[PHT, P], Awaitable[R]]) -> Callable[Concatenate[PHT, P], Awaitable[R]]:
    async def wrapper(
        self: PrefixHelper,
        *args: P.args,
        **kwargs: P.kwargs
    ) -> R:
        ...
    return wrapper

@decorator
async def foo(self: PrefixHelper, x: int) -> str:
    ...

reveal_type(foo)  # Type of "foo" is "(PrefixHelper, x: int) -> Awaitable[str]"
```this works

@devout barn ^

[devout barn]

Yes this worked 😄
but annotating the outer function return type with the same thing (except enclosed by Callable[foo, bar] does not seem to typecheck)

    @staticmethod
    def _event_lock(
        wait_for_previous: bool = True, set_on_fail: bool = True
    ) -> Callable[
        [Callable[Concatenate[CPT, P], Awaitable[R]]],
        Callable[Concatenate[CPT, P], Awaitable[R]],
    ]:
        def decorator(
            func: Callable[Concatenate[CPT, P], Awaitable[R]]
        ) -> Callable[Concatenate[CPT, P], Awaitable[R]]:
            async def wrapper(self: CachingPod, *args: P.args, **kwargs: P.kwargs) -> R:
                if (
                    not self.__wait.is_set()
                    and wait_for_previous
                    and self.__has_started
                ):
                    await self.__wait.wait()
                self.__wait.clear()
                self.__has_started = True
                try:
                    r = await func(self, *args, **kwargs)  # Problem over here
                    # Argument of type "CachingPod" cannot be assigned to parameter of type "CPT@_event_lock"
                    # Type "CachingPod" cannot be assigned to type "CPT@_event_lock"
                    self.__wait.set()
                    return r
                except Exception as e:
                    if set_on_fail:
                        self.__wait.set()
                    raise e

            return wrapper

        return decorator

A mild change, PHT TypeVar changed to CPT
PrefixHelper was renamed to CachingPod
(everything was renamed appropriately)

the decorator function does typecheck

the return type of _event_lock causes problems

[upbeat wadi]

self: CPT should fix it

[devout barn]

thanks for the solution, it works 👍

[grave fjord]

anyone know why this doesn't typecheck? https://mypy-play.net/?mypy=latest&python=3.10&gist=5c0bbe4a641bf48239cf11c507eb53a0&flags=strict

[oblique urchin]

hm I guess because KeyboardInterrupt isn't compatible with other kwargs it accepts. I think this should be accepted though

[fierce ridge]

ah, Concatenate. i forgot about that one

i wish it was + or & instead...

[grave fjord]

also the type seems totally wrong

https://mypy-play.net/?mypy=latest&python=3.10&gist=22d8ee9b6c8a0599ad0fb9c59edd8a47&flags=strict this is closer - still doesn't work

[trim tangle]

Is typing in tests really that important?

[grave fjord]

yes!

[trim tangle]

You can just run the test to see if it works, right?

I guess it helps a bit with autocompletion

[grave fjord]

type checking tests found a load of issues with the public type interface of anyio for example

[fierce ridge]

imo yes

[hearty shell]

Everything must have types, types everywhere

[trim tangle]

ah

[fierce ridge]

types are tests

the type checker is the test runner

why not test your tests?

[grave fjord]

Only unsafe is kw only - and it's not in the stub

[oblique urchin]

the other args could also be passed as keywords though

[hearty shell]

Wasnt there a syntax involving the star operator to make concatenation shorter?

I think I might be confusing it with tuple vars though

[fierce ridge]

thatd be nice too

in fact thats a great idea

[oblique urchin]

PEP 677 proposed that but was rejected

[hearty shell]

Ah right

[fierce ridge]

Callable[[Foo, Bar, *WrappedParams.args, *WrappedParams.kwargs], [ReturnType]]

[oblique urchin]

I guess we could independently make Callable[[int, *P], T] work though

like that ^

[hearty shell]

Yeah that would be great, could tuple vars be confused with that, you couldn't use them here right?

[oblique urchin]

you can actually: https://peps.python.org/pep-0646/#type-variable-tuples-with-callable

so yes, that could be confusing

[grave fjord]

Right but the typeshed says they are kw only

[fierce ridge]

two questions:

1. how do i correctly annotate ModelReference.from_instance? i know that PEP 673 Self will fix this, but that's not released yet (right?)
2. is it possible to type-hint ModelReference such that the type of its attribute id is the same as the type of its parameter's attribute Model.id?

from typing import Any, ClassVar, Generic, TypeVar

_Model = TypeVar('_Model', bound='Model')
_ModelReference = TypeVar('_ModelReference', bound='ModelReference[Any]')

class Model:
    collection: ClassVar[str]
    id: Any  # to be specified by subclasses

class ModelReference(Generic[_Model]):
    model_cls: type[_Model]
    id: Any

    def __init__(self, model_cls: type[_Model], id: Any):
        self.model_cls = model_cls
        self.id = id

    @classmethod
    def from_instance(cls: _ModelReference, model: _Model) -> _ModelReference:
        return cls(type(model), model.id)

if i do this

class Frob(Model):
    collection = 'frobs'
    id: str

i want to be able to write

frob1 = Frob('zxcv')
frob_ref = ModelReference.from_instance(frob1)

and the type checker should know that frob_ref.id: str

[oblique urchin]

1. You need cls: type[_ModelReference]

Also, PEP 673 is available in typing_extensions. mypy doesn't support it yet though

[fierce ridge]

ah silly. let me see if type[] fixes it

ok that fixed the first problem. silly mistake, ty

second one remains unsolved 😛

[oblique urchin]

2. I don't think there's a simple way, you probably need another type parameter

[fierce ridge]

from typing import Any, ClassVar, Generic, TypeVar

_Id = TypeVar('_Id')
_Model = TypeVar('_Model', bound='Model[Any]')
_ModelReference = TypeVar('_ModelReference', bound='ModelReference[Any]')

class Model(Generic[_Id]):
    collection: ClassVar[str]
    id: _Id

class ModelReference(Generic[_Model[_Id]]):
    model_cls: type[_Model]
    id: _Id

    def __init__(self, model_cls: type[_Model], id: _Id):
        self.model_cls = model_cls
        self.id = id

    @classmethod
    def from_instance(cls: type[_ModelReference], model: _Model) -> _ModelReference:
        return cls(type(model), model.id)

###### 

class Frob(Model):
    collection = 'frobs'
    id: str

    def __init__(self, id: str):
        self.id = id

frob1 = Frob(id='asdf')
frob_ref = ModelReference.from_instance(frob1)

reveal_type(frob_ref.id)

main.py:13: error: Type variable "__main__._Id" is unbound
main.py:13: note: (Hint: Use "Generic[_Id]" or "Protocol[_Id]" base class to bind "_Id" inside a class)
main.py:13: note: (Hint: Use "_Id" in function signature to bind "_Id" inside a function)
main.py:35: note: Revealed type is "_Id?"
Found 1 error in 1 file (checked 1 source file)

close, but it's still confused (and also makes the code horrifically ugly & annoying to use)

i'm not really sure how typevars and generics are supposed to interact. would appreciate any good articles or guides on the subject

[oblique urchin]

A TypeVar can't itself be generic, so _Model[_Id] doesn't make sense

[fierce ridge]

right, mypy informed me as much 🙂

[hearty shell]

HKT when

[oblique urchin]

when you write a PEP

[hearty shell]

I am joking, I saw the github issue, seems it would be immensely difficult to implement :{

[fierce ridge]

for what it's worth, idk how you'd even do this in idris or haskell, without explicitly defining an interface/typeclass

ocaml maybe is easier because things are structurally typed

so is there no way to do this at all?

i can do class ModelReference(Generic[_Model, _Id]): but then i don't know how you'd assert that type(_Model.id) === _Id

this works of course, but then you have to manually annotate frob_ref which is also very icky

from typing import Any, ClassVar, Generic, TypeVar

_Id = TypeVar('_Id')
_Model = TypeVar('_Model', bound='Model[Any]')
_ModelReference = TypeVar('_ModelReference', bound='ModelReference[Any, Any]')

class Model(Generic[_Id]):
    collection: ClassVar[str]
    id: _Id

class ModelReference(Generic[_Model, _Id]):
    model_cls: type[_Model]
    id: _Id

    def __init__(self, model_cls: type[_Model], id: _Id):
        self.model_cls = model_cls
        self.id = id

    @classmethod
    def from_instance(cls: type[_ModelReference], model: _Model) -> _ModelReference:
        return cls(type(model), model.id)

###### 

class Frob(Model):
    collection = 'frobs'
    id: str

    def __init__(self, id: str):
        self.id = id

frob1 = Frob(id='asdf')
frob_ref: ModelReference[Frob, str] = ModelReference.from_instance(frob1)

reveal_type(frob_ref.id)

it'd be great if i could write this

_Id = TypeVar('_Id')
_Model = TypeVar('_Model', bound='Model[_Id]')

i see, this actually would be considered HKT https://github.com/python/typing/issues/548

too bad

[oblique urchin]

right, TypeVar bounds can't themselves be generic at the moment

[terse sky]

maybe I misunderstood the above

typevar bounds being generic isn't HKT though?

actually, I probably am misunderstanding

[soft matrix]

I'm honestly considering doing this next after type var defaults

Or atleast trying to

Or maybe intersection

Cause I don't know why that stalled

[oblique urchin]

there was someone saying they were preparing a PEP on intersection, right?

[soft matrix]

Yeah I remember it on typing sig I thought they were aiming for before the 3.11 feature freeze

[oblique urchin]

too late for that probably

[trim tangle]

waterfall 😔

rust be like: "hey you remember this feature you suggested yesterday after 5 glasses of wine? it's in nightly!"

[oblique urchin]

Honestly we can mostly achieve that with typing-extensions

I feel like for non-syntax typing features we really don't need to worry too much about Python versions

some people seem hesitant to use typing-extensions though

[soft matrix]

Yeah I don't really get why people don't like to use it

Also I don't think there'd be much point implementing an Intersection SpecialForm considering Union is deprecated

[oblique urchin]

It would still allow you to use intersection-the-concept before (at the earliest) 3.12. We're even adding typing.Unpack now even though it's technically unnecessary on 3.11

[soft matrix]

Oh well fair enough

[hearty shell]

You can always just use the hacks from the likes of dry-returns :P

[soft matrix]

But with future annotations enabled you could still do Foo & Bar at runtime

[oblique urchin]

until you try to do it in a TypeVar bound or generic base or type alias 🙂

[brisk hedge]

(and it'll stay in nightly for as long as you remember)

[quaint dust]

I think I understand what this error is saying to a point, but I'm not sure how to fix it? I was assuming that tuple[BaseException, str] would allow for any exception?

[oblique urchin]

I think you're missing a type[]

[quaint dust]

Ok then no nvm I have no idea what its saying 😄

[oblique urchin]

HTTPError (the class) is compatible with type[BaseException], not with BaseException

[quaint dust]

what does type do in type hints?

[hearty shell]

def foo(a: type[float]): ...
foo(float)  # OK
foo(int)  # OK
foo(bool)  # OK
foo(str)  # Error
foo(3.14)  # Error

def bar(a: float): ...
bar(3.14)  # Ok
bar(3)  # Ok
bar(float)  # error

Basically in the first one the arguments that are allowed are the ones that pass the issubclass(float), and the other one are isinstance(float)

I mean, terrible example because here that would not be the case at runtime

because ints are only "subclasses" of floats in the eyes of mypy

But in general this is the difference

[quaint dust]

hm good to know, that probably wouldve helped to know a while ago

Im still getting a similar error tho

"""
Argument of type 'dict[int, tuple[Type[HTTPError], Literal['[HTTP ERROR] : User not found']]]" cannot be assigned to parameter 'catch_errors' of type 'dict[int, tuple[Type[BaseException], str]] | None' in function 'make_api_request'
"""```

[hearty shell]

Do you modify this dictionary in any way inside the context you defined the type hint?

or do you just access its contents

[quaint dust]

Nope

oh I see

[hearty shell]

Ah, then try from collections.abc import Mapping, and use that instead of dict

[quaint dust]

probably could just turn the whole thing into a tuple honestly Im iterating over it anyways

[hearty shell]

The problem here is that dict[int, tuple[Type[HTTPError], Literal[...] ]] is not a subtype dict[int, tuple[Type[BaseException], Literal[...] ]] because dict is invariant, just changing to a Mapping should fix the issue

[quaint dust]

gotcha thanks that worked

[spark sandal]

Hello!
So I'm using generics in my code, and I found a usecase where I need to enforce stricter generics testing, say I have some code like

import random
from typing import TypeVar

_T = TypeVar("_T")

def sample(a: _T, b: _T) -> _T:
    return a if random.randint(0, 1) else b

c = sample(1, "hello")

here pylance from VS code assumes c: int | str. While this makes sense, but I want this sort of thing to error. Is this intentional behaviour? If so, is there a way to modify this and make it error?
If this is not intentional, can this be a bug in one of the typing tools?
Quick skim through python docs does not give anything about this
(please @ me if anyone replies, and thanks in advance!)

[hearty shell]

Not sure what you want to error here, do you want to constrain T to be of a single type?

[spark sandal]

yup

[hearty shell]

_T = TypeVar("_T", int, str)

[spark sandal]

well I don't want to constrain it to particular types, more like constrain it within the function to be homogenous

sample should be able to take any two types which are not bound to particular types, but both args must be the same type

The typechecker must allow sample(1, 2), sample("hi", "hello") or sample(AnyClass(args1), AnyClass(args2))

[hearty shell]

Ah alright, there is certainly a way I think, I don't know off the top of my head, I will see if I find something and ping you, otherwise I think you can do that using Variadicts but that is not what you want

[spark sandal]

thanks 👍

yeah my actual usecase is more complex so var-args/var-kwargs don't help me

[hearty shell]

@spark sandal Well, sort of a solution

https://github.com/python/mypy/issues/6559#issuecomment-864492740

But like not really

[hearty shell]

I didnt realise that the semantics of that get quite murky

[acoustic thicket]

funny how mypy's repo has 2.1k open issues and pyright has 11

granted mypy has more users

[hearty shell]

Oh wow, that is actually suspiciously low xD

If this is anything to go by, it seems that number is always low, I thought at one moment that pyright hard around 500 issues but apparently not

[boreal ingot]

also in my experience pyright is fast at addressing/fixing issues, never reported a bug to mypy so idk how fast they handle bugs.

[acoustic thicket]

yeah eric is great at what he does as far as ive seen

[spark sandal]

ah that can work, but it's not the most optimal of solutions. It would be handy to have some kind of new decorator API for this, like @typing.strict_generics for functions that have this usecase, that does nothing at runtime ofc

[grave fjord]

it fails with any **kwargs I've made a ticket here though: https://github.com/python/typeshed/issues/7542

[dim flume]

How many runtime type checkers are there other than typeguard, beartype, pydantic and pytypes

Oh, beartype mentions some

https://github.com/seandstewart/typical this is another one they mention

[brazen jolt]

Is this an issue with pyright or is there some actual reason why annotating method parameter with socket.socket when there's "socket" in __slots__ fail like this:

I can't see any reason why this would make any sense, just because I'm internally setting socket shouldn't override the global socket when annotating, and certainly not only when it's present in __slots__

[acoustic thicket]

hmm cannot reproduce on pyright playground

[soft matrix]

Have you tried updating?

[steep nest]

i think so

[trim tangle]

Pyright playground is usually a few versions behind

You can see the version if you go to devtools and inspect the requests

[acoustic thicket]

😔

[cosmic tree]

I'm having a bit of an issue making a TypeGuard example from PEP647 work with mypy. Here is my snippet ran with mypy v0.942 on Python 3.8:

from typing import TypeVar, Set, Any, Type
from typing_extensions import TypeGuard

_T = TypeVar("_T")

def is_set_of(val: Set[Any], type: Type[_T]) -> TypeGuard[Set[_T]]:
    return all(isinstance(x, type) for x in val)

def foo() -> Set[Any]:
    ...

some_set = foo()
if is_set_of(some_set, str):
    reveal_type(some_set)  # Revealed type is "builtins.set[_T`-1]"

I would have expected Set[str] as a revealed type, as it is the case with pyright

• am I doing something wrong?
• is this a well known issue in mypy ?
• does the _T`-1 thingi have a specific meaning?

[oblique urchin]

should be a mypy bug. you can check https://github.com/python/mypy/labels/topic-typeguard to see if it's known

the `-1 stuff is just to make TypeVars unique. it indicates that mypy didn't resolve the TypeVar correctly here

[cosmic tree]

Oh okay thanks, I'll pass through the labeled issues and file a new one if I can't find it

[oblique urchin]

I think it's https://github.com/python/mypy/issues/11428 though your example is simpler than the OP there

[cosmic tree]

Oh Indeed, thanks again. I'll put my example as a comment in this case

[indigo locust]

Is there a plugin for improved automatic (as-you-type) type checking for pycharm?

I've tried the official mypy plugin, but either it doesn't do what I'm wanting, or, I'm using it wrong

[acoustic thicket]

tbh ive had a much better time with pyright/pylance in vsc for typechecking
i've used the mypy plugin for pycharm too, but it felt pretty clunky

[trim tangle]

Is there any way of making a TypeGuard method?

T = TypeVar("T", covariant=True)
E = TypeVar("E", covariant=True)

class Ok(Protocol[T]):
    def value(self) -> T: ...

class Err(Protocol[E]):
    def error(self) -> E: ...

class Result(Protocol[T, E]):
    def is_ok(self: object) -> TypeGuard[Ok[T]]: ...
    def is_err(self: object) -> TypeGuard[Err[E]]: ...


def f(r: Result[int, str]):
    if r.is_err():
        "I want `r` to be `Err[str]` here"

[blazing nest]

What if you define it outside of the class, then assign it into the class as a class variable? Does it become a class variable then?

!e ```python
def test(obj):
print(obj)

class Test:
method = test

Test().method()

[rough sluice]

@blazing nest :white_check_mark: Your eval job has completed with return code 0.

<__main__.Test object at 0x7f8cb4ab3c10>

[trim tangle]

nope, doesn't work

(at least with pyright)

[trim tangle]

😔

anyway, this will not work quite as expected anyway, because the else branch of is_ok doesn't imply is_err, and vice versa

[acoustic thicket]

funky

ig a staticmethod isnt the worst solution here

[trim tangle]

I would just use a standalone function

[acoustic thicket]

mm

[trim tangle]

I don't think there's a way to make this work safely with patma either

[hearty shell]

How is that unusual use case?

There are so many libraries with .is_... methods

[hearty shell]

Also, dry-returns implements that and IIRC that part does not require a mypy plugin

[trim tangle]

"that" what?

[hearty shell]

Well, and "Option type" kind of like that

But I was just looking at the source and it seems it inherits from stuff that is definitely mypy plugin based so idk

https://github.com/dry-python/returns/blob/master/returns/result.py

[trim tangle]

Yeah, it even supports patma. But last time I checked, patma with pyright didn't even quite protect from typos in attribute names?..

it has an unwrap() method which raises an exception if the value is not there, it seems

[devout barn]

Hey, how would I annotate a coroutine function?
is Callable[..., Awaitable[Any]] correct?

[trim tangle]

Can you be more specific maybe?

Do you really need an async function that accepts any arguments whatsoever and returns something unknown?

But yeah, generally, a callable that returns an awaitable is what you need

[fierce ridge]

async def rest(x: int, y: int) -> int:
    total = x + y
    await asyncio.sleep(total)
    return total

the type of rest should be Callable[[int, int], Awaitable[int]]

[grave fjord]

You might want a Coroutine there though

[soft matrix]

import typing

from dataclasses import Field, field, dataclass

T = typing.TypeVar("T")

class OneOf(typing.Generic[T]):
    __match_args__ = ("field", "value")
    field: T  # ideally this should be Field[T]
    value: T

@dataclass(repr=False)
class Something:
    with OneOf() as foo:
        bar: int = field(...)
        baz: str = field(...)


something = Something()

match something.foo:
    case OneOf(Something.bar, value):
        reveal_type(value)  # should be int
    case OneOf(Something.baz, value):
        reveal_type(value)  # should be str
```should this actually type check?

i feel like value should be narrowed to int but (at least with pyright) its Unknown

[hearty shell]

How does that even work at runtime?

[soft matrix]

black magic

OneOf is sort of like tagged union where only one value inside it can be set at once

[twilit badge]

Is an annotation like this possible? py def my_decor(*, **some_kwargs) -> Callable[[Callable[P, R]], Callable[P, R]]: def decorate(Callable[P, R]) -> Callable[P, R]: ... return decorate problem with this is that pyright considers P and R to be unbound, since they're only used in the return type of my_decor, not in it's parameters, but I want to signify that the parameters will stay the same after the decoration and my decorator also takes some arguments and uses an inner decorate function to perform the actual decoration.

[soft matrix]

i dont think P and R shouldnt be unbound there, you could however just use a type var bound to a callable to avoid this

[twilit badge]

ah, good idea. Pyright does report P and R as unbound there though

or rather, it says "Type Variable R has no meaning in this context"

[hearty shell]

But how is it ever set any value?

[soft matrix]

i havent shown that bit because its not relevant

[upbeat wadi]

well for one def decorate(Callable[P, R]) doesn't work, nor does def my_decor(*, **some_kwargs)

[soft matrix]

yeah i think you need to either update your version of pyright or fix your code

from typing import Callable, TypeVar, ParamSpec

P = ParamSpec("P")
R = TypeVar("R")


def my_decor() -> Callable[[Callable[P, R]], Callable[P, R]]:
    def decorate(t: Callable[P, R]) -> Callable[P, R]:
        return t

    return decorate```doesnt show anything for me

[twilit badge]

ah yeah, updating pyright did the trick

[hearty shell]

Interesting that it also cant type check Something.bar to anything unless you set OneOf.field in a wrapper

class OneOf(typing.Generic[T]):
    __match_args__ = ("field", "value")
    field: Field[T]
    value: T

match something.foo:
    case OneOf(Something.bar as field, value) as oneof:
        reveal_type(oneof.field)  # Type of "oneof.field" is "Field[Unknown]"
        reveal_type(field)  # Type of "field" is "int"

[soft matrix]

maybe this is a bug

the code actually works at runtime

[hearty shell]

Now, it is on patma or dataclasses hack xD

[soft matrix]

hmm this doesnt work for me still

oh wait it needs the field wrap thats weird

[hearty shell]

Yeah that is what I mean, if you take that out reveal_type(field) is also Unkown

Although I think it can be any wrapper

[soft matrix]

im gonna open an issue

[rough sluice]

:incoming_envelope: :ok_hand: applied mute to @rustic gull until <t:1648284420:f> (9 minutes and 59 seconds) (reason: duplicates rule: sent 4 duplicated messages in 10s).

[hearty shell]

@fickle stirrup You can use type alias to not have to repetitively type the same thing

[acoustic thicket]

lmao

[round berry]

hi all,
tiangolo uses typing in his FastAPI template and I'm wondering why he does this TypeVar Stuff for CreateSchemaType and UpdateSchemaType. Why not just add BaseModel to the class?

I am still very new to the topic of typing.

from typing import Any, Dict, Generic, List, Optional, Type, TypeVar, Union

from fastapi.encoders import jsonable_encoder
from pydantic import BaseModel
from sqlalchemy.orm import Session

from app.db.base_class import Base

ModelType = TypeVar("ModelType", bound=Base)
CreateSchemaType = TypeVar("CreateSchemaType", bound=BaseModel)
UpdateSchemaType = TypeVar("UpdateSchemaType", bound=BaseModel)


class CRUDBase(Generic[ModelType, CreateSchemaType, UpdateSchemaType]):
    def __init__(self, model: Type[ModelType]):
        """
        CRUD object with default methods to Create, Read, Update, Delete (CRUD).
        **Parameters**
        * `model`: A SQLAlchemy model class
        * `schema`: A Pydantic model (schema) class
        """
        self.model = model

[acoustic thicket]

to use a simpler example, suppose we have these classes:

class Parent: ...
class Child1(Parent): ...
class Child2(Parent): ...

then a signature like this

def f(x: Parent) -> Parent: ...

would mean that f could accept an instance of Parent, Child1 or Child2, and return an instance of Parent, Child1, Child2. the param type and the return type aren't related, so it could take a Child1 and return a Child2.

on the other hand,

T = TypeVar("T", bound=Parent)
def g(x: T) -> T: ...

would mean that the type g returns is exactly the same as the type of the arg you give it. if you gave it a Child1, it would return a Child1, if you gave it a Child2 it would return a Child2

[trim tangle]

also see https://decorator-factory.github.io/typing-tips/tutorials/generics/ 👀

[round berry]

thank you for the detailed explanation. I.e. the Generic definition in the Class only adds more possible types? And is a bit more detailed so you can see you can add both CreateSchemaType and UpdateSchemaType?

[round berry]

thanks for the link 🙂

[indigo locust]

So, I asked a few days ago

But since its a new day there's maybe a new crowd

Is there some way to give PyCharm's type checker a bit more juice? I'm talking about the check-while-you-type application of type checking

[steady seal]

putin rocks

[trim tangle]

what about putin's rock collection?

[median ledge]

So, I have

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from typing import Any

and I get this error:

    def __init__(self, path: Path, *args: Any, **kwargs: Any):
NameError: name 'Any' is not defined

aren't type hints ignored?

[void panther]

only if you use the annotations future import

[median ledge]

oh

[void panther]

otherwise they're normal expressions

[median ledge]

thanks :D

[trim tangle]

https://github.com/microsoft/pylance-release/issues/996 😔

I know it's probably not a very high-priority issue, but it's been a long time.

[minor nimbus]

Is this a MyPy bug or am I losing my mind?: https://mypy-play.net/?mypy=latest&python=3.10&gist=87fe95ea95e62eabc7e0a96ac078f875

URLType being a str in disguise, should be accepted anywhere a str normally would be, no?

[oblique urchin]

nope, invariance strikes again

mypy is correct

[minor nimbus]

But, but, but

[soft matrix]

mappings key type isnt covariant

[minor nimbus]

I made it a Mapping

ughhhhh

[trim tangle]

is there any actual issue with this though? 🤔

or is it just a limitation?

[soft matrix]

not in this case probably

[hearty shell]

I remember is being something about the getmethod

[soft matrix]

but it is actually unsafe

[trim tangle]

how tho?

[hearty shell]

https://github.com/python/typing/pull/273#issuecomment-243864718

[soft matrix]

anyways if you want to have a covariant key you need your own mapping protocol

[minor nimbus]

So, the key being a covariant typevar bound to str?

actually, binding should make it covariant by default, no?

[trim tangle]

ohhh right I get it, Mapping[URLType, str] is not a Mapping[str, str] beacause Mapping[URLType, str].__getitem__ can assume that its argument is a URLType, not just any string

[brisk hedge]

Does that example suggest Mapping should be contravariant on its key?
Well, I guess that would make it more annoying in real code

[oblique urchin]

Though you'd need a pretty exotic Mapping for that to matter

(replying to @trim tangle )

[brisk hedge]

Oh wait no mapping has keys() I think?

[oblique urchin]

yes

[brisk hedge]

so invariance is required?

[oblique urchin]

was just going to reply that that makes contravariance unsafe 🙂

[brisk hedge]

Yeah I don't remember the abcs off by heart

[minor nimbus]

Well, the point of func there is to accept any mapping that has str as keys, including TypedDict instances.

I didn't think that using a NewType would make it complain there

[soft matrix]

maybe just use a TypeAlias?

[trim tangle]

If it's unsafe to access why can't you just wear a mask? 🙄

[oblique urchin]

is mask a metaphor for NewType?

[trim tangle]

no just literally wear a mask while writing the code

[minor nimbus]

I guess so, although I liked the benefits of using NewType, like making sure you pass it around properly.

By the way, why on earth TypedDict inherits from Mapping[str, Any]?

Shouldn't it be Dict[str, Any]?

Mappings aren't even mutable

[trim tangle]

If it was a Dict[str, Any], you could add an arbitrary key to it

[hearty shell]

Any mutable mapping is also a mapping

[trim tangle]

I'm not sure I'm totally bought into NewType to be honest. If it's a separate concept with its own rules and invariants, why not make a proper class for it? That way you won't be able to construct an invalid value. (which you absolutely can do with a NewType)

[minor nimbus]

Hmm, I see

[brisk hedge]

from the pep:

[oblique urchin]

it doesn't add runtime overhead or mess with serialization

[brisk hedge]

A TypedDict isn’t consistent with any Dict[...] type, since dictionary types allow destructive operations, including clear(). They also allow arbitrary keys to be set, which would compromise type safety

[trim tangle]

well... validating your stuff naturally incurs an overhead 🤷

it just changes where it happens

[minor nimbus]

IIRC NewType is just an easy way to make a new distinct type without going through the trouble of making a separate class for it, or bothering with casting

You can cast a value by passing it into the constructor, like in my code:

URLType = NewType("URLType", str)
URLType("https://google.com")

[trim tangle]

class UserId:
    def __init__(self, raw: str) -> None:
        if len(raw) != 16:
            raise ValueError
        if set(raw) - ALLOWED_CHARS:
            raise  ValueError
        self._raw = raw

    def raw(self) -> str:
        return self._raw

NewType would serve the purpose well (like Haskell's NewType) if you could export the type but not the constructor.

[oblique urchin]

at work we use integer ids for all kinds of objects (users, comments, etc.). All of those have their own dedicated NewType. We don't necessarily do validation for them (that costs an expensive database query), but the NewTypes ensure that you don't pass the wrong kind of id to a function

[trim tangle]

I never suggested doing a query, just check e.g. that it's not negative or something

[trim tangle]

That is, other modules can consume e.g. an Email and use it in annotations, but not do Email("yep this is totally an email")

[oblique urchin]

That doesn't really gain you that much. And now every time we serialize it, we can't just serialize an int, but have to do something else that costs more bytes.

[brisk hedge]

the Type alias <-> NewType <-> namedtuple <-> dataclass hierarchy of type wrappers

[trim tangle]

I guess you can do py if TYPE_CHECKING: Email = _Email and then make some kind of note that Email should only be imported with TYPE_CHECKING. But it is a hack

oh hmmm actually

if TYPE_CHECKING:
    Email = _Email
else:
    Email = do_not_call_this

and then do_not_call_this will show some error if you try to construct it

[brisk hedge]

sometimes cheaper is more worth it

[hearty shell]

But why use a constructor instead of a function that just returns the NewType "instance"

[trim tangle]

because someone outside the module that upholds this validation can just do UserId(" ;;;;''''''%%%<script> nice ID here ")

[hearty shell]

Ah I see what you mean

[brisk hedge]

I don't think this is really something you can generalize to all use cases

[soft matrix]

Doesn't this violate pythons nothing is really private philosophy

Nothing else in python typing is really like this

[oblique urchin]

you can also always just do cast() 🙂

[trim tangle]

of course you can just do these "unsafe" things, but in this case UserId would be a public type with a public __init__

[hearty shell]

user_string = UserId.__new__()

What now 😂

[brisk hedge]

the goal of newtypes is more often to catch logic errors rather than security flaws
if your use case needs to validate, you should do that

[trim tangle]

So having UserId do the validation in the __init__ is the most fool-proof solution, I think. (you can circumvent it, but it requires some extreme dedication). But apparently the runtime cost is just too much

So maybe NewType is a good idea for internal stuff.
But for stuff like user input or remote API responses, you need to validate it anyway

[minor nimbus]

Guys, I'm so lost within what I just wrote

_JSON_T = TypeVar("_JSON_T", bound=Mapping[str, Any])

def json_load(path: Path, defaults: _JSON_T) -> _JSON_T:
    ...

so this is the function definition

actually, no

now it is

And now I have this URLType in the mix, which is just a str in disguise

which should fit into this, but it doesn't

I think I'll seriously just use a TypeAlias instead

[trim tangle]

Another issue might be that a NewType isn't an "opaque" type. If the underlying data of an ID changes (for example, yesterday it was an int and now the API owners decided that it's better a string), you might discover that a lot of your code depends on that ID being an integer.
But with a class, you might decide to make its representation only "public" to the module where it's defined. In that case, changes to it are localized.

[minor nimbus]

and call it a day

[trim tangle]

@minor nimbus what do you want to do?

[minor nimbus]

lemme craft some sample code

from pathlib import Path
from typing import Any, NewType, Mapping, Dict, TypedDict, TypeVar

SETTINGS_PATH = Path("settings.json")
HASHES_PATH = Path("hashes.json")
URLType = NewType("URLType", str)
_JSON_T = TypeVar("_JSON_T", bound=Mapping[str, Any])
Hashes = Dict[URLType, str]

class Settings(TypedDict):
    one: int
    two: str
    three: bool

default_settings: Settings = {
    "one": 1,
    "two": '2',
    "three": False,
}
default_hashes: Hashes = {}

def json_load(path: Path, defaults: _JSON_T) -> _JSON_T:
    ...

# everything below should type check properly
loaded_settings: Settings = json_load(SETTINGS_PATH, default_settings)
loaded_hashes: Hashes = json_load(HASHES_PATH, default_hashes)

I got the first one to work, somehow

After struggling with the fact that TypedDict wasn't accepted where Dict[str, Any] was

but now the second one doesn't

[brisk hedge]

are you sure you want both of these distinct behaviors (TypedDict and arbitrary dict) to be covered by one function

[minor nimbus]

this omits the fact that whatever is read from the underlaying JSON file will actually be what was saved and nobody tinkered around with it "offline"

actually, that sample misses the fact that hashes is stored in a different file

lemme fix that

[minor nimbus]

But yes, I do. That's the whole point of it - save and load a JSON-compatible dictionary from a filepath, no matter if it's a normal dict with string keys or TypedDict.

def json_save(path: Path, contents: Mapping[str, Any]) -> None:
    ...

saving has the same issue of Mapping rejecting URLType as the key type

Somebody mentioned creating my own mapping with covariant key type, but I got kinda lost on how to do so

Maybe like this?

_K_co = TypeVar("_K_co", covariant=True)
_V_co = TypeVar("_V_co", covariant=True)

class KeyCovMapping(Mapping[_K_co, _V_co]):
    ...

although _K_co needs to be a string, and bound=str kinda didn't work last time I checked

[brisk hedge]

does hashes need to be mutable?

[minor nimbus]

no, hashes themselves are also strings

Hashes = Dict[URLType, str]

it's a mapping of URLs to hashes

oh, you mean like, the dictionary?

[brisk hedge]

yes

[minor nimbus]

Well

it's gonna be modified once it's loaded

to be saved later

soooo

[brisk hedge]

you hit a fundamental problem trying to make the return value compatible with both immutable and mutable dicts -- the return type has to be invariant on its typevars

[minor nimbus]

ugh

What about overloads?

Is there a way to express "any TypedDict" returns that TypedDict, and any other dict returns that any other dict?

the defaults dict is actually copied within the function, which is why Mapping[str, Any] works

[brisk hedge]

"any TypedDict" is just Mapping[str, Any]

[minor nimbus]

right

so that's what I have already, via the bound typevar

[brisk hedge]

you could cast

[minor nimbus]

I mean, the only problem there is right now is with the second line

json_load(HASHES_PATH, default_hashes) gives Value of type variable "_JSON_T" of "json_load" cannot be "Dict[URLType, str]"

which isn't very helpful on it's own

but Dict[str, str] appears to work

which is why I asked here about the NewType being

[oblique urchin]

your bound might need to be Mapping[Any, Any]

[minor nimbus]

bruh

At this point, I may as well make URLType an alias of str and forget using NewType then

Changing to Hashes = Dict[str, str] makes both lines valid

[trim tangle]

I have a system with a hierarchy like this (hypothetically): Storage InMemoryStorage FileSystemStorage S3Storage RedisStorage right now all Storage objects have these methods: ```py
def all_keys(self) -> Iterator[str]: ...
def read(self, key: str) -> Optional[bytes]: ...
def save(self, key: str, contents: bytes) -> str:
"""
Save a file by a key, converting or escaping the key as required by the storage.
Return the formatted key.
"""
def pop(self, key: str) -> Optional[bytes]: ...

Take a look at this function:```py
def migrate_files(src: Storage, dest: Storage) -> Iterator[str]:
    """
    Move all the files from `src` to `dest`.
    While doing so, yield all the keys by which the files will be accessible.
    """
    for src_key in src.all_keys():
        contents = src.pop(src_key)
        if contents is None:
            continue
        dest.save(src_key, contents)
        yield src_key
``` Can you spot the bug? It's not really obvious unless you dive a bit deeper into the semantics of `save`. This is the correct ending of the function: ```py
        dest_key = dest.save(src_key, contents)
        yield dest_key
``` How would you prevent such mistakes from happening?

I have something like this in mind: Storage should be changed to be generic:

class Key(ABC):
    def display(self) -> str: ...

K = TypeVar("K", bound=Key)

class Storage(Generic[K], ABC):
    def all_keys(self) -> Iterator[K]: ...
    def read(self, key: K) -> Optional[bytes]: ...
    def save(self, raw_key: str, contents: bytes) -> K:
    def pop(self, key: K) -> Optional[bytes]: ...

Then you'll have something like this: ```py
def migrate_files(src: Storage[K1], dest: Storage[K2]) -> Iterator[K2]:
"""
Move all the files from src to dest.
While doing so, yield all the keys by which the files will be accessible.
"""
for src_key in src.all_keys():
contents = src.pop(src_key)
if contents is None:
continue
dest.save(src_key, contents)
yield src_key # type error!

now the type checker is complaining

However, this can get more complicated and then this breaks. For example, I might have Storages with Keys and also with Folders.
Folders let you list the files that belong to them, move files between them and all the usual stuff you do with folders. This could be one way to do it:

class Dir(ABC):
    def display(self) -> str: ...

D = TypeVar("D", bound=Dir)

class Storage(Generic[K, D], ABC):
    ...
    def read_dir(self, dir: D) -> list[K | D]: ...
    def remove_dir(self, dir: D) -> None: ...
``` But now this `Storage` object will need to be passed around. It will quickly get more methods and become a god class. It would be nice to have `Dir` objects themselves be responsible for doing stuff like this, `pathlib.Path`-style. But how can we tie these three (or more) related entities together?

They could just be concrete types with hard-coded dependencies on the particular key, directory etc., but the idea here is to write generic code that works with different kinds of storages.

[fierce ridge]

@trim tangle the "stupid" solution is InMemoryStorageKey: TypeAlias = str

or class InMemoryStorageKey(str): pass

i don't understand the Dir part

[north widget]

how to type hint if the second variable could be any type?

[hearty shell]

What do you mean?

[halcyon forum]

how do i get the type hint of a variable? like this

x: int = 10

get_type_hint(x) # should return int

# here is an incorrect one
y: str = 1

get_type_hint(y) # should return str

[hearty shell]

It lives on the global __annotations__

[halcyon forum]

but that only works on functions

[hearty shell]

!e

x: int = 10
y: str = 1

print(globals()['__annotations__']['x'])
print(globals()['__annotations__']['y'])

[rough sluice]

@hearty shell :white_check_mark: Your eval job has completed with return code 0.

001 | <class 'int'>
002 | <class 'str'>

[halcyon forum]

:O

wtf magic

thx

[hearty shell]

that is not the best way to get them though

[soft matrix]

Or you can use the module from sys.modules

[pastel egret]

Just do __annotations__['x']?

[hearty shell]

Oh right, lmao

[halcyon forum]

xd

[pastel egret]

But indeed, it's better to use typing.get_type_hints() with the module object - it handles evaluating stringified annotations (forward references) and a few things like that.

[trim tangle]

Why do you want this?

[fierce ridge]

but there's no way to get access to the current module object, is there?

or is there some way to apply the parse/eval logic in get_type_hints to an arbitrary __annotations__ dict?

This is often the same as obj.__annotations__. In addition, forward references encoded as string literals are handled by evaluating them in globals and locals namespaces
seems straightforward, but the implementation is very not trivial https://github.com/python/cpython/blob/main/Lib/typing.py#L2117-L2214=

can you do get_type_hints(sys.modules[__name__])??

[soft matrix]

ye

[fierce ridge]

nice, it works https://replit.com/@maximum__/get-type-hints-of-current-module#main.py

x: str = 'hello'
y = x
z: int = 99

import sys, typing
print(typing.get_type_hints(sys.modules[__name__]))

{'x': <class 'str'>, 'z': <class 'int'>}

[minor hedge]

Is there a reason to use one of Optional[Union[...] vs Union[..., None] over the other?

[oblique urchin]

they're equivalent to the type checker so it's a matter of taste

I prefer the latter because it's shorter, doesn't require an import, and is a bit more general

[minor hedge]

thank

[minor hedge]

When should you use TypeVar?
What does covariant do for typevars? Do you need it if you have a bound already?

and is there a good guide on typing stuff? was currently reading through https://peps.python.org/pep-0484 but wondering if there's something else to look at

[trim tangle]

I have a short tutorial series, but it's in an "alpha" stage
https://decorator-factory.github.io/typing-tips/tutorials/generics/

[oblique urchin]

the mypy docs (https://mypy.readthedocs.io/en/stable/index.html) are pretty thorough too

[minor hedge]

👀 neat thank

[devout barn]

How do I type narrow this properly?

    def find(self, key: KT) -> VT:
        v = self.get(key, MISSING)
        if v is MISSING:
            raise KeyError(key)
        return v  # type: ignore (v is not MISSING due to previous check)

Apparently pyright thinks that I am returning MISSING (we raise an KeyError beforehand if it's MISSING)

[hearty shell]

The default returns a missing type

[oblique urchin]

May need to make MISSING an element of a one-member enum

[hearty shell]

So all the type checker knows is that the default arguement has type missing, but not that it is Literally that Missing object

I think?

I was gonna say

Yeah it is, you can see that when you put the default as 5 for example, the return type is Union[Any, int], and not Union[Any, Literal[5]]
But actually there is not way to encode identity in pythons type system

[fierce ridge]

how is MISSING defined?

[devout barn]

it was MISSING = object()

I made it into a enum now pyright seems to accept this

Like this

class _Sentinel(_Enum):
    MISSING = object()

[fierce ridge]

wouldn't you have to write _Sentintel.MISSING then? or am i misunderstanding

[devout barn]

yeah, I'd have to

but I can write

MISSING = _Sentinel.MISSING

and linters should be able to accept that

although I'll stick with the first method

[fierce ridge]

i actually think MISSING = _Sentinel.MISSING would be a lot nicer to use

[devout barn]

there are only a minimal amount of reference (it's actually 4 times), and I already wrote it before mentioned it

Thanks btw!

[fierce ridge]

class _Missing:
    pass

MISSING = _Missing()

def is_missing(value: object) -> TypeGuard[_Missing]:
    return isinstance(value, _Missing)

what if you did this @devout barn ?

i'm not sure it's any better or more ergonomic than the one-member enum though

[devout barn]

This seems to work, yes.

[fierce ridge]

i kind of like the ergonomics of being able to write x is MISSING instead of is_missing(x) though

[devout barn]

Now I am confused on which method to use 👴

[fierce ridge]

what is _Enum btw?

just enum.Enum?

[devout barn]

it is just aliased

yeah

to indicate privacy

[fierce ridge]

i think i like the enum method better

it also lets you easily define other sentinels if you want them

and x is MISSING is elegant

(hopefully your actual use case is more sophisticated than this, because why not just do v = self[key] and re-raise the resulting KeyError?)

[rustic gull]

when should you typehint variables? when theyre gonna get mutated many times?

[oblique urchin]

when your type checker tells you to

or gets confused otherwise

often when it starts out as an empty list/dict that you fill in later

[rustic gull]

yeah i saw examples on it with lists so thats why i asked such a question

thanks!

[fierce ridge]

i only suggest type hinting variables that might have an ambiguous type. for example items = [] is ambiguous

so i would write items: MutableSequence[Whatever] = [] there

[rustic gull]

yeah it makes sense

[fierce ridge]

also i generally write type hints for class and instance attributes

[north widget]

Is that the way of type hinting for class objects?

[rustic gull]

why are wildcard imports not allowed in pyright? isn't that a linting thing or does it have something to do with types

i can type: ignore it but it just seems odd

[covert dagger]

it is a separate rule from the normal type checking rules (reportWildcardImportFromLibrary), so you can disable that in your config

but it is a bad idea to do those, so it is good that it complains by default

[fierce ridge]

it's a lot easier if you post your code in a codeblock. i can't copy and paste from a screenshot

!code

[rough sluice]

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

[fierce ridge]

you would write this B: type[traitlets.traitlets.MetaHasTraits] = A.__class__

[jagged sigil]

Print ~~~

[lean island]

I wish to be good at programming

[weary linden]

@brisk scarab @weary linden

[brisk scarab]

Is Samuel Dietz Abi here?

[mint inlet]

Quick question: does anyone have reasoning for why type[...] might be superior to (...) -> ... for a function that instantiates?

I have a vague idea that maybe for a function that both accesses class attributes and instantiates, but that instantiation isn't type safe -- is there something that's type safe that would be worse under the second typing?

[soft matrix]

type[T] should always meet (...) -> T so I don't see this ever being an issue if you only call the type

[blazing nest]

I find it inferior, because you can't get the arguments out of type[...]

That's the only reason I use callable over type though, if I need to use it with ParamSpec

[grave fjord]

I don't think type[T] does meet (...) -> T

You can make __new__ return all sorts of stuff

[oblique urchin]

Type checkers assume that it does though

[grave fjord]

ah mypy does ban it

Incompatible return type for "__new__" (returns "int", but must return a subtype of "Ham")

Callable[..., T] isn't a very useful type as you don't know what to call it with

[oblique urchin]

well at least you know what it returns 🙂

[grave fjord]

There's a whole bunch of places where you do need a type - isinstance/issubclass pytest.raises etc

[indigo locust]

Does anyone know a way to convince PyCharm's type checker to understand custom generic descriptors?

[grave fjord]

Show you code?

https://youtrack.jetbrains.com/issue/PY-47698 it's just broken

[indigo locust]

from typing import Generic, TypeVar

T = TypeVar('T')

class Descriptor(Generic[T]):

    def __get__(self, instance, prototype) -> T:
        ...


class IntDescriptor(Descriptor[int]):
    ...


class Object(object):
    attribute : Descriptor[int] = Descriptor()

As a contrived example, but I've tried everything I can think of

[indigo locust]

Yeah 😐 I've checked this out. Seems like a big fail, but its been years and there doesn't seem to be any interest in fixing it

[grave fjord]

I never run the type checker in my editor

Just run mypy in tox

[indigo locust]

Tox?

_> More than anything, I just wish there was a way to upgrade the static checker with mypy

Or something. But I've tried all the plugins but I can't seem to get them to work

[grave fjord]

There's plugins, but afaik it doesn't work with the go to definition tools

[hasty hull]

I'd recommend using pyright instead, it doesn't have support for plugins but it does support much more than mypy does and bugs are fixed insanely fast

[indigo locust]

Pyright instead of mypy

[hasty hull]

yes

but if you do need plugins you should stick with mypy

[indigo locust]

And the way your suggesting I use it is to just run it over my own code manually

[hasty hull]

Yes you'd have to use the CLI

Or switch to another editor that supports using Pyright as a language server, like VSCode

but that's an extreme change

[fierce ridge]

i do wish it was easier to write mypy plugins

[hasty hull]

same, it is painful

[indigo locust]

Its funny how, though we may be the pinnacle of theory and practice in some ways

We can't seem to get our shit together in others XD

[indigo locust]

So here's my problem, maybe you guys can point me in the right direction

Running mypy/pyright/whatever manually on my own code is fine

But I'm writing a framework as a tool for other to use in their own code (middleware is the term?) which is quite descriptor heavy. The core functionality of the framework is an element class with oodles upon oodles of properties. As a matter of saving code, it'd be really helpful to write custom descriptors

But that would throw at least all PyCharm users not comfortable with running their own checkers under the bus. I don't have any proof, but I'd imagine most other python IDEs are similarly weak in the static checking department

Ultimately, PyCharm itself aside, I think what I'm struggling with is that I feel I need to cater to the lowest common denominator in terms of autocompletion/prediction/tooling. Expecting my users to use this checker with this IDE

Seems unrealistic. I feel as though the responsibility should rest with me to come up with a graceful solution. Any thoughts?

[hasty hull]

Surprisingly a lot of editors are actually pretty good with this as a lot of them support LSP which means you can use Pyright as a language server. For example: https://github.com/fannheyward/coc-pyright

However any editors that roll their own type checker, like PyCharm, may or may not support it. While you can test these editors out and try to optimise to support them as well, it can be very difficult. If there's a fundamental feature that they don't support then there's nothing that you can do apart from file feature requests really. You can document that users of certain IDEs will experience degraded DX and suggest they switch to a different editor for the optimal experience.

Depending on the missing feature(s), it may actually not take that much time, for example with my project, PyCharm users were getting a much worse experience due to a lack of TypedDict autocomplete, I commented on the issue for that and it was added in less than a month: https://youtrack.jetbrains.com/issue/PY-40007

[indigo locust]

Hmmmmmmmmmm

https://tenor.com/view/hmmm-thinking-batman-gif-14744673

[nova venture]

is there anyway to force that a type hint is not abstract?

[hearty shell]

What do you mean?

[nova venture]

I need to force that a type hint has an annotation. typing.Annotated doesn't work with this since it's optional, so I came up with using a ClassVar on the class. But ClassVars are only guaranteed to be present in non-abstract classes:

class String(ABC):
    accepted_characters: ClassVar[Sequence[str]]

class NumberString(String):
    accepted_characters = "0123456789"

StringConsumer = Callable[[String], int]

def consume_string_1(string: String):
    return 0

def consume_string_2(number_string: NumberString):
    return 0

both consume_string_1 and consume_string_2 are considered to satisfy StringConsumer by a type checker, but I want to change this so only consume_string_2 is accepted

such that at runtime if I use typing.get_type_hints to extract the annotation of consume_string's parameter it's guaranteed to have a non-None accepted_characters

more generally, I'm looking to force that certain metadata on parameters is available at function declaration time. at function invocation time is easy, since a type checker will restrict instantiation of arguments to non-abstract classes that are guaranteed to have the meta data specified. But neither Annotated nor the above example allow metadata to be forced at function declaration time

[hearty shell]

I am not sure what you are tying to do? When you say granteed, do you mean at type checking or at runtime? Are you trying to make so that a function is only considered a StringConsumer if the argument object it consumes has a accepted_characters class variable?

Or are you trying to make so that in order to subclass String, one must declared a accepted_characters and make that enforced by the typechecker

[nova venture]

"When you say granteed, do you mean at type checking or at runtime?" -- both, runtime assumes that a type checker passed so at runtime the check doesn't actually have to happen and there will be no try / except for missing attribute

"Are you trying to make so that a function is only considered a StringConsumer if the argument object it consumes has a accepted_characters class variable?" -- exactly, such that my framework can attempt extraction of accepted_characters and have a guarantee that it exists if my function could be passed to this type-annotated framework in the first place in the eyes of the type checker (meaning I need to find the right annotation here)

"Or are you trying to make so that in order to subclass String, one must declared a accepted_characters and make that enforced by the typechecker" -- also this, currently it's not enforce that subclasses actually specify the ClassVar so it's possible for it to be None even though the type within the ClassVar isn't Unioned with None.

[hearty shell]

Alright, so after testing a bit I think what you are tying to do may not be possible unless I misunderstood the problem, just to confirm

both consume_string_1 and consume_string_2 are considered to satisfy StringConsumer

That should not be the case

[nova venture]

Exactly, I only want the latter to satisfy it

[hearty shell]

No but I mean it should be the opposite, because Callable[[NumberString], int] is not a subtype of Callable[[String], int] due to contravariance. Maybe someone else might know a different approach but I don't see a way around. I thought you could use Protocols for something like this but I have no idea what contravariance would mean when using protocols

[minor hedge]

Why does slicing/defining a list raise errors about invariance vs passing it in directly is fine?

def func(x: list[Union[int, str]]):
    ...

func(['5'])  # OK

x = ['5']
func(x)   # err
func(x[:2]) # err
func(['5'][:2]) # err

[hearty shell]

from typing import Union

def func(x: list[Union[int, str]]):
    x.append('foo')

lst = [5]  # Type of "x" is "list[int]"

func(lst)  # Safe, presumibly

sum(lst) # Boom

Just the way it is inferred

[minor hedge]

im getting errors at trying to pass it to func being invariant though?

[hearty shell]

What do you mean

[minor hedge]

thing: list[str] = [ ... ] # bunch of strings

class X:
    def __init__(self, input: list[Union[..., str]]):

X(thing) # err

[hearty shell]

func(['5'][:2]) # err also this probably doesnt work because the type checker has to resolve ['5'] before it can infer that is should be list[int | str], unlike when you func(['5'])

[hearty shell]

The same thing applies, __init__ could be mutating the list, even though that would be really unexpected

You want to do def __init__(self, input: Sequence[Union[..., str]]):

[minor hedge]

huh 🤔

so the err isn't from the func not liking it, but that thing is supposed to be only str and func could mess up the type of thing

[hearty shell]

I mean both, the function doesnt like it because it expects list[Union[..., str] and you gave it a list[str], these two types are not compatible, and it is made that way because func could mess the type of the passed argument

[minor hedge]

huh

then sequence should be used instead of list for things like tuples/lists that dont specificalyl require one?

why doesn't list[union[..., t]] not like list[t]?

[hearty shell]

It is because of the invariance of the list, what that means is that list[X] and list[Y] are never compatible, no matter the relation between X and Y. Sequence on the other hand is covariant, meaning that the type of the Sequence[T] changes in accordance with T.

[minor hedge]

huh

thank

[hearty shell]

Np

[hearty shell]

Also, tuple is fine since it is immutable

[stray tide]

from typing import Callable, Any

def foo(func: Callable[int, int]) -> Callable[int, int]:
    return func

foo(lambda bar: bar ** bar)(15)
``` is this correct?

[delicate swallow]

@nova venture Do you mean like annotating a function such that it only accepts certain inputs?

[delicate swallow]

At least that's what I'm getting at based on your example here

[acoustic thicket]

Callable[[int], int]

[stray tide]

ty

[glass mirage]

hi there! I've a question about type aliases and using | with from __future__ import annotations

In Python 3.7-3.10 I can use the typing shortcuts like this instead of using Dict and Union:

from __future__ import annotations

data = {
    "name": "bob",
    "age": None,
}

def save(data: dict[str, str | None]) -> None:
    ...

And in 3.10 I can put that in a TypeAlias to avoid repetition:

DataTypeAlias = dict[str, str | None]

def save2(data: DataTypeAlias) -> None:
    ...

But in 3.7-3.9 that gives:

    DataTypeAlias = dict[str, str | None]
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Am I doing something wrong? thanks!

[proud brook]

Mind you enlighten me and tell me the difference? I'm curious

[acoustic thicket]

just how callable's annotation works
the first thing is a list of types representing the arguments of the callable and the second thing is the callable's return type

if there were two int arguments, itd be Callable[[int, int], int]

[trim tangle]

The future import only affects annotations. dict[str, str | None] isn't an annotation in this context, it's just a normal expression you save into a variable. So you'll have to use a string or the "old" way.

[hearty shell]

You can just do DataTypeAlias: TypeAlias = "dict[str, str | None]" thought

and then import TypeAlias from typing_extensions

[glass mirage]

Thanks, this works!

from __future__ import annotations
from typing_extensions import TypeAlias

data = {
    "name": "bob",
    "age": None,
}

DataTypeAlias: TypeAlias = "dict[str, str | None]"

def save2(data: DataTypeAlias) -> None:
    ...

[hearty shell]

One thing to note is that if this is library code you might wanna wrap the TypeAlias import in a TYPE_CHEKING if, if you dont want to add that as a library dependency

[dapper yacht]

How do I typehint this properly?

from functools import partial
import typing

def myfunc(*, myint: int, mystr: str) -> str:
  return str(myint) + mystr

myfunc_prefilled = partial(myfunc, myint=0)

class MyFuncPrefilledProtocol(typing.Protocol):
  def __call__(self, *, mystr: str) -> str:
    ...

def anotherfunc_using_partialfunc(partialfunc: MyFuncPrefilledProtocol) -> None:
  ret = partialfunc(mystr="test")  # Expected type 'MyFuncPrefilledProtocol', got 'partial[str]' instead 
  print(ret)

[vocal jasper]

def __init__(self, name: str, age: int, gold: int) -> None:
        self.name = name
        self.age = age
        self.gold = gold

When I started the linter, it expected me to add type annotations for all the variables, but it is also saying i should do that for self as well ? How would one do that

[south topaz]

` not '

[vocal jasper]

Thanks !

[south topaz]

also, shouldnt be a space between py and the ` i believe

[vocal jasper]

its after i guess, '''py

[vocal jasper]

I added the types to all variables, but not sure what I should add for self, the name of class is Brothers

[south topaz]

oh yeah, now i remember

[trim tangle]

Can you tell what linter you are runnimg, and what message exactly you get?

[vocal jasper]

I'm using flake8 and this here is the error popup (parameter) self: Self@Brothers
Missing type annotation for self in methodflake8(ANN101)

not exactly error, is blue underline an error ?

[trim tangle]

If it's flake8-annotations, it has some horrible defaults

Are you using mypy?

Or any other type checker

[vocal jasper]

I can switch to mypy no issues, is this always the case with flake8 ?

When I created a small PR in python, discord guides mentioned something about using flake8 so that was my default one

[trim tangle]

Flake8 doesn't check types. Flake8-annotations will just tell you that your function is missing annotations. But it won't help you if you use a string where an int is expected

The latter is the job of a type checker, like mypy or pyright

[vocal jasper]

Is there a link, where I can read more on this ? I don't get the difference. mypy and flake8 are both linters right so they both should do the jobs right ? or flake8 just whines that something is wrong and mypy will point whats wrong

[trim tangle]

x: str = 42

Flake8 is ok with this. Because it does not understand the semantics of type hints.

Mypy will point out the mistake here.

If you want to read more, there are some resources in the pins

[vocal jasper]

That clears it up a little. I'll take a look at the pins. Thanks for your time !

[minor hedge]

how do you type for mixins?

something like ```py
class Mix: ...
class Main: ...
class Sub(Main): ...

class SubMix(Mix, Sub): ...

def func(x: ???):

func(SubMix)```

[soft matrix]

Currently it's hard

[minor hedge]

😭

[soft matrix]

Oh

Umm

Does func only takes SubMix? Or does it take any subclass of Mix and Sub?

[minor hedge]

any subclass of Main + mix

[soft matrix]

Right so your options are either curl up in a ball and cry until actual intersections are introduced or use a Protocol for some duck typing

[minor hedge]

how does protocol work? I tried that right now but I might be using it wrong

[soft matrix]

!d typing.Protocol

[rough sluice]

typing.Protocol

class typing.Protocol(Generic)```
Base class for protocol classes. Protocol classes are defined like this:

```py
class Proto(Protocol):
    def meth(self) -> int:
        ...
```  Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing), for example...

[minor hedge]

do attributes need to be properties or can I just add an attr and type it

[soft matrix]

A typed attribute should be fine

[minor hedge]

have a feeling my typing is probably wrong somewhere 😔
my mixed class passes isinstance with the protocol but I got no idea how when subclasses will work or no

[soft matrix]

they should

[minor hedge]

Not sure if this is too little to go on and also not quite sure what I've done
both ActualItem classes pass isinstance with Proto

ZV = TypeVar('ZV', bound='DiffClass')
class Mix:
    def __init__(
        self,
        a1: Optional[Callable[[ZV], bool]] = None,
        a2: Optional[Callable[[ZV, Proto], Any]] = None
    ):
        ...

# Sub/Sub_2 are subclasses of Main
class Mixed(Mix, Sub): ...

#########
# In a different file
class ActualItem_1(Mixed): ...

class ActualItem_2(Mix, Sub_2): ...

class SubDiffClass(DiffClass):

def aa1(v: SubDiffClass, i: ActualItem_1): ...

def a2(v: SubDiffClass) -> bool: ...

def aa2(v: SubDiffClass, i: ActualItem_2): ...

ActualItem_1(
    a2=aa1 # param 1 and param 2 don't match
)

ActualItem_2(
    a=a2, # no errors here
    a2=aa2  # only param 2 doesn't match
)

[soft matrix]

whyve you done TypeVar[]?

[minor hedge]

O I mistyped that

[west abyss]

What's the difference between .pop() and .remove()?

[scarlet nacelle]

this channel is for typehinting.

but .pop() removes a list index and returns the object that was in it, while remove() removes an index by the object and does not return it

[honest vector]

I am wondering is there a command I can use to generate pyi files from a pyx file or would it be smarter to write a pyi stub file by hand?

[dim trail]

!e im having some issues with pyright (newest version 1.1.234) when trying to inherit from collections.abc.ValuesView. It seems pyright fails to understand the init of the abc.ValuesView superclass.

import collections.abc

class MyView(collections.abc.ValuesView):
    def __iter__(self):
        for key in self._mapping:
            yield key + 1

for key in MyView({1: "a", 10: "b"}):
    print(key)

as you can see, this works fine in python. Pyright throws me this error though:

G:\t.py
  G:\t.py:6:25 - error: Cannot access member "_mapping" for type "MyView"
    Member "_mapping" is unknown (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations```

Edit: messed up the example, it should be KeysView but that doesn't matter really

Would anyone know why pyright is unable to see the init of ValuesView? ValuesView inherits its init from MappingView which looks like this:
```py
class MappingView(Sized):

    __slots__ = '_mapping',

    def __init__(self, mapping):
        self._mapping = mapping

[rough sluice]

@dim trail :white_check_mark: Your eval job has completed with return code 0.

001 | 2
002 | 11

[trim tangle]

@dim trail I would make an issue on the microsoft/pyright repo

although maybe it's a problem in typeshed?

[void panther]

the stubs don't document it I assume

[hearty shell]

https://github.com/python/typeshed/blob/ae6ff79c0e14cd054bf07374e7091a098a46cf1f/stdlib/typing.pyi#L900

[rough sluice]

stdlib/typing.pyi line 900

class ValuesView(MappingView, Iterable[_VT_co], Generic[_VT_co]):```

[dim trail]

thats... dissapointing

[trim tangle]

what's the point if it's undocumented? 🤔

[dim trail]

hm?

[trim tangle]

I meant, why have this method in the base class if it's not documented

[dim trail]

which method?

[trim tangle]

https://github.com/python/typeshed/blob/ae6ff79c0e14cd054bf07374e7091a098a46cf1f/stdlib/typing.pyi#L900-L901

[rough sluice]

stdlib/typing.pyi lines 900 to 901

class ValuesView(MappingView, Iterable[_VT_co], Generic[_VT_co]):
    def __init__(self, mapping: Mapping[Any, _VT_co]) -> None: ...  # undocumented```

[hearty shell]

what if you inherit from dict_values?

that one seems to have a mapping

https://github.com/python/typeshed/blob/ae6ff79c0e14cd054bf07374e7091a098a46cf1f/stdlib/_collections_abc.pyi#L71-L75

[rough sluice]

stdlib/_collections_abc.pyi lines 71 to 75

@final
class dict_values(ValuesView[_VT_co], Generic[_KT_co, _VT_co]):  # undocumented
    if sys.version_info >= (3, 10):
        @property
        def mapping(self) -> MappingProxyType[_KT_co, _VT_co]: ...```

[hearty shell]

oh wait

lmao

@final

[dim trail]

well, i dont need the method mapping, i need the attribute _mapping which is defined in abc.MappingView._init_

its not hard to work around the fact that its not documented, but i thought i was doing something wrong

question: should i raise an issue about this in typeshed? im not sure how you would even stub a attribute (i havent really worked with stubs at all)

[void panther]

not much to lose if you do, from the typing pov the class is not particularly useful when its only attribute is not available

[dim trail]

yeah...

im a bit confused: yall linked to stdlib/typing.pyi for the stubs that are (apparently?) used for collections.abc
would stdlib/typing.pyi even be the correct place for documenting that attribute, since collections.abc dont necessarily have to match other implementations that can be typedhinted as, for example, MappingView?

further: why would pyright use the typing.pyi hints for collections.abc?

ah, _collections_abc.py imports them directly from typing.pyi

[trim tangle]

not sure if that's related, but since 3.9 these imports from typing are deprecated in favour of collections.abc

[dim trail]

odd, im on 3.9.7 with this issue

aight done, if anyones curious
https://github.com/python/typeshed/issues/7576

[trim tangle]

I guess you could just define the __init__ yourself tbh

[grave fjord]

Why would a ValuesView yield keys?

[dim trail]

because i messed up the example

same difference tho for KeysView, they both inherit self._mapping from MappingView

[grave fjord]

I think you're supposed to implement the constructor to use it

[trim tangle]

that sounds reasonable

I think __init__ is often not considered part of interface in Python. You'd rarely run it in a polymorphic way

it also often breaks LSP

[fierce ridge]

I don't agree with this. Mypy does check the LSP for init, and it should be considered part of the interface

Exception types make a mess of this

[pastel egret]

What I do is redefine _mapping in the new view class, which then lets you define it as the specific mapping you're doing a view on so you can access its internals. Also suppresses a private attribute access warning, which is handy.

[terse sky]

That's definitely not a standard concept in polymorphism

In most languages with OOP, the constructor wouldn't be considered a part of the polymorphic interface

[oblique urchin]

It does not: https://mypy-play.net/?mypy=latest&python=3.10&gist=3e01a11f93cd458391fda83af82860ca

[terse sky]

Init isn't exactly the same as a constructor but practically speaking it is the analogue

In statically typed languages you can't construct polymorphically

In python you can but such things are mostly going to be outside the static type system and are relatively rare

[indigo locust]

If I want to implement a method similar to a dictionaries 'get' method

Wherein there is an optional default argument, and that argument changes the return type from the value-type of the dict to 'value-type | default-type'

Like, how do I do that...?

[oblique urchin]

https://github.com/python/typeshed/blob/master/stdlib/typing.pyi#L929

[rough sluice]

stdlib/typing.pyi line 929

@overload```

[oblique urchin]

This is how dict.get itself is typed

[indigo locust]

    @overload
    def get(self, __key: _KT) -> _VT_co | None: ...
    @overload
    def get(self, __key: _KT, default: _VT_co | _T) -> _VT_co | _T: ...

This was going to be my follow up question

Whether one could overload with different arguments, accepting for those with option args

This is actually quite helpful, thank you

[fierce ridge]

Huh, for some reason i thought it did, but i must have been confused

[trim tangle]

simple example: object's __init__ takes 0 arguments, its subclasses often don't 🙂

Although, I guess, it's a stretch. You could say the same about hashing.

[chrome hinge]

Why does this not work?

def __total_constructor(name: str) -> Callable[["State"], Amount]:
    def wrapper(self: State) -> Amount:
        return sum(getattr(kind,name+"_per_volume")*vol for kind, vol in self.blocks)
    return wrapper

@dataclass
class State:
    blocks: list[tuple[BlockKind,Amount]]
    mass = __total_constructor("mass")

Pylance error:

Expression of type "(State) -> Amount" cannot be assigned to declared type "(self: Self@State) -> Amount"
  Type "(State) -> Amount" cannot be assigned to type "(self: Self@State) -> Amount"
    Position-only parameter mismatch; expected 1 but received 0

the problem seems to be State vs Self@State, which I'm not sure how to solve.

EDIT: disregard that. I omitted an old definition of def mass later, after removing it it works.

[chrome hinge]

Actually, now my problem is that pylance now thinks I'm missing arguments (the self) when I do state.mass(). Any way of making it realise it's an instance method?

[trim tangle]

I tried, didn't have any luck tbh.

Maybe you could make a custom descriptor?

mypy has the opposite problem - it thinks all Callable attributes are methods. Not sure how to combine these

[pastel egret]

It kinda seems like we need a DescCallable type to handle this distinction or something.

[acoustic thicket]

crude workaround

@dataclass
class State:
    blocks: list[tuple[BlockKind,Amount]]
    def mass(self) -> Amount:
        return __total_constructor("mass")(self)

[fierce ridge]

hmm... valid. but yeah i think that might be a special case

[trim tangle]

I wonder how useful LSP is really, in a strict sense.

From my experience, very often classes don't specify a lot about their contracts explicitly.

Like, if you override a method with raise NotImplementedError, it's probably not the expected implementation. But in other cases - not so much

case in point: Mappings. Is defaultdict a mapping? Who knows, Mapping doesn't specify its laws anywhere!

There are some implicit expectations about mappings that people might disagree on

[fierce ridge]

well it's important/necessary if you have a covariant type declaration

[fierce ridge]

I was just thinking about the parameters and their types

"laws" of behavior are nonexistent or at best suggestions, most of the time

[terse sky]

I think it's pretty obvious that Mapping is non mutation API

That's why there's MutableMapping

:-)

[foggy thicket]

Hello! How do I say "This is a callable which takes a variadic of str instances" ?
I tried doing Callable[[tuple[str, ...]], Any], but that doesn't seem quite right

Callable[[str, ...], Any] isn't valid either

Callable[..., Any] is valid (but it doesn't convey that we have str instances)

[soft matrix]

You need to use a callable protocol

Or wait for TypeVarTuple to support bounds

[hearty shell]

why are bounds necessary?

Doesnt Callable[[Unpack[tuple[str, ...]]], Any] do what they want?

[foggy thicket]

Unpack? 👀

[hearty shell]

from typing_extensions import Unpack

pyright only

[foggy thicket]

Oh

[hearty shell]

for now*

That = Callable[[Unpack[tuple[str, ...]]], Any]

def foo(a: str, b: str): ...
def bar(a: str, b: str, c: int): ...

a: That = foo  # Ok
b: That = bar  # Error

[soft matrix]

Oh huh

I didn't know that worked

[brisk heart]

Wait does that mean I can finally annotate something as

tuple[str, Unpack[tuple[int, ...]]]

Cause if it's indented for this kinda use that's a game-changer for me

[hearty shell]

Yup you can, https://peps.python.org/pep-0646/#unpacking-concrete-tuple-types

Only limitation is that you can only unpack once per type parameter

Although no real reason is given, only for unpacking TypeTupleVars

And even for TypeTupleVars, if bounds were a thing, the ambiguity stated as a reason would not really apply I think

[pastel egret]

The reason is that Unpack[x] is a backwards compatibility replacement for *x which 3.11 is going to allow in these contexts.

[hearty shell]

Syntactically, you will be able to do tuple[*(1,2), *("foo", "bar")]

Nothing in the language prevents or will prevent both of those scenarios

Second, more than one instance of a star-unpack can occur within an index:
array[*idxs_to_select, *idxs_to_select] # Equivalent to array[1, 2, 1, 2]
Note that this PEP disallows multiple unpacked TypeVarTuples within a single type parameter list. This requirement would therefore need to be implemented in type checking tools themselves rather than at the syntax level.

[rustic gull]

how can I add \r\n to file

[grave fjord]

You probably want #tools-and-devops ?

[rustic gull]

ok

I asked there

[devout barn]

Hey is there any good place which explains about variance in typing? (Invariant, covariant and contravariant in typevar)?

[soft matrix]

Variance - Typing Tips

https://decorator-factory.github.io/typing-tips/tutorials/generics/variance/

[nova venture]

Is there a way to enforce that all instance fields of a subclass are of a certain type? For example:

DeserializableField = int | str | DeserializableObject | None

class User(DeserializableObject):
    name: str
    parent: User | None 

I want any subclass of DeserializableObject to have all of its fields be instances of DeserializableField

[oblique urchin]

Not with the static type system, but you could do it in __init_subclass__

though you'd need a runtime notion of "instance of a union"

[nova venture]

thank you, not perfect since at type-check time would be best but checking on class declaration time is better than on use time

runtime notion of "instance of union" would be isinstance(x, typing.get_args(union)) right?

[oblique urchin]

Not exactly if stuff like Any and Literal is involved

[rustic gull]

is typing.Genericfor linking argument types with the return type? so if i satisfied the argument with a string it would link the return annotations?

[oblique urchin]

you seem to be thinking about generic functions, Generic is for generic classes

[rustic gull]

and what is TYPE_CHECKING for i see it all the time

[rustic gull]

ive only really seen

T = TypeVar("T")
class A(Generic[T]):
    ...

so i am confused

[oblique urchin]

the definition of list in typeshed is a good example: https://github.com/python/typeshed/blob/master/stdlib/builtins.pyi#L837

[rough sluice]

stdlib/builtins.pyi line 837

class list(MutableSequence[_T], Generic[_T]):```

[oblique urchin]

the _T indicates what type is in the list

[rustic gull]

alright

[oblique urchin]

so there is def append(self, __object: _T) -> None: ..., because if you append to a list of T, you have to give another T object

[oblique urchin]

it indicates some code should be executed only by the type checker. At runtime TYPE_CHECKING is False, so code in if TYPE_CHECKING won't get executed

it's often used for imports that only the type checker needs

[rustic gull]

im still quite confused on it sorry ive been dealing with generics for a while and i dont seem too catch on

[rustic gull]

ah i see why would it be True in some cases?

not knowing some types so it would be unknown or?

[oblique urchin]

It's True to a static type checker, False at runtime

[rustic gull]

ah i see

thank you! im still confused on the generics tho i read a tutorial and docs but i still cant catch on

[north widget]

Hello. I am learning writing type hints and not too familiar with it. There are variables in the line 15 with statement. I want to type hint the urllib.request.urlopen() as well. Is there anyway to mark the type for it?

Also the following code is that clean? can it be more explicit on the types of each variables?

Question 2: I want to type hint generator that returns instances of future. My trial is generator[concurrent.futures._base.Future] but it returns error. Is that any alternatives? thank you

import concurrent.futures
from email import generator
import urllib.request

URLS = [
    "http://www.foxnews.com/",
    "http://www.cnn.com/",
    "http://europe.wsj.com/",
    "http://www.bbc.co.uk/",
    "http://some-made-up-domain.com/",
]

# Retrieve a single page and report the URL and contents
def load_url(url: str, timeout: int) -> bytes:
    with urllib.request.urlopen(url, timeout=timeout) as conn:
        return conn.read()


# We can use a with statement to ensure threads are cleaned up promptly
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    # Start the load operations and mark each future with its URL
    future_to_url: dict[concurrent.futures._base.Future : str] = {
        executor.submit(load_url, url, 60): url for url in URLS
    }
    completed_threads: generator[concurrent.futures._base.Future] = concurrent.futures.as_completed(future_to_url)
    
    for future in completed_threads:
        url = future_to_url[future]
        try:
            data = future.result()
        except Exception as exc:
            print("%r generated an exception: %s" % (url, exc))
        else:
            print("%r page is %d bytes" % (url, len(data)))

[runic tapir]

You don't need to. Your type checker will read https://github.com/python/typeshed/blob/be64b01e6cee5cbcb98e8934abacf00f01acbeb5/stdlib/urllib/request.pyi#L52-L61 automatically, and know what type that returns, without you needing to add any annotation.

[rough sluice]

stdlib/urllib/request.pyi lines 52 to 61

def urlopen(
    url: str | Request,
    data: _DataType | None = ...,
    timeout: float | None = ...,
    *,
    cafile: str | None = ...,
    capath: str | None = ...,
    cadefault: bool = ...,
    context: ssl.SSLContext | None = ...,
) -> _UrlopenRet: ...```

[acoustic thicket]

(also the annotation for dicts is [T, U] not [T: U])

[little hare]

what do typecheckers use internally to be able to show a user where something is defined or referenced without actually running and instantiating an object?

[runic tapir]

static analysis?

[oblique urchin]

typeshed for the standard library, installed stubs (which may also come from typeshed) otherwise

And sometimes also, just parsing the code

[acoustic thicket]

email.generator also looks a bit weird, you likely meant to use collections.abc.Generator

[little hare]

but how would those help with getting all of the references?

[oblique urchin]

oh that's just static analysis

[little hare]

this references everything that refers to the content attribute on a Message instance, even if they're in different files

[runic tapir]

the stubs don't help with that. For those, it parses the code into an AST and finds it out from that.

or, attempts to.

[little hare]

is static analysis ast parsing?

[runic tapir]

ast parsing is a type of static analysis

"static analysis" just means figuring out what code will do without running it

[north widget]

thank you. so urlopen() checks type within itself.

I was trying to add type hints for every variable of the script to increase the readability of the code, so i am curious is there any ways to mark type hint for with. ... statement?

[runic tapir]

as opposed to "dynamic analysis", which would be figuring out what code does by monitoring it as it runs

[little hare]

then what do language servers have to do with type checking*?

*asked this question elsewhere and someone said you'd need a language server

[oblique urchin]

you generally shouldn't need to add type hints for every variable

locals should only be annotated if they're otherwise ambiguous (your type checker will tell you)

[runic tapir]

adding type hints where they're unnecessary will mostly just make code less readable. But yes, you can do something like ```py
conn: SomeConnectionObject
with urllib.request.urlopen(url, timeout=timeout) as conn:

[oblique urchin]

focus on annotating function parameters and return types

[runic tapir]

yeah, that's where 98% of the value lies.

[little hare]

from what's been said I could just parse ast of a file and go from there

without needing a language server

[runic tapir]

nah, that's a misunderstanding of what a language server is for

[acoustic thicket]

by language server are you referring to the language server protocol

[oblique urchin]

They both need to understand the code. In your example, to find all places where Message.content is referenced, you need to know what variables are of type Message

[little hare]

no idea

[oblique urchin]

which is basically what a type checker does

[runic tapir]

language servers are designed to provide an interface between editors/IDEs and syntax-aware tools (refactoring tools, static analysis, type checking, linting, etc).

[acoustic thicket]

because the LSP is just how an extension communicates with the ide afaik

[runic tapir]

the idea behind the language server protocol is that, if you wrap a tool in a language server, you can use it with any editor/IDE that knows about language servers

[little hare]

so if i want to be able to find the first place an attribute is defined then ast parsing would be the best solution, if inspect can't help

[north widget]

@oblique urchin @runic tapir
I am quite confusing when I read the script that heavily depends on class.method.attr (eg: BeautifulSoup, Concurrent.future, so I was going to mark the type of each of them explicitly so I would know what the line is actually doing.

What do you think about such ?

[runic tapir]

in the past, integrating a tool into an IDE was an N*M problem - for each IDE, write an integration for each tool. Language servers turn that into an N+M problem: for each IDE, teach it to speak the language server protocol, and to launch language servers. For each tool, teach it to speak the language server protocol.

[runic tapir]

it sounds like a bad idea that will make your code harder to read.

[hearty shell]

Your editor should be aware of what those variables are without you telling what it is because supposedly someone already did the work

[oblique urchin]

Is this true in practice though? Feels like LSP is still mostly a Microsoft thing

[runic tapir]

I'm using pyright and clangd with neovim. Both are awesome.

[nova venture]

from collections.abc import Mapping
from typing import TypedDict

JSON = int | str | Mapping[str, "JSON"] | None

class MyDict(TypedDict):
    a: int
    b: str

def consume_type(a: type[JSON]):
    pass

consume_type(MyDict)

Why does this result in

error: Argument of type "Type[MyDict]" cannot be assigned to parameter "a" of type "Type[int] | Type[str] | Type[Mapping[str, JSON]] | Type[None]" in function "consume_type"
    Type "Type[MyDict]" cannot be assigned to type "Type[int] | Type[str] | Type[Mapping[str, JSON]] | Type[None]"
      "Type[MyDict]" is incompatible with "Type[int]"
      Type "Type[MyDict]" cannot be assigned to type "Type[int]"
      "Type[MyDict]" is incompatible with "Type[str]"
      Type "Type[MyDict]" cannot be assigned to type "Type[str]"
      Type "Type[MyDict]" cannot be assigned to type "Type[Mapping[str, JSON]]"
        TypeVar "_VT_co@Mapping" is covariant
          Type "object" cannot be assigned to type "JSON" (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations 

when running with pyright?

[runic tapir]

pyright is actually pretty incredible - it can complete keys in values typed as TypedDict

[nova venture]

it seems pyright is trying to make MyDict satisfy ever type of the union instead of just one in above example

[runic tapir]

like, ```py
from typing import TypedDict

class Movie(TypedDict):
name: str
year: int

def get_name(movie: Movie) -> str:
return movie["

which is way smarter than I've ever seen any other Python completer be 🙂

[little hare]

huh i just came across another snag, I'll be back in #internals-and-peps

[runic tapir]

the language server protocol is pretty new, but it's quickly becoming ubiquitous.

[acoustic thicket]

pycharm does it!!!

[oblique urchin]

here's a simpler example of I think the same thing ```from collections.abc import Mapping
from typing import TypedDict, Union

class MyDict(TypedDict):
a: int
b: str

def consume_type(a: Mapping[str, Union[str, int]]):
pass

m: MyDict = {"a": 1, "b": "x"}
consume_type(m)

[runic tapir]

what does it use for the static analysis? some proprietary JetBrains stuff?

[acoustic thicket]

dunno

[oblique urchin]

its own type checker

[acoustic thicket]

its typechecker is well

[oblique urchin]

(so basically yes)

[acoustic thicket]

subpar

[north widget]

Is there anyways to type hint for a generator that returns a specific data type? eg
completed_threads: generator[concurrent.futures._base.Future] = concurrent.futures.as_completed(future_to_url) (It is in a wrong syntax btw)

[nova venture]

pyright says this about that:

error: Argument of type "MyDict" cannot be assigned to parameter "a" of type "Mapping[str, str | int]" in function "consume_type"
    TypeVar "_VT_co@Mapping" is covariant
      Type "object" cannot be assigned to type "str | int"
        "object" is incompatible with "str"
        "object" is incompatible with "int" (reportGeneralTypeIssues)

but where is it pulling object from?

[runic tapir]

JSON = int | str | Mapping[str, "JSON"] | None
...
def consume_type(a: type[JSON]):
    pass
...
consume_type(MyDict)
``` You've declared that `consume_type` takes a parameter that is `JSON` or a subtype of `JSON`, and then you passed it a subtype of `dict`, rather than a subtype of `JSON`.

[runic tapir]

https://mypy.readthedocs.io/en/stable/kinds_of_types.html#generators

[oblique urchin]

https://peps.python.org/pep-0589/#type-consistency

"First, any TypedDict type is consistent with Mapping[str, object]. "

[runic tapir]

oh, I see

[north widget]

so completed_threads: generator[concurrent.futures._base.Future] = concurrent.futures.as_completed(future_to_url) is correct? but it returns error in the IDE

[runic tapir]

no, that's not even remotely close to correct. Did you check my link?

[runic tapir]

You either want to from types import Iterator and type it as Iterator[concurrent.futures.Future], or you want to from types import Generator, and type it as Generator[concurrent.futures.Future, None, None]

[nova venture]

ah, I thought it would be inferred as Mapping[str, int | str] instead. I don't understand the PEP too much here, so why is the inference more lax (inferring object) compared to inferring V as the union of all instance field annotations?

[runic tapir]

and you should never be typing something as a private type of some library, if you can avoid it.

[north widget]

how to avoid typing? by own judgment?

[oblique urchin]

I think it's because TypedDict types can have arbitrary additional keys (due to inheritance)

[hearty shell]

they give an example in the pep

class A(TypedDict):
    x: int

class B(TypedDict):
    x: int
    y: str

def sum_values(m: Mapping[str, int]) -> int:
    n = 0
    for v in m.values():
        n += v  # Runtime error
    return n

def f(a: A) -> None:
    sum_values(a)  # Error: 'A' incompatible with Mapping[str, int]

b: B = {'x': 0, 'y': 'foo'}
f(b)

[acoustic thicket]

shouldn't using @final work then?

[runic tapir]

If MyDict (and every subtype of it) were treated as Mapping[str, str], then it would be an error to do ```py
class MyDictSubtype(MyDict):
c: int
consume_type(MyDictSubtype)

which violates liskov substitutability. If your function works on a type, it needs to also work on subtypes of that type

[nova venture]

do you think it's worth filing a feature request on pyright to make @final TypedDict get inferred with union of value annotations instead of object? or would that require a pep

[hearty shell]

I think it should but support for that was iffy, there was some discussion around that

[runic tapir]

you could bring up the topic on https://mail.python.org/archives/list/[e-mail]/ - I think that's where most of these conversations happen

[nova venture]

this actually becomes possible if @final TypedDicts were inferred more strictly, and things such as being deseriazable would be possible to express type hint time

[oblique urchin]

I think this is something type checkers could do on their own. pyright's whole treatment of @rustic gull TypedDicts is homegrown

[runic tapir]

the PEPs for how type hints should be treated are frustratingly vague in a lot of places.

which means a lot of Python type checkers do things in their own subtly incompatible ways.

[north widget]

Reading PEP483. Why this script would cause error? Thank you

[oblique urchin]

you can't left-shift a float

[north widget]

How does the second condition of sub-typing: every function from first_type is also in the sets of functions of second_type helps the script runs better?

[oblique urchin]

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for <<: 'float' and 'int'

[north widget]

thank you. understand a lot of after that.
so the compiler would check subtype relationship in each operations?

In this script, it doesn't return error, but got a weird return None

[oblique urchin]

the Python compiler ignores type annotations

you need to run a static type checker (mypy, pyright) to get annotations checked

the None in your example is just because that's what append_pi returns

[north widget]

this script intuitvely should not return None doesn't it? and it does not return error either @oblique urchin

[oblique urchin]

you're printing the return value of append_pi, and append_pi returns None

there's no error because the Python runtime is happy to let you put whatever you want in a list

[north widget]

lol. i saw my problem. thank you

[north widget]

In the same example, the type of the my_list: list[int] should be changed to be list[float] after appending a float number into it?

I don't see why PEP483 said, append_pi(my_list) # Naively, this should be safe...?

my_list = [1,2,3]
# Here we "mistakely" consider int is a subclass of flaot. i.e. all int are also float.
my_list += [3.14]
# After that, the list become list[float].
my_list[-1] << 5 
# of course this would fails.

[hearty shell]

.

This is pretty much why

What specifically are you unsure about?

[north widget]

what's the meaning of safe? I mean, if you consider my_list is in type list[float], you would get the error in the final line

[hearty shell]

but it is not

[north widget]

wont*

[hearty shell]

it is List[int]

that is the problem

[north widget]

int is a subclass of float, and after appending a float value into a list[int] , it is still a list[int]?

[hearty shell]

Well, no, but how is they type checker suppose to know, if you had a list of ints, you passed it to a function to do its own thing and suddenly your list now has a float

[north widget]

Ah, one more point. Python is a dynamic type language so the list should be list[float] after appending a float

[oblique urchin]

I think you're fixating too much on the append. The example is trying to show that naively, you'd think it's safe to pass a List[int] to a function accepting List[float], because int is a subtype of float, but it's not

[hearty shell]

But it is, in fact (if int was an actual subclass of float) when you ask python if that is in fact a list of floats it would tell you yes, but then that is one of the things that static typing is good for

How are you suppose to keep track of what type things are if it is just changes indirectly through mutations

[north widget]

I am still not quite understand what this example of PEP483 is showing, but I still learn a lot during the discussion on this example. So I am happy with that anyways

[north widget]

I would add a type hint func... -> list[float] on def append_pi()?

so that the reader knows the list would be list[float]

and they suppose not to left shifting the list after it

[oblique urchin]

The type system doesn't have a way to express that calling a function changes the type of one of its arguments

And it's not so much about "the reader knows", but about what a static checker accepts. The power of type annotations is that the static checker will check that the annotations are correct and tell you if they're not.

[north widget]

PEP483: what is the meaning of UserID and int are in the same type?

He meant duck-typing?

[north widget]

what is "type system"? so I don't know to express that calling a function changes the type of one of its arguments

[oblique urchin]

It's talking specifically about structural typing (i.e., Protocols, but those were added to the type system later)

[oblique urchin]

The type system is the set of rules and objects specified in PEP 484 and followup PEPs

[trim tangle]

Poor final 😔

[oblique urchin]

mypy flat out doesn't allow @final on a TypedDict

[hearty shell]

What is the reason behind not having an __add__ method on sequences?

Came across that while trying to answer an earlier question, I had no idea x)

from typing import Sequence
from operator import concat

def foo() -> Sequence[int]: ...

a: Sequence[int] = foo()
b: Sequence[int] = foo()

c = a + b  # Not Fine
d = concat(a, b)  # Fine

[oblique urchin]

https://github.com/python/typeshed/issues/6985

[hearty shell]

Yeah I had just found it, my bad, it makes sense right because not every sequence...

Although it does mean that concat is itself not well defined

a = range(5)
b = range(6)
c = concat(a, b) # Fine

although on a separate note, intuitively adding ranges should chain them imo, but that probably does have a million issues that I cant think of right now xD

[acoustic thicket]

thatd only make sense if a.step == b.step and a.stop == b.start which i don't think is a very common scenario

[hearty shell]

The step and stop doesnt need to be the same

!e

from itertools import chain

print(list(chain(range(1, 10, 3), range(20, 30))))

[rough sluice]

@hearty shell :white_check_mark: Your eval job has completed with return code 0.

[1, 4, 7, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]

[acoustic thicket]

ah chain in that way

thought you meant range + range = range

[hearty shell]

I mean that is what I am saying range + range = range could mean

although this could become rather ad hoc, "adding" ranges would be fine, but how would you type the operator, I guess iterable would be fine but then you would be making stuff into generators implicitly

[acoustic thicket]

\🤔

[hearty shell]

Actually nvm, as noted in the github issue, you can't add tuple and list so there is no reason why you would have to make range + not_range_but_iterable allowed either

[north widget]

Hello. VS code "go to definition" brings me here

VS code "go to declaration" brings me here

how is this .pyi generated? thank you

[trim tangle]

A *.pyi file is a "stub file". It just describes the types of symbols in a module

Very often they're not generated but hand-written

This particular file can be found somewhere in typeshed: https://github.com/python/typeshed

or maybe in some of Pylance's own stubs

[buoyant swift]

I believe @solid sleet has a nice library for that

[solid sleet]

oh, it's for real continuous ranges, which aren't even sequences!

[terse sky]

IMHO adding any two iterables should chain them, I always get annoyed at how much of a mouthful chain is in python

and chain.from_iterable should really be called flatten, also such a mouthful

but I think making these things work in python is annoying, you'd need to "first class" it into the language

[oblique urchin]

You can do [*a, *b] to chain any two iterables, but that does turn them into a concrete list

(or tuple or set with the equivalent syntax)

[soft matrix]

Jelle is this new overload introspection for pyanalyze?

[oblique urchin]

It will be useful for pyanalyze yes

I also led the beartype guy know about it and he seemed happy

Right now pyanalyze monkeypatches typing.overload so it can support runtime overloads

[soft matrix]

Oh right cool

[twilit badge]

How could I tell the type checker that I've added some extra method to my class? I was thinking of a generic protocol, that could somehow inherit everything from the class it's generic over, but I've no idea how to actually implement this. Is something like this even possible?

For example: ```py
class MyCls:
def foo(self) -> int:
...

class MyExtendedProto(Generic[T]):
# Should inherit everything from T, but additionaly define:

def custom_method(self) -> bool:
    ...

def add_method(klass: T) -> MyExtendedProto[T]:
MyCls.custom_method = lambda self: True
return MyCls

MyExtendedCls = add_method(MyCls)
reveal_type(MyExtendedCls.foo) # should be (self) -> int
reveal_type(MyExtendedCls.custom_method) # should be (self) -> bool

[soft matrix]

Not without an intersection type no

[twilit badge]

yeah, I figured it probably wouldn't be possible

[twilit badge]

are there some plans on adding that already?

[soft matrix]

i think they stalled

[fiery obsidian]

There is a way to tell that a function returns the type of one of the element given in the arguments? like:

def test(a, b)->type(b)
  return b

[hearty shell]

from typing import TypeVar, Any

T = TypeVar('T')

def test(a: Any, b: T) -> T:
    return b

[fiery obsidian]

type-hints now are correct but i have anyway a function that tells me unexpected argument giving that variable returned from that function :L i think that i'll use type: ignore

[hearty shell]

Could you show the function giving the error?

[fiery obsidian]

def next_turn(self, player: 'Player'):
  """Set the turn next to the given player"""
  if not self.started:
    raise GameError(message="Game not started")
  self.player_turn = next_in_list(self.players_in_game, player)

#calling next_turn method:
self.big_blind_player = next_in_list(self.players_in_game, self.small_blind_player)
self.next_turn(self, self.big_blind_player) #Unexpected argument: self.big_blind_player

#next in list:
def next_in_list(iterable_list: List[T], item: T)-> T:
    return iterable_list[(iterable_list.index(item) + 1)] if iterable_list.index(item) < len(iterable_list) - 1 else iterable_list[0]

[hearty shell]

self.next_turn(self, self.big_blind_player) Where are you calling this from? Why are you passing self?

[fiery obsidian]

from a class named Game

[fiery obsidian]

weeeeell i'm stupid

thanks

[hearty shell]

Type checking saves the day once again

[fiery obsidian]

hahahah exactly

[hearty shell]

On a separate note

this is interesting

pyright:

from typing import TypeVar

T = TypeVar('T')

def id1(a: T) -> T:
    return a
    
def id2(a):
    return a

reveal_type(id1(5))  # Type of "id1(5)" is "int"
reveal_type(id2(5))  # Type of "id2(5)" is "Literal[5]"

Actually, even more
pyright:

a: Literal[5] = 5
reveal_type(id1(a))  # Type of "id1(a)" is "int"

mypy:

a: Literal[5] = 5
reveal_type(id1(a))  # Revealed type is "Literal[5]"

🤔

[trim tangle]

Yep, it does this analysis but on a very shallow level

[hearty shell]

Is it not counter intuitive though, that not providing an annotation results in better type inference?

And that pyright, which usually infers very hard, cannot do Literal[5] -> Literal[5] while mypy can

[terse sky]

Well, literals are pretty unintuitive in python to start with

[blazing nest]

How would you do literals if you could go back to before they were created?

[terse sky]

not really sure if I'd have them exist

I'm not sure if they have any role other than allowing you to annotate existing code, and maybe that's not nearly a good enough reason, idk

[covert dagger]

hmm https://mypy-play.net/?mypy=latest&python=3.10&gist=0395333c3acd13b6909552de90d64058

[hearty shell]

They allow you to have "enums" without having enums, what is not to like about them? Apart from restricting themselves to well, literals only...

[hearty shell]

🤔

[covert dagger]

Literal[5] -> Literal[5] breaks down here, maybe that's why pyright widens the Literal for TypeVars

[terse sky]

I mean, why use them instead of using an enum in new code?

There is no really good reason

which is probably why almost no languages have them

And they add complexity because the static type system now potentially needs to be able to answer questions about when it knows a value, statically

which is what all the above examples are showing

comparatively, enums are very very very simple, you either construct an enum by referencing a constant directly or by using an explicit function

[covert dagger]

they'll be useful in new code with the shape typing stuff, that PEP 646 used as motivation