#type-hinting

1 messages ยท Page 17 of 1

cyan glen
#

Can we not use Optional?

oblique urchin
cyan glen
#

Oh ok, so just type hint like normal. Withoit considering sometimes it can be empty?

young zealot
oblique urchin
young zealot
#

ouch, ok. I guess in this case I should switch to using something like an abstract class instead of protocol?

oblique urchin
young zealot
#

hmm, well then I guess I could make a non-abstract base class and just make the functions raise NotImplementedError?

heady flicker
#

Is there a way to type hint that my decorator also acts like a staticmethod decorator? Specifically to be recognized by pyright.

heady flicker
lunar dune
heady flicker
#

Huh, didn't know that. Cool.

#

Alex, maybe you know the solution to my issue too? :D

lunar dune
#

Show me what your decorator looks like now? ๐Ÿ™‚

heady flicker
#

Okie

#

One sec

lunar dune
#

Worst comes to few worst, you could always do the ol'

if TYPE_CHECKING:
    from builtins import staticmethod as my_awesome_decorator
else:
    def my_awesome_decorator(func):
       ...

hack

heady flicker
#

This is a rough typing-only draft of the thing I'm working on. Essentially I want to say that my decorator applies staticmethod to the function and then returns a class that has a similar signature to that function.


class AlterResponseInstruction:
    def __call__(self, data: dict[str, Any]) -> None:
        ...


def alter_response(
    model: type[BaseModel], *changes: Change
) -> Callable[[Callable[[dict[str, Any]], None]], AlterResponseInstruction]:
    ...

class CollapseEventRequest:
    @alter_response(
        EventAPIResource,
        Change("request", type_old=str, type_new=uuid.UUID),
    )
    def alter_event_api_response(data: dict[str, Any]):
        data["request"] = data["request"]["id"]
#

I don't want my users to have access to self on any of these "alter" methods but I also don't want them to write staticmethod manually every time

#

I guess this decorator is applied at import time so I could just check the signature and raise the error but type hint it as staticmethod because the users are not supposed to call this method anyways but I really wanted to type hint it nicely :(

lunar dune
#

It returns an instance of AlterResponseInstruction, not the AlterResponseInstruction class itself, right?

heady flicker
#

Yup

lunar dune
#

Adding a __get__ method (inside an if TYPE_CHECKING block if necessary) to the AlterResponseInstruction class might help

rough sluiceBOT
#

stdlib/builtins.pyi line 135

def __get__(self, __instance: _T, __owner: type[_T] | None = None) -> Callable[_P, _R_co]: ...```
lunar dune
#

Not sure though; I'm on my phone and don't have pyright handy to experiment with ๐Ÿ™‚

heady flicker
#

It doesn't because the error happens at the definition of alter_event_api_response. Tried just now.

I.e. It indicates that we should use a different type for "data" argument.

#

Feels like this behavior is hardcoded into pyright.

strange tree
#

Hi all, I was looking at the threading module and thinking, why is Thread not using ParamSpec to type check that the callable arguments are passed?

Something like this (simplified on purpose)

P = ParamSpec("P")
T = TypeVar("T")

class Thread:
    def __init__(target: Callable[P, T], args: P.args, kwargs: P.kwargs): ...
slender timber
#

can i use kwargs in Annotated?

trim tangle
#

but slicing notation doesn't support keywords

oblique urchin
#

!pep 637

rough sluiceBOT
#
**PEP 637 - Support for indexing with keyword arguments**

Link

Status

Rejected

Python-Version

3.10

Created

24-Aug-2020

Type

Standards Track

oblique urchin
#

(rejected)

trim tangle
#

relatable ๐Ÿ˜”

slender timber
#

why tf does everything keep getting rejected

#

anyways i'd be looking into dataclass_transform

#

Make the field() function do somethings

lunar dune
soft matrix
#

Wouldn't target need to be pos only?

lunar dune
trim tangle
#

I mean, if we ignore kwargs

#

๐Ÿ’€

#

Conclusion: just use partial or a lambda

lunar dune
jade viper
#

What is the correct way to tyoe hint any kind of number? (int, float, decimal.Decimal, np.int32, np.int64 and so on)

lunar dune
#

It depends on what you want to do with that number

jade viper
#

It's gonna be written into a value in a dict which in turn is gonna be transformed to a CSV file

#

No further operations on it

trim tangle
#

not sure about mypy but maybe too

#

for kwargs support, I think

lunar dune
trim tangle
jade viper
#

Is it right?

lunar dune
#

typing.SupportsFloat is a pretty good approximation for most numeric types

#

But excludes complex, which doesn't have a __float__ method

jade viper
#

Complex shouldn't be allowed in this case

#

Thanks guys:)

lunar dune
jade viper
#

lol, all real numeric types

#

my bad

#

Since I'm dealing with finances complex numbers didn't even come to mind honestly haha

trim tangle
#

I think they do have some application in economics

lunar dune
# jade viper Since I'm dealing with finances complex numbers didn't even come to mind honestl...

This is partly why I was asking what you wanted to do with them. The spectrum of "numeric types" in Python is so broad, that there's loads and loads of ways in which they can interoperate badly, so you have to think specifically about how your "numbers" in your specific use case need to be used in order to know how to annotate them.

Need to call float() on them before adding them together? Use typing.SupportsFloat. Need to call abs() on them? Use SupportsAbs. Etc. etc.

The problem with numbers.Number is it's an excessive generalisation that doesn't actually tell you anything useful about the type you're working with -- this is [one of several reasons] why type checkers don't support it.

jade viper
lunar dune
trim tangle
#

accept Decimal and propagate the headache to the caller

jade viper
trim tangle
#

less polymorphism = often less headache ๐Ÿ™‚

jade viper
trim tangle
#

a concrete type is definitely less mental straining than INumberatable

jade viper
#

lol

#

While I'm here, what is the correct way to type hint a str that MUST only be three upper case characters?

trim tangle
#

a huge literal

jade viper
lunar dune
trim tangle
#

lies

jade viper
#

Anyway, thank you everyone! ๐Ÿ™‚

trim tangle
#

at least I would consider a 3-tuple

lunar dune
trim tangle
#

I love template string types in TypeScript

jade viper
#

Is there a way to type hint a str that must be only characters at least?

trim tangle
trim tangle
void panther
jade viper
trim tangle
#

@jade viper here you go

jade viper
#

LOL

celest iron
trim tangle
#

pyright appreciation moment

lunar dune
trim tangle
#

I think I should put some limits... otherwise some smartass will eat all my cloud money

#

that smartass also might be me

jade viper
#

Also, another curiosity question

#

Would Pydantic work with stuff like NewType?

lunar dune
celest iron
jade viper
#

lol fair enough ๐Ÿ˜†

#

Again, thank you everyone!:)

#

Have a great week

trim tangle
#

I only run mypy on friday because of how long it takes.
You could say my code is weekly typed

celest iron
trim tangle
#

like partial mentioned above

#

also NamedTuple, Enums and such

upper flame
#

Just merged last big PR for our monorepo migration from mypy to pyright. ๐Ÿ˜Ž
Lots of respect and kudos to mypy team. Pyright seems like a better fit for our org (we did some extensive research).
Type checks are much faster now ๐ŸŒž

heady flicker
#

@upper flame Would you kindly explain why you picked pyright instead?

slender timber
# 
# Protocol
@runtime_checkable
class SupportsGet(Protocol[_T_co]):
    """Protocol for a read-only descriptor."""
    @overload
    def __get__(self, obj: None, objtype: Any = None) -> Self: ...
    @overload
    def __get__(self, obj: EventProxy, objtype: Any = None) -> _T_co | None: ...

@runtime_checkable
class SupportsSet(Protocol[_T_contra]):
    """Protocol for a read-write descriptor."""
    def __set__(self, obj: EventProxy, value: _T_contra) -> None: ...

# Impl
class PropBase(abc.ABC, SupportsGet[_T_co], SupportsSet[_T_contra]):
    def __get__(self, obj: EventProxy | None, objtype: Any = None) -> _T_co | Self | None: ...

results in:

Method "__get__" overrides class "SupportsGet" in an incompatible manner
  Return type mismatch: base method returns type "Self@SupportsGet[_T_co@SupportsGet]", override returns type "_T_co@PropBase | Self@PropBase[_T_co@PropBase, _T_contra@PropBase] | None"
    Type "object* | PropBase[_T_co@PropBase, _T_contra@PropBase] | None" cannot be assigned to type "SupportsGet[_T_co@SupportsGet]"
      "object*" is incompatible with protocol "SupportsGet[_T_co@SupportsGet]"
        "__get__" is not present
heady flicker
#

And pyright is right here. You added another item in the union: Self

slender timber
#

i don't understand

heady flicker
#

Oh wait, I misread the code. I get it now

#

Feels like you'll need to provide overloads on the implementation too.

slender timber
#

why?

heady flicker
#

Because the implementation is not necessarily compatible with these kinds of overloads

slender timber
#

it should be

# 
class PropBase(abc.ABC, _SupportsGet[_T_co], _SupportsSet[_T_contra]):
    def __init__(self, *ids: EventEnum, default: _T_co | None = None, readonly: bool = False):
        self._ids = ids
        self._default = default
        self._readonly = readonly

    def _get_event(self, ins: EventProxy) -> AnyEvent | None:
        for id in self._ids:
            if id in ins.events:
                return ins.events.first(id)

    @property
    def default(self) -> _T_co | None:  # Configure version based defaults here
        return self._default

    @abc.abstractmethod
    def _get(self, event: AnyEvent) -> _T_co | None: ...

    @final
    def __get__(self, obj: EventProxy | None, objtype: Any = None) -> _T_co | Self | None:
        if obj is None:
            return self

        event = self._get_event(obj)
        if event is not None:
            return self._get(event)

        return self.default
#

shouldn't this be a bug in typecheckers?

brazen jolt
#

Why doesn't functools.lru_cache preserve function parameters? That's so annoying...

image.png
soft matrix
#

because pep 612 didnt give a way to do it

brazen jolt
#

hmm, I see, thanks

slender timber
#

how do I concatenate two types (not Union, that's an OR, I want an AND)

soft matrix
#

depends the answer is probably you cant

#

the only way to currently do it is to subclass

slender timber
#

fuck me, TIL I could use descriptor fields in dataclasses

slender timber
# 
class _SupportsGetSetItem(Protocol[_KT_contra, _KV]):
    def __getitem__(self, key: _KT_contra) -> _KV: ...
    def __setitem__(self, key: _KT_contra, value: _KV) -> None: ...

why is this protocol not compatible with list and dict both?

oblique urchin
slender timber
# oblique urchin it should be. what are you seeing exactly?

oki i have this code

class _SupportsGetSetItem(Protocol[_KT_contra, _KV]):
    def __getitem__(self, key: _KT_contra) -> _KV: ...
    def __setitem__(self, key: _KT_contra, value: _KV) -> None: ...

class _ValidationMixin(_SupportsGetSetItem[_T, Any]):
    subcons: _SupportsGetSetItem[_T, c.Construct[Any, Any]]

    def __getitem__(self, key: _T) -> Any:
        child = super().__getitem__(key)
        childcon = self.subcons[key]

        if TYPE_CHECKING:
            assert isinstance(childcon, c.Struct)

        if isinstance(child, dict):
            return _ValidatingDict(childcon, **child)
        elif isinstance(child, list):
            return _ValidatingList(childcon, *child)
        return child

    def __setitem__(self, key: _T, value: Any) -> None:
        self.subcons[key].build(value)
        super().__setitem__(key, value)

class _ValidatingDict(StrDict, _ValidationMixin[str]):
    def __init__(self, struct: c.Struct[Any, Any], **kwds: Any) -> None:
        self.subcons = struct._subcons  # Dict[str, c.Construct[Any, Any]]  --- ERROR
        super().__init__(**kwds)

class _ValidatingList(AnyList, _ValidationMixin[SupportsIndex]):
    def __init__(self, struct: c.Struct[Any, Any], *args: Any) -> None:
        self.subcons = struct.subcons   # List[c.Construct[Any, Any]] --- ERROR
        super().__init__(*args)
#

it says

Cannot assign member "subcons" for type "_ValidatingDict"
  "Dict[str, Construct[Any, Any]]" is incompatible with protocol "_SupportsGetSetItem[str, Construct[Any, Any]]"
    "__getitem__" is an incompatible type
      Type "(__key: str, /) -> Construct[Any, Any]" cannot be assigned to type "(key: _KT_contra@_SupportsGetSetItem) -> _KV@_SupportsGetSetItem"
        Position-only parameter mismatch; expected 1 but received 0
    "__setitem__" is an incompatible type
      Type "(__key: str, __value: Construct[Any, Any], /) -> None" cannot be assigned to type "(key: _KT_contra@_SupportsGetSetItem, value: _KV@_SupportsGetSetItem) -> None"
        Position-only parameter mismatch; expected 2 but received

and

Cannot assign member "subcons" for type "_ValidatingList"
  "List[Construct[Any, Any]]" is incompatible with protocol "_SupportsGetSetItem[SupportsIndex, Construct[Any, Any]]"
    "__getitem__" is an incompatible type
      No overloaded function matches type "(key: _KT_contra@_SupportsGetSetItem) -> _KV@_SupportsGetSetItem"
    "__setitem__" is an incompatible type
      No overloaded function matches type "(key: _KT_contra@_SupportsGetSetItem, value: _KV@_SupportsGetSetItem) -> None"
oblique urchin
#

oh you need to make the params positional-only in your protocol

#

sorry I missed that

slender timber
#

is that a new thing?

#

do i need to do it always?

oblique urchin
#

I think pyright has always worked that way

#

the argument kind is part of the type

slender timber
#

wow it worked

oblique urchin
#

the protocol says you can do _SupportsGetSetItem().__getitem__(key=42)

#

and list doesn't let you do that

slender timber
#

yea ryt

#

is my above error because of the same thing?

oblique urchin
#

sounds like it might be, your protocols don't have pos-only parameters there

#

oh but there's also an object, that sounds like mypy incorrectly inferring a type

slender timber
#

it is pyright actually

upper flame
brisk hedge
#

I agree, the faster the feedback loop, the more eager devs are to run tests / static analysis early and often

trim tangle
#

For me the killer feature in pyright is the language server

#

Hints on hover, autocomplete and so on

heady flicker
#

Can relate

rustic gull
#

Is it possible to type-hint a function in a way that tells the type checker that code doesn't proceed after the function is called?

Like how would the end function be annotated here for example?
(the type checker still tells me that None is not subscriptable even though it can't be None)

import sys
def end():
    sys.exit()

def func(l: list | None):
    if l is None:
        end()
    return l[4]
heady flicker
#

Never or Noreturn

wicked scarab
#

Hello, I'm trying to make a different return value depending on it's child. I've tried this:

class Base:
    def __call__(self, *args, **kwargs):
        subclasses = {
            Foo: "a",
            Bar: "b"
        }
        return subclasses[self.__class__]

class Foo(Base): ...
class Bar(Base): ...

But Pyright complains:

Argument of type "Type[Base]" cannot be assigned to parameter "__key" of type "Type[Foo] | Type[Bar]" in function "__getitem__"
  Type "Type[Base]" cannot be assigned to type "Type[Foo] | Type[Bar]"
    "Type[Base]" is incompatible with "Type[Foo]"
    Type "Type[Base]" cannot be assigned to type "Type[Foo]"
    "Type[Base]" is incompatible with "Type[Bar]"
    Type "Type[Base]" cannot be assigned to type "Type[Bar]"
```  Any Idea?
soft matrix
# 
class Base:
    @overload
    def __call__(self: Foo, *args, **kwargs) -> Literal["a"]:
    @overload
    def __call__(self: Bar, *args, **kwargs) -> Literal["b"]:
    def __call__(self, *args, **kwargs):
        subclasses = {
            Foo: "a",
            Bar: "b"
        }
        return subclasses[self.__class__]

class Foo(Base): ...
class Bar(Base): ...```this is dumb but will work
#

not really sure why you cant just override call in the subclass

wicked scarab
#

that's kinda messy

soft matrix
#

thats one of the main draws of subclassing, it allows for separation of logic lol

wicked scarab
soft matrix
#

consider using just normal methods then that allow you to take out bits

wicked scarab
#

basically I'm using lambda for each class

        subclasses = {
            Equals: lambda first, second: kwargs["first"] == kwargs["second"],
            NotEquals: lambda first, second: kwargs["first"] != kwargs["second"],
            LowerThan: lambda first, second: kwargs["first"] < kwargs["second"],
            GreaterThan: lambda first, second: kwargs["first"] > kwargs["second"],
            # and some more here
        }

If I don't use this, then I would need to make __call__ function on each subclass

soft matrix
#

you can definitely write a higher order function that does this for you

#

and you can just do something like __call__ = this_higher_order_function(operator.eq) inside the body of the class

young zealot
#

Is there a way to "replace" a superclasses' method on a subclass so that mypy doesn't complain about mismatching method signature?

solemn sapphire
#

@override?

oblique urchin
oblique urchin
#

# type: ignore is correct because an incompatible override is type unsafe, so you should explicitly mark it as being unsafe

solemn sapphire
#

ah, your right

young zealot
oblique urchin
#

if you have a function def f(a: A): a.meth() and a class B(A): def meth(self, some_arg): pass

#

then f(B()) will fail at runtime

solemn sapphire
#

what if A is an abstract class

young zealot
oblique urchin
solemn sapphire
#

ye, now that I think about it, it does seem like a weird thing to do

young zealot
#

yeah in my case I'm "clobbering" some classes from a library and exposing my own versions to make things cleaner/better/more convenient, so the "base" class is not meant to be exposed so it should be "safe" in this case, so i'll just use # type: ignore[override]

soft matrix
#

sounds like you shouldnt use inheritance here at all in this case

#

should definitely be composition

young zealot
#

there are other complications with that, where at the moment I believe this is the best way

soft matrix
#

well if everything is incompatible then passing it to any method that takes its superclass is problematic

young zealot
#

yeah in this case its not a problem because the superclass is not exposed, only my version, and everything that is exposed uses my version

cosmic cave
#

What is the most ergonomic way to statically validate that a class implements a Protocol? Ideally I'd like this validation to happen close to the class keyword

soft matrix
#

just subclass the protocol then your type checker should let you know

cosmic cave
#

unfortunately that is not possible due to the reasons we discussed here: https://discord.com/channels/267624335836053506/1119407419471171676
Declaring a read-only @property on the protocol prevents subclasses from having a writable attribute with that name
And yet when I do validation like the following hack, it proves that typecheckers do consider the class to be compatible with the protocol:

__typecheck: MyProtocol = cast(MyClass, None)
#

Is the dummy assignment here the most ergonomic approach?
One problem is it needs to be wrapped in quotes if declared near the class keyword:

__typecheck: MyProtocol = cast('MyClass', None)
class MyClass:
soft matrix
#

does it have to be an instance var?

#

could just do a settable property proxying the attribute

cosmic cave
#

has to be an instance var, yeah

#

the goal is to implement a protocol that says "object must gave a gettable attribute named bar that will give you an int"
Then to write classes that implement that protocol, with static verification that they do. So that any drift between protocol and implementing class are caught statically

soft matrix
#

well im not sure then you might just be stuffed

cosmic cave
#

I'll stick to this trick, seems like the best option. __typecheck: MyProtocol = cast('MyClass', None)
The forward reference issue with type declarations continues to be a real sore point in python typechecking

soft matrix
#

https://discuss.python.org/t/cast-syntax-for-static-typing/25174 :)

Discussions on Python.org
Cast syntax for static typing

Currently to cast from one type to another in typing you have to use the function typing.cast which requires a runtime cost and is not very pleasant to use. i propose a built-in syntax for this: a as Type this re-uses the as keyword from imports and with statements, this is better over the existing typing.cast because it can be a no-op at runt...

cosmic cave
#

Yeah coming from TypeScript, IMO python really needs better syntax for zero-cost typing

mint inlet
tranquil turtle
#

there is typing.assert_type
so you can do ```py
typing.assert_type(MyClass(), MyProto)

#

or typing.assert_type(typing.cast(MyClass, None), MyProto) (i case MyClass doesnt support creation without arguments)

oblique urchin
tranquil turtle
#

๐Ÿ˜ญ

cosmic cave
#

Also, constructing the class is often more complex than passing zero params

#

Even if it is a hack solely for the typechecker, typechecker expects params to be passed.

#

Wrapping in if TYPE_CHECKING also doesn't help because the typecheckers will raise an error.

They're correct to complain, because subclassing from a protocol inherits the protocol's methods and properties

mint inlet
#

ah alright

cosmic cave
#

Typescript has a way to declare read-only via annotations, it'd be nice if python had similar

interface Foo {
  readonly bar: number;
}
tranquil turtle
#

python has @property

#

if property was generic: ```py
class Foo:
x: property[int]

oblique urchin
cosmic cave
#

Are you saying that it is currently being discussed? If so, is there somewhere to read about that ongoing discussion?

oblique urchin
#

!pep 705

rough sluiceBOT
#
**PEP 705 - TypedMapping: Type Hints for Mappings with a Fixed Set of Keys**

Link

Status

Draft

Python-Version

3.12

Created

07-Nov-2022

Type

Standards Track

oblique urchin
cosmic cave
#

Something like foo: ReadOnly[int]?

oblique urchin
cosmic cave
#

Oh and the discussion says that pyright already supports it, that's cool

#

Is this only for typeddict? Some of the discussion makes it seem like it's specific to typeddict, can't be used on normal classes and protocols

oblique urchin
cosmic cave
#

Oh so it's actually not at all relevant to my protocol issue?

hallow flint
#

yes, i use dummy assignments for this kind of thing. the cast seems like a reasonable additional trick

#

(and i don't know why people worry about the cost of cast, it's very very cheap. esp if the annotation is a string)

slender timber
#

there's also Final for readonly variables from a type checker POV

daring marsh
#

anyone knows why 'operation' variable is like that?

image.png
viscid spire
#

which means you assign to it but don't use it somewhere else

scenic verge
cosmic cave
upper flame
#

Hi friends! I have this toy example of a real thing I need to type...
I don't see what's wrong with it.

from typing import Any, Callable, TypeVar,Generic
from typing_extensions import ParamSpec

P = ParamSpec("P")
R = TypeVar("R")

class Future(Generic[P, R]):
    def __init__(self, f: Callable[P, R], *args: P.args, **kwargs: P.kwargs) -> None:
        self.f = f
        self.args = args
        self.kwargs = kwargs
    
    def get(self) -> R:
        return self.f(*self.args, **self.kwargs)
    
    def resolve(self) -> Any:
        res = self.get()
        while isinstance(res, Future):
            res = res.get()
        return res

def transformer(func: Callable[P, R]) -> Callable[P, Future[P, R]]:
    def wrapper(*args: P.args, **kwargs: P.kwargs):
        return Future(func, *args, **kwargs)
    return wrapper

@transformer
def b() -> int:
    return 1

@transformer
def a() -> int:
    return b() # type error on this line: main.py:33: error: Incompatible return value type (got "Future[[], int]", expected "int")  [return-value]

print(a().resolve())
#

Both mypy and pyright report the same error...

#

It works just fine in runtime

trim tangle
#

but you're saying that a() returns an integer

#

what are you trying to do?

upper flame
#

Right, but would not

@transformer
def a() -> int:

applying the transformer decorator should change the return type of a as well, should not it?

soft matrix
#

it does change the return type

upper flame
#

Oh we have this complex system with decorators, I'm trying to improve VSCode expirience to avoid red squiggless everywhere

soft matrix
#

your issue is that you are returning that return from a

upper flame
#

I guess to make things easier visually here is another example

@transformer
def sum(x: int, y: int) -> int:
    return x + y

@transformer
def a(x: int, y: int) -> int:
    return sum(x, y)

print(a(2, 3).resolve())
trim tangle
# upper flame Right, but would not ```python @transformer def a() -> int: ``` applying the tra...

Decorators are syntax sugar for a function call: ```py
def transformer(func: Callable[P, R]) -> Callable[P, Future[P, R]]:
...

def pre_b() -> int:
return 1

pre_b is Callable[[], int]

b = transformer(pre_b) # b is Callable[[], Future[[], int]]

def pre_a() -> int:
result_of_b = b() # is a future
return result_of_b
# ^ here you're returning a Future[[], int]. But the function signature says you're returning an int!

a = transformer(pre_a)

upper flame
#

ok groking on this....

#

ok would it be fair to say that the return type of the function inside pre_a should MATCH the actual return type? But THEN decorator is applied on top of it (because it's like the outer call on the function)

#

The b = transformer(pre_b) outer call

trim tangle
#

The return type of the pre_a function only refers to that particular function. The decorator does not impact that

upper flame
#

got it, I think I follow

#

is there a way to type it?

trim tangle
# upper flame got it, I think I follow

An example from the stdlib would be contextlib.contextmanager

from collections.abc import Iterator
from typing import IO
from contextlib import contextmanager

@contextmanager
def foo() -> Iterator[IO[str]]:
    with open("file.txt", encoding="utf-8") as file:
        print("Opened file!")
        yield file
        print("Closing file!")

reveal_type(foo)

the inner function returns an Iterator while the type of foo for an outsider is something like ContextManager[IO[str]]
https://pyright-playground.decorator-factory.su/?gzip=H4sIAIE-j2QC_12PzQ6CMBCE732KtaeSiGdjojHxxIkHIMaUssUmpUvK-vf2FhEOzHG-neyMjdSBIe_RsKMw7HRtwHU9RYaCMWqmKOx4xJ_ehXZh5eQaCoxv9q6eyd_pdNAtRiHOK6NBC5ZIZZCflhdVUVYDx-v1ICDp5fgO1GNQ0jqPO36z3AIGQ03qcJQPtvleZqAHGPkUGtVHF1jJMkWx-bGNzBb6cegndx24eBrGdXNCRHyi9rc0GlVqm4kvDZc6dSkBAAA%3D

trim tangle
# upper flame is there a way to type it?

In your case what you really have is: the function under the transformer can return either the result R or a Future[P, R]
So something like ```py
def transformer(func: Callable[P, R | Future[P, R]]) -> Callable[P, Future[P, R]]:

#

not sure if it will work tbh

#

also I hope you don't have spin-looping futures in actual code ๐Ÿ˜›

upper flame
#

Cool, let me try something with overloads...

#

and a union

#

oh actually no overloads needed?

#

spin-looping futures in actual code
the system is similar to https://www.sematic.dev/ -- it runs every transformer on a different k8s pod

#

And resolve lives also in a separate resolver which orchestrates the execution (and sometimes it spin-loop I think, but I didn't look into this part)

trim tangle
#

If you want to do asynchronous I/O with futures, there's already stuff for that in the standard library ๐Ÿ™‚

#

though I don't really understand what you're doing so

upper flame
#

yeah it doesn't matter that much honestly ๐Ÿ™‚

#

Thank you for the help!

#

ok I kind of suspecting that this is not typable...

#

Tried playing around with

@overload
def transformer(func: Callable[P, R]) -> Callable[P, Future[P, R]]:
    ...

@overload
def transformer(func: Callable[P, Future[P, R]]) -> Callable[P, Future[P, R]]:
    ...
trim tangle
#

What errors do you get?

trim tangle
upper flame
#

same problem

Expression of type "Future[(x: int, y: int), int]" cannot be assigned to return type "int"
  "Future[(x: int, y: int), int]" is incompatible with "int"P
#

I think what I really want to express is:
have one return type IF you are under @transformer

#

And otherwise have a different return type

trim tangle
#

Well this is still incorrect:

def a() -> int:
    return b()

you need to change it to -> Future[something, int]

#

actually I'm not sure how to specify ParamSpec like that

#

Future[[], int] I think

upper flame
#

The desire is to have uniformed composable transformers: it should not matter is b called from a or from the non-transformer code

#

but the return types would be different

#

depending on is the caller function decorated or not

trim tangle
#

I think I am really confused about what you're trying to do

upper flame
#

haha sorry

#

any chance you can jump on a call in one of the rooms?

trim tangle
#

I don't think so

upper flame
#

ok no worries

#

let me try to write down something concise

trim tangle
#

I'll come back in like 30minutes btw

upper flame
#

thank you

upper flame
#
  1. There was a design choice that pre-dates my time, to use the same https://peps.python.org/pep-3107/ annotations that we all love for PEP484 for another purpose: they are used in runtime.
  2. In this example above that would correspond to doing a runtime checks inside the resolve() loop. So something like that
    def resolve(self) -> Any:
        res = self.get()
        while isinstance(res, Future):
            res = res.get()
            # check that what we returned matches the signature
            assert type(res) == res.f.get_return_type()  # <-- this is just a psudo-code, not real code
        return res
  1. I'm trying to make type hints double-purposed which already sounds like an somewhat alarming idea, but that's the objective. I'm not sure how far I can move the needle, but hopefully some? Maybe with some type lies and trickery. The main objective is not to have red squiggles for developers in VSCode without applying type-ignore, even if that means just giving up on any strictness of the checker (i.e. return Any all other the place is also in the space of possibilities).
  2. We have many many lines of code written without much respect to the type checker but they do conform to (2).
  3. That was kind of a high-level narrative, now to the details of this particular @transformer decorator. The way (1) dictates the type hints to be written and (4) conforms to it is that you can write the following things and they will work in runtime more or less how you'd expect
def sum_non_transformer(a: int, b: int) -> int:
    return a + b

@transformer
def sum_transformer(a: int, b: int) -> int:
    return a + b

def user_non_transformer():
    assert sum_non_transformer(1, 2) == 3
    # we are outside of a transformer and heance we need to call resolve explicitly
    assert sum_transformer(1, 2).resolve() == 3

@transformer
def user_transformer() -> Tuple[int, int]:
    # this example illustrates that when inside the trasnformer you can use another transformer without thinking about how it would be executed
    # the execution is orchestrated by Resolver on a remote machine and there is a lot of things going on
    # but as a user, it seems just like a normal function call.
    # we can mix transofrmer and non-transforemer calls,
    # note the return type is `Tuple[int, int]` -- that's what the runtime system expects and uses, not the `Tuple[int, Future[int]]`
    return (sum_non_transformer(1, 2, sum_transformer(4, 5))

def user_user_non_transformer():
    assert user_transformer().resolve() == (3, 9)
  1. It's not out of the table to try to change how the system works, or maybe force all the users to type their return types with Future[int] instead of int. Maybe that's the right way? But it's kind of like a big ergonomic hit.
    So the question is can something be improved without shading too much blood.
PEP 3107 โ€“ Function Annotations | peps.python.org

Python Enhancement Proposals (PEPs)

#

I guees their answer is to type it like this:

def func(func: Optional[Callable] = None) -> Union[Function, Callable]:
#

Not sure I was able to make it concise ๐Ÿคฃ

trim tangle
#

I don't think there's a non-invasive way of doing that tbh

#

maybe you could use async/await syntax

upper flame
#

yeah I was typing and just though about it ๐Ÿ™‚

#

it would make so much sense to have explicit async / await and not try to hide it

trim tangle
#

yeah, mixing differently "coloured" functions like this doesn't really work out

#

There's stuff like green threads and gevent but not sure where it's at nowadays

upper flame
#

ok thank you for baring with me -- it's a very useful exercise to bounce this with someone who understands the python typing better then me ๐Ÿ™‚

trim tangle
#

I would be careful with assuming that a random stranger knows something more than you

upper flame
#

lol, why?

trim tangle
#

I might just be really confident in what I'm saying ๐Ÿ™‚

upper flame
#

I think that's what I will do

from typing import Any, Callable, TypeVar,Generic, Union, overload
from typing_extensions import ParamSpec

P = ParamSpec("P")
R = TypeVar("R")

class Future(Generic[P, R]):
    def __init__(self, f: Callable[P, R], *args: P.args, **kwargs: P.kwargs) -> None:
        self.f = f
        self.args = args
        self.kwargs = kwargs
    
    def get(self) -> R:
        return self.f(*self.args, **self.kwargs)
    
    def resolve(self) -> Any:
        res = self.get()
        while isinstance(res, Future):
            res = res.get()
        return res

def transformer(func: Callable[P, Any]) -> Callable[P, Any]:
    def wrapper(*args: P.args, **kwargs: P.kwargs):
        return Future(func, *args, **kwargs)
    return wrapper

@transformer
def sum(x: int, y: int) -> int:
    return x + y

@transformer
def a(x: int, y: int) -> int:
    return sum(x, y)

print(a(2, 3).resolve())
#

At least we are going to have auto-complete for the transformer arguments

#

and the return types -- whatever

celest iron
#

but tbh, i think colored functions are only annoying for library writers

soft matrix
#

i didnt think anyone would ever defend gevent over asyncio lol

#

but we can all agree asyncio is bad

#

trio <3

tranquil turtle
#

can you explain (or point me to some article) why asyncio is bad?

mellow drum
#

I don't want to be too critical of it. It's really impressive, actually, and it's a whole lot better than nothing. But it's also confusing and hard to use. Sometimes that's the way things go when you're on the cutting edge.

celest iron
#

they've been adding more trio related ideas in the latest asyncio anyways, like nurseries iirc

#

there's a lib called anyio which basically just has an api the same as trio but allows u to use asyncio as a backend

soft matrix
trim tangle
#

as a user of asyncio I can confirm that it's ๐Ÿ…ฟ๏ธain

slender timber
#

why is there an inconsistency in the naming of protocols?

  • __len__: Sized
  • __bytes__: SupportsBytes
spiral fjord
#

Sized isn't a protocol it's an abstract base class

tranquil turtle
#

What is the difference?

rare scarab
#

You can use the abc with isinstance

dull lance
#

ABCs use nominal typing whereas Protocols use structural typing
(That being said, many of the ABCs in collections.abc have their issubclass check overridden to essentially support structural typing)

#

(Fixed my statement)

tranquil turtle
oblique urchin
lunar dune
# spiral fjord Sized isn't a protocol it's an abstract base class

For all intents and purposes, you should think of Sized as a runtime-checkable protocol. It behaves in basically exactly the same way as a runtime-checkable protocol at runtime, and typecheckers think it's a protocol because of some lies we tell them in typeshed. The only significant difference between Sized and the typing protocols is that it's implemented slightly differently at runtime and doesn't have Protocol in its mro โ€” and that's basically because we can't have the collections.abc module import typing, or Python's startup time could slow down somewhat

lunar dune
spiral fjord
#

Thanks for the clarification ๐Ÿ™‚

#

It does appear to be that the ones in typing use Supports* while the ones in collections.abc have a name made up for them.

slender timber
#

Ideally there should have been a Protocol for every dunder method, to avoid that jank in our code.

tranquil turtle
# 
class X:
    @property
    def x(self) -> float: ...

class Y(X):
    x: float # "x" incorrectly overrides property of same name in class "Position"
``` why this is invalid? how can i force it to be valid?
#

hmm, i just realized this code wont run (because Y().x = ... would cause X.x.__set__ logic, but it is not defined) ๐Ÿ˜ƒ

#

nvm

#

thank you guys

young zealot
soft matrix
#

no

young zealot
#

I guess I could use the inspect module to check the signature?

oblique urchin
young zealot
#

i guess it also won't satisfy the type checker without doing an ignore on it

young zealot
oblique urchin
young zealot
#

ok

umbral stump
#

I recently started to use type hinting and I use vscode. I noticed it comes with pyright. I have internal company lib that does not provide type hinting and I generated them using stubgen but it's been a challenge since I decided to be serious and enable "strict mode", I had to ignore couple of rules because whole code turned red. I fixed most of inconsistencies but there are others where I can't do much and I don't want to continue ignoring rules. Now, my first question is: what do you guys use with vscode, pyright or mypy?

soft matrix
#

Pyright

#

But why'd you use strict if your code doesn't conform to it?

#

Just set it to basic until you have all the stubs complete

umbral stump
#

learning purpose mostly. And about the wait until I get the stubs, is it the only way? it might take some time for that. Does that mean projects that use this libs are doomed to use basic mode?

upper flame
#

Curious, what are the current state-of-the-art techniques for increasing the presents of the type hints in the codebase?
I know about

  • pyre infer
  • monkeytype
    Are there any ML models that can add type hints?
oblique urchin
upper flame
oblique urchin
# upper flame Any chance it was recorded?

https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-gQIo/edit#heading=h.btsfybnycgt9 has notes and a recording (though poor quality)

Google Docs
Typing Meetup Notes

Typing Meetup Notes PyCon Typing Summit: April 20, 2023 Jelle Zijlstra, The State of Typing 2023 Slides Recording (this one unfortunately starts from the middle of the talk) Alex Waygood, Common Typing Pain Points from User Questions Slides Recording Someone: What about class decorators? And c...

tranquil turtle
upper flame
hallow flint
oblique urchin
hallow flint
#

not that i'm aware of. for editor integration there's copilot and some of the fancier copilot beta things, like copilot chat

tranquil turtle
tranquil turtle
upper flame
bold trout
#

kill me

async def get_table_content(filename: str, tablename: str, _from: int, to: int) -> Dict[str, List[Dict[str, str | int | bool | None]] | List[Any]]:```
brisk hedge
#

have you tried a TypedDict?

unkempt wren
#

Hey there, i'm confused about this, is it me or mypy has a bug ?

image.png
trim tangle
#

(also this is Pylance and not mypy)

unkempt wren
#

I guess Pylance leverages other tools including Mypy, but I'm not quite sure about that
The code is a simple condition that checks if len(someList) is the same as the count() of a Django QuerySet :

    # Erreur si toutes les ressources ne sont pas trouvรฉes
    if len(resource_ids) != await resources_qs.acount():
        return 400, MessageResponse(
            message="Une ou plusieurs ressources sont introuvables"
        )
#

type of resource_ids is set[int]

trim tangle
#

I guess Pylance leverages other tools including Mypy
Pylance uses pyright (well... kinda), but mypy is totally unrelated

unkempt wren
#

ok
maybe it's confused by the use of await

#

What project should I report this to ?

trim tangle
unkempt wren
#

I'll try to make this quickly

#

unfortunately I can't reproduce this in a minimal test case. It must have to do with Django ORM / Pydantic.. I'll leave it as is for now.

soft matrix
#

Have you tried restarting the lsp?

wicked scarab
#

Which one is generally preferred,

@overload
def func(a: str, highest: Literal[False] = False) -> list[int]: ...
@overload
def func(a: str, highest: Literal[True] = True) -> int: ...

or

@overload
def func(a: str, highest: bool = False) -> list[int]: ...
@overload
def func(a: str, highest: bool = True) -> int: ...
tranquil turtle
#

second is wrong

#

first is also not good i think

# 
@overload
def func(a: str, highest: Literal[False]) -> list[int]: ...
@overload
def func(a: str, highest: Literal[True]) -> int: ...
@overload
def func(a: str, highest: bool = ...) -> int | list[int]: ...
def func(a: str, highest: bool = False) -> int | list[int]: # your impl
wicked scarab
#

but pyright doesn't seem to be mad at it

wicked scarab
acoustic thicket
#

from the signature i feel like these should just be two different functions

oblique urchin
slender timber
#

overload is best for describing C++ API bindings

wicked scarab
acoustic thicket
#

my subjective opinion is yes they should

hasty jungle
brisk hedge
#

overloads and multiple inheritance are the two ways to perform type intersections

cedar sundial
#

Random example but hopefully this demonstrates my point. Is it possible to do this? Unpack a TypeVarTuple to retrieve an element from the tuple.

from typing_extensions import Unpack
from typing_extensions import TypeVarTuple
from typing import Generic

Ts = TypeVarTuple("Ts")

def check_something(x): ...

class Car(Generic[Unpack[Ts]]):
    def start(self, direction: str, *args: Unpack[Ts]) -> None:
        some_variable, *args = args # Here

        check_something(some_variable)
        self.stop(*args)

    def stop(self, *args: Unpack[Ts]) -> None:
        ...

Just experimenting with this to understand it a little better

image.png
soft matrix
#

the issue might be that youre using the parameter name still (args = args)

#

try a different name and theres a chance it might work?

cedar sundial
#

Yeah, looked into that and got something else with the same list[Never] type.

        some_variable, *t_args = args

        check_something(some_variable)
        self.stop(*t_args) # Unpacked argument cannot be used for TypeVarTuple parameter
oblique urchin
#

maybe your signature should be def start(self, direction: str, some_variable: str, *args: *Ts)

novel elbow
#

How do I get mypy to return an ast with typing information? I can't find any docs on it

cedar sundial
soft matrix
#

good luck on mypy docs for internals though :p

#

you can probably hack something together that does something vaguely along those lines but not all nodes in ast are annotatable

novel elbow
#

Yeah you helped me about 6 months ago with a similar issue, I managed to get types for variables but I wouldn't know what scope they're in which made it a bit useless if you had shadow named variables

#

I don't understand how an ide like pycharm knows the type of a variable if you click on it if they don't use an ast with typing information

soft matrix
#

it guesses if its pycharm

#

but in all seriousness it will go through the ast a some point but its more complicated than just that

# 
class Foo[T]:
    x: T
class Bar[T](Foo[T]): ...
class Baz(Foo[str]): ...
class Spam(Baz, Bar[int]): ...

def foo(bar: Bar[int]):
    ...

foo(Spam())```how come this doesnt warn you about incompatible types/lsp violations?
#

cause Spam() is allowed to be passed to a Bar taking function where x would be an int or something which seems wrong

oblique urchin
#

what does it think Spam().x is?

soft matrix
#

str

oblique urchin
#

guess it uses the first base

soft matrix
#

yeah it seems that way

#

i remember @hallow flint did something similar, did you ever do anything about it?

hasty jungle
oblique urchin
soft matrix
#

!e py class str_int(str, int): ...

rough sluiceBOT
#

@soft matrix :x: Your 3.11 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 1, in <module>
003 |     class str_int(str, int): ...
004 | TypeError: multiple bases have instance lay-out conflict
oblique urchin
soft matrix
#

they did i thought

oblique urchin
#

but your object could be passed to a function that takes a Foo[str] and do foo.x = "x", and also passed to a function that takes a Foo[int] that does foo.x = 3

soft matrix
#

oh maybe not

tranquil turtle
#

!e ```py
class X: slots = 'a',
class Y: slots = 'b','c'
class Z(X, Y): ...

rough sluiceBOT
#

@tranquil turtle :x: Your 3.11 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 3, in <module>
003 |     class Z(X, Y): ...
004 | TypeError: multiple bases have instance lay-out conflict
tranquil turtle
#

Do typecheckers know about this?

soft matrix
#

pyright doesnt seem to

hasty jungle
oblique urchin
soft matrix
#

fwiw c++ doesnt like this

image.png
hasty jungle
soft matrix
#

variance is inferred when using pep 695 syntax

#

but in this case its invariant yes

hasty jungle
#

okay! get it

oblique urchin
#

in general if the type parameter is an output type it's covariant, if it's an input type it's contravariant, if it's both it's invariant

hasty jungle
#

so because of invariance in that case, there could/shoud be no instance of Spam, if the type checker can infer non equality of int/str

oblique urchin
#

in this case it's an attribute that supports both getting and setting, so it's invariant

hasty jungle
#

but the issue is more about Spam here, rigth? that type actually shouldnt be instantiated, that's what you meant?

oblique urchin
open hawk
#

Hey folks, I've got a strange situation. I'm trying to make a "GenericMask" container, basically something that wraps an ordered set of a type T to statically know if a member of the type T is in the Mask. So, I've got

T = TypeVar('T')
class GenericMask(Collection[Hashable[Generic[T]]]):

    def __init__(self, mask: Iterable[T]):
        self._mask: dict[T: None] = {t: None for t in mask}

    def __contains__(self, item) -> bool:
        return item in self._mask

    def __iter__(self) -> Iterator[T]:
        return iter(self._mask)

    def __len__(self):
        return len(self._mask)

    def __str__(self):
        return repr(self)

    def __repr__(self):
        return f"{self.__class__.__name__}({', '.join(k.name for k in self._mask)})"

    def __eq__(self, other):
        return not set(self).symmetric_difference(other)

    def __hash__(self):
        return hash(tuple(c for c in self))

Attempting to have GenericMask be a Collection of Generic Hashables, so I can write

class ConstellationMask(GenericMask[Constellation]):
  ...
#

etc

tranquil turtle
#
  1. why are you using dict?
  2. dict[T: None] is not valid
open hawk
#

But, PyCharm keeps giving me a type error when I call

next(iter(constellation)).some_field_of_Constellation

saying it's a Hashable

soft matrix
#

Collection[Hashable[Generic[T]] isn't a valid base either

open hawk
#

Yeah, that's my problem,

#

I mean, I messed with it since it was working

tranquil turtle
#

You should bound T to Hashable

open hawk
#

But it wasn't giving me the right type hints before

open hawk
#

What's the difference?

open hawk
#

it should have read dict[T, None]

soft matrix
open hawk
#

What do you mean?

#

oh, language server protocol?

#

So I won't get error checking if I pass it something without __hash__ ... Well that's annoying

oblique urchin
#

no, Liskov substitution principle

open hawk
#

... wat

oblique urchin
#

object defines __hash__ but some subclasses of object are not hashable

open hawk
#

Right, I guess I could have figured that out. But the results are the same

#

I just checked, lists aren't hashable, but I can make a

class ListMask(GenericMask[list]):
  ...

and the type checker doesn't complain

#

Okay, well, I guess I'll just make it generic on T and not worry about binding to to Hashable. Actually, I will bind it to Hashable, because it's acting the way I want it to now, there's just no point.

hallow flint
soft matrix
#

have a similar issue with generics and subclassing where things should have been incompatible but werent

hallow flint
spice locust
#

why do TypeVars need a name parameter? seems a little bit redundant

#

looking at the source it seems that it is only used in 4 contexts:

  • pickling
  • __reduce__
  • __repr__
  • introspection
trim tangle
rough sluiceBOT
#

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

list[~T]
trim tangle
#

so yeah

#

And yeah, pickling as you mentioned. Though that's a bit strange

spice locust
#

maybe this is more of an old solution and could be done better now

trim tangle
#

NewType, collections.namedtuple and such also accept a name

#

it might make debugging easier at very little expense, I guess

acoustic thicket
#

so does creating classes with type()

trim tangle
#

What would you print instead of list[~T]? list[<TypeVar object>]?

spice locust
#

could the name be retrieved from the instance name by using a metaclass or something?

oblique urchin
trim tangle
#

No, you can't introspect assignments like this

oblique urchin
#

Note that in 3.12 you'll no longer need to repeat the name

#

!pep 695

rough sluiceBOT
#
**PEP 695 - Type Parameter Syntax**

Link

Status

Accepted

Python-Version

3.12

Created

15-Jun-2022

Type

Standards Track

acoustic thicket
#

watch me getsource()

trim tangle
#

yeah it's not possible without hacks ๐Ÿ™‚

spice locust
# oblique urchin !pep 695

yeah i just skimmed through this pep while looking for type params, and thats what sparked my curiosity with typevars

#

doesn't a metaclass's __new__ constructor accept a name param?

#

since it inherits from type

#

or wait, that only applies when using the class Foo: syntax doesnt it

trim tangle
#

yep

#

so if you call type or other metaclass manually, you will need to provide the name manually

#

class is kinda like calling a metaclass and then assigning it to a variable

spice locust
#

yup, but in that case it is able to grab the name of the 'variable' (where the variable is an instance of the metaclass)

oblique urchin
trim tangle
#

yep

oblique urchin
#

class Foo(A): pass is (approximately) syntactic sugar for Foo = type('Foo', (A,), {})

spice locust
#

i understand

trim tangle
#

One trick I've seen in TypeScript is to use object destructuring (pseudocode):

const {T} = typeVar();
// same as:
const T = typeVar().T;
``` but we don't have that
#

Like the example in this readme https://github.com/practical-fp/union-types ```ts
import { impl, matchExhaustive, Variant } from "@practical-fp/union-types"

type Shape =
| Variant<"Circle", { radius: number }>
| Variant<"Square", { sideLength: number }>

const { Circle, Square } = impl<Shape>()

slender timber
#

what use is annotating self or cls?

#

flake8-annotations ANN101 wants you to

soft matrix
#

theres basically never any point

#

only thing i can think of is for overloads on generic types or when decorators do funky things that alter the type

tranquil turtle
# 
x: float = True
``` this is valid ๐Ÿง
#

i understand why, but it is weird

# 
    def __imod__(self: P, other: float | _PointCommon, /) -> P:
        if isinstance(other, _PointCommon):
            ...
            return self
        if isinstance(other, float): # error   Unnecessary isinstance call; "float" is always an instance of "float"
            ...
            return self
        raise TypeError
``` pyright complains about second `isinstance` check
is there a way to avoid this?
mypy is happy with this code
viscid spire
wicked scarab
# 
for i in range(1):
    pass

try:
    print(i)
except:
    pass
```  Why does pyright still says `"i" is possibly unbound`? (btw, sorry to interrupt your question denball)
soft matrix
#

because it cant statically determine that range will always iterate

soft matrix
mild igloo
wicked scarab
#

But I have caught it though?

trim tangle
#

Type checkers don't really know what will happen if you do something wrong

#

They just flag stuff that's "invalid" to do. In runtime it might cause an exception, cause something bad down the line, or not cause any trouble at all

#

I would just put # type: ignore, or initialize i = 0 before the loop

wicked scarab
#

okay, thanks

trim tangle
#

It is a bit unfortunate, but there currently isn't a way to specify non-empty iterables for this

wicked scarab
#

I see, will they possibly add support for that?

trim tangle
soft matrix
#

yeah it just doesnt give you squiggles it just says it in a dialouge box

jade viper
#

How do I type-hint a string that has a default value?

tranquil turtle
#

String cannot have s default value

#

Function arguments can

jade viper
#

Yes, that's what I mean lol

#

A string argument that has a default value

tranquil turtle
#

x:str=""

jade viper
#

Just the regular str type hint then?

tranquil turtle
#

You can use Literal and @overload if you want

jade viper
#

How'd I use @overload in this case?

soft matrix
#

can you show your code?

oblique urchin
#

chances are you don't need overloads, that's generally only needed when there's some complex dependency between arguments/return values, like if particular string values produce a different return type

#

it would look something like this ```python
@overload
def f(x: Literal["int"] = ...) -> int: ...
@overload
def f(x: Literal["str"]) -> str: ...
def f(x: str) -> int | str:
if x == "int":
return 0
elif x == "str":
return ""
raise ValueError("wat")

jade viper
#

Oh I see

#

Thank you everyone!:)

slender timber
#

are there any plans for integrating ruff with type checkers for such issues?

#

or maybe a mypy plugin?

soft matrix
#

whyd it need to be integrated into the type checker?

oblique urchin
#

for what it's worth pyanalyze recognizes that iterating over a nonempty range() always enters the loop. It's hardcoded though, along with a few other types.

slender timber
soft matrix
#

i honestly dont think ive seen any false +ves from ruff

#

also i think doing ipc between rust and python would be hellish

slender timber
soft matrix
#

oh lol

slender timber
#

ipc isn't the only solution, you can use some inproc solution

#

but IPC overhead won't be much probably?

soft matrix
#

yeah but that doesnt mean its fun to work with

oblique urchin
#

what specifically would ruff be able to help with?

oblique urchin
#

the stuff about __orig_class__ feels very random, could use some context to explain why that's a problem

slender timber
#

Can I make a shortcut for Annotated[int, ClassName] like E[ClassName]?

soft matrix
oblique urchin
#

it only makes sense if you know what GenericAlias normally does

#

and I don't think a lot of people know that off the top of their head

#

I only know because we discussed this point a little while ago

soft matrix
#

ok ill do a bit more build up to it then

oblique urchin
#

I guess the audience for PEPs is basically the Steering Council. People on the SC are expected to have a strong general knowledge of how Python works, but they may not know every detail of how the type system or the runtime works.

#

so you shouldn't assume too much knowledge from readers

jade viper
#

Any idea why this doesn't work guys?

#

!e

from typing import IO
from io import BytesIO

def validate_subtype(*args: any, subtype: type):
    for item in args:
        if not issubclass(type(item), subtype):
            raise TypeError(f"Object '{item}' must be a child of type '{subtype().__class__.__name__}', but is of type '{item.__class__.__name__}'.")
            
print(validate_subtype(BytesIO(), subtype=IO))
rough sluiceBOT
#

@jade viper :x: Your 3.11 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 9, in <module>
003 |     print(validate_subtype(BytesIO(), subtype=IO))
004 |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
005 |   File "/home/main.py", line 7, in validate_subtype
006 |     raise TypeError(f"Object '{item}' must be a child of type '{subtype().__class__.__name__}', but is of type '{item.__class__.__name__}'.")
007 | TypeError: Object '<_io.BytesIO object at 0x7f193f1b8ea0>' must be a child of type 'IO', but is of type 'BytesIO'.
jade viper
#

BytesIO inherits from BinaryIO, which inherits from IO

image.png
oblique urchin
jade viper
oblique urchin
jade viper
#

Ooooh I see

#

I wanted to check if an object I receive as a function parameter is any kind of "buffer"

#

I thought there'd be more types than BytesIO and StringIO so I thought up this solution

#

But guess I'll just check if it's one of those two then

#

Ty:)

viscid spire
#

!e

from fishhook import hook
from typing import TypeVar

T = TypeVar("T")

@type
def FunctionType(): ...

@hook(FunctionType)
def __getitem__(self, _):
  return self

def f(x: T, y: T) -> None:
  print(x, y)

f[int](3, 5)
rough sluiceBOT
#

@viscid spire :white_check_mark: Your 3.11 eval job has completed with return code 0.

3 5
viscid spire
#

and just let typecheckers handle the rest

soft matrix
#

returning self is no good imo

oblique urchin
#

also using fishhook is no good

viscid spire
#

well I did say the naive implementation ๐Ÿ˜„

viscid spire
soft matrix
#

i already have the code working ;)

#

somewhere in /usr/bin

celest iron
#

fishhook is funny haha.
reminds me of kotlin's extension functions

#

tho those are scoped, why are sweet

blazing nest
viscid spire
#

is there a way to typehint what exceptions may be raised from a function?

slender timber
#

You can document exceptions raised from the function body itself

viscid spire
#

perhaps NoReturn could get a subscript with the exception type (or a union of types)?

# 
from typing import NoReturn
def f() -> NoReturn[ValueError]:
  raise ValueError()
slender timber
#

and those empty parentheses aren't needed

viscid spire
#

ik and I thought about that and idk decided to include them

slender timber
#

but it comes to the point where with overloads and stuff, you basically describe you function so much, that the actual logic is represented by typing itself

viscid spire
#

I also considered that

#

consider for example int

slender timber
#

even if NoReturn accepted generic parameters, its exact scope would be?

viscid spire
#

only allowed in return typehints

slender timber
#

The function body itself or an exception raised and not caught from another function?

viscid spire
#

hmm good point

#

in which case you would need to include that

slender timber
#

because if its the first, its like an incomplete hint

viscid spire
#

it would be any error that can arise when the function is called

slender timber
#

and the second can be too vast to cover

viscid spire
#

ik, it's true

#

how do languages which require this typing go?

#

cause like doesn't Java require you to write what errors can arise?

slender timber
#

java has it and you only mention the exceptions explicitly thrown by the function

viscid spire
#

hey, if it ain't broke, don't fix it!

slender timber
#

C++ has a noexcept keyword which optimizes code gen for functions where its used

#

exception handling isn't cheap, and requires extra code

tranquil turtle
#

NoReturn (now it is the same as Never) is supposed to declare that function never returns:

def f() -> NoReturn: ...
f()
print('hello') # unreachable!

i think it would be better to declare possible exceptions in different way: ```py
-> Raises[int, KeyError, ValueError] # int is a return type (in case it does return)
-> Raises[int, (KeyError, ValueError)] # alternative syntax
-> Raises[Never, (KeyError, ValueError)] # never returns, raises KeyError, ValueError or hangs

-> int | Raises[Exception] # raises or returns int
-> Raises[Exception] # raises always
-> Never | Raises[Exception] # raises or hangs
-> Raises[E1] | Raises[E2] # raises E1 or E2
-> Raises[E1, E2] # same as previous

@Raises[KeyError, ValueError] # maybe decorator would be better?

viscid spire
#

raises int lol

slender timber
#

If Python had a ResultOrError idiom like Rust, it would have been much better to type hint

viscid spire
# 
int | Raises[KeyError, ValueError]
tranquil turtle
tranquil turtle
viscid spire
#

I also thought of writing Raises before I posted xd

#

but thought maybe something else that already exists can be used instead

#

Raises is certainly a good word

slender timber
#

putting it into the return type hint is not the correct place. because an exception isn't returned, its raised, a decorator makes more sense

#

if in future it can be used for optimization rather than have a purely aesthetic purpose, it would be good

viscid spire
#

what would the decorator actually do?

slender timber
#

in the beginning it can be just a no-op, like typing.cast

viscid spire
#

I was thinking as such

tranquil turtle
#

typechecker can treat Raises[...] as Never, so T | Never is just like T and everything works like before
but typeckeckers can also do additional logic

slender timber
#

but tools like beartype can use it to check whether an exception other than those gets raised

viscid spire
#

problem might be if you want to write overloads for exceptions

tranquil turtle
slender timber
tranquil turtle
#

-> Raises[Never]

#

funny thing

#

it always raises, but it can never happen...

slender timber
#

i doubt how much really exception information can be useful for type checkers, like what will they enforce in terms of how type hints are enforced?

green gale
#

or at least that's my interpretation

slender timber
#

i believe mostly, exceptions aren't raised explicitly but are propagated, and sometimes you want this propagation

green gale
tranquil turtle
green gale
#

right, but the entire concept of Raises[Never] is ill formed

#

so that declaration should be what's invalid

tranquil turtle
slender timber
#

And that's no exclusive to exceptions either

tranquil turtle
green gale
#

annotating exceptions just seems annoying in general tbh

#

just because there are so many and you don't care to write them out

tranquil turtle
#

annotating entire stdlib and forcing all lib maintainers to annotate their exceptions too would be very painful

green gale
#

i mean, typeshed could help, but it'd be annoying to write out all the possibilities without libraries liberally aliasing

slender timber
#

and then there's functions which use return values to indicate success or failure

green gale
#

including stdlib

#

which would be a massive migration

slender timber
#

Imo type checkers make you write code which is more lbyl than eafp

tranquil turtle
#

-> Raises[...] (or similar syntax) would be useful to declare that function propagates all exceptions from underlying functions (it is not raising by itself, it is not cathing anything)

def f1() -> int | Raises[E1]: ...
def f2() -> int | Raises[E2]: ...
def f() -> int | Raises[...]: # same as `-> int | Raises[E1] | Raises[E2]`
    return f1() + f2()
green gale
tranquil turtle
tranquil turtle
green gale
#

right

slender timber
# green gale wat

yea like without type checkers that piece of code would have been a try... except, but since the type checker knows the types now, we need to use isinstance and such

green gale
tranquil turtle
#

LBYL: look before you leap
EAFP: easier to ask forgiveness than permission

green gale
#

yeah I'm a thoroughly static type-pilled person so

slender timber
#

Like the more typed you go, the more LBYL your code gets

#

Previously what would have been a direct dict key lookup wrapped in a try except, now becomes an if key in typed_dict for TypedDict

tranquil turtle
#

EAFP is sometimes unavoidable (sometimes in concurrency, for example)
also EAFP is sometimes faster because you dont have to ask for permission, you just do what you want and ask for forgiveness in 1% of cases

green gale
#

but i see your point

slender timber
#

EAFP is faster in python because the asking for permission part can be costlier due to lookups, but handling exceptions is definitely much more costlier

mellow drum
#

If we annotated all exceptions, then everything would need Raises[KeyboardInterrupt] and that would be extremely annoying. Maybe it's a special case?

#

Or maybe Raises should require a subclass of Exception and not a subclass of BaseException.

#

Also you'd need to figure out ExceptionGroup. I guess that's effectively a union.

brisk hedge
#

in general exception annotations are really annoying for static typing

#

C++17 removed throw in function signatures (and it was considered bad practice), and everyone hates java's checked exceptions (c# removed its own checked exceptions!)

#

the big statically typed langs have tried it and decided wholeheartedly that it sucks

#

(ignoring effect systems for now)

narrow sigil
brisk hedge
#

or has different runtime semantics

#

you can't inspect the result of it very easily

narrow sigil
#

can you provide an example of that

brisk hedge
#

easy, you can't overload it

#

a or b either returns a or b, never anything including both

narrow sigil
#

i was gonna check but i couldn't find a good list

brisk hedge
#

yeah, the operators and, or, is, not, is not can't be overloaded (and not in / in can only return booleans)

#

i think comparisons might be restricted to only booleans as well? not sure

narrow sigil
#

so it's because they wanted t1 | t2 to be overloadable and usable in expressions without changing the semantics of or

trim tangle
#

yep

trim tangle
#

What line do you get the error on?

#

according to the signature, It is valid for your function to return e.g. None or float, but Flask will not accept that

#

In your case you can mark the return type as -> str because you're always returning a string

#

wdym?

#

-> Flask.Response, I presume

#

You can check the source code of the library to see all the type annotations

#

editors like VSCode/PyCharm let you press a button on a variable and jump to its definition in library code

heady flicker
# tranquil turtle `-> Raises[...]` (or similar syntax) would be useful to declare that function pr...

I think, mypy + returns allows you to do a similar thing. Though a lot more convoluted, functional, and monad-based :D

https://github.com/dry-python/returns

GitHub
GitHub - dry-python/returns: Make your functions return something m...

Make your functions return something meaningful, typed, and safe! - GitHub - dry-python/returns: Make your functions return something meaningful, typed, and safe!

trim tangle
#

monad = based

heady flicker
#

Nope. Basedmypy = based

stray summit
#

anyone aware aware of a way to define callable protocols with different overloads that have some keyword only arguments which are permissible as passed in function beside having multiple callable protocol classes and taking the union?

brazen jolt
#

Just use typing.overload on __call__ of that protocol? Maybe I'm missunderstanding something, could you give some more detail to what you're trying to do?

stray summit
#

i have a config object that has a optional function attribute which must match one of the calling conventions - a overloaded protocol couldnt match that as i need a one of, not a all of

#

it might be best to introduce a wrapper type that unifies the call patterns so that the outer type is mosre straightforward

brazen jolt
#

yeah, probably, I don't think there's a way to do this in any other way than a union of protocol classes, each with their own __call__, sadly, there's no way to simplify that as Callable doesn't support specifying kw only arguments directly afaik

#

just hide that behind a type alias and use that, but yeah, it is annoying

#

Why do you need to support various function signatures though? That's pretty rare when taking in a function as parameter

stray summit
sacred spindle
#

Greetings from #pyqtgraph

I come with a head-scratcher, not sure if this is feasible at all, but right now, I'm trying to address a pyright unknown import symbol

For a little context, pyqtgraph has a Qt-abstraction layer, where Qt.py can be overly simplified as ....

try:
    from PyQt6 import QtCore, QtWidgets, QtGui
except ImportError
    try:
        from PySide6 import QtCore, QtWidgets, QtGui
    except ImportError: 
        print("PyQtGraph needs Qt bindings to work...")

Throughout the library, we reference QtCore, QtWidgets, and QtGui via

from pyqtgraph.Qt import QtCore, QtWidgets, QtGui

This results in the each of the imports (QtCore, QtWidgets and QtGui) having a pyright error for unknown import symbol

What would be a good way to get type-annotations working here? For now, just trying to get pyright, might try pylance or mypy later...

image.png
trim tangle
#

it may be giving such error because the QtCore, QtWidgets, isQObjectAlive will be indefined in one of the branches
(just a conjecture)

sacred spindle
#

hmm... hand't considered doing it, suppose I can do that.. (if it makes any difference, the layer is actually inside pyqtgraph/Qt/__init__.py)

trim tangle
#

also why are you importing QtCore twice?

#

though that's probably irrelevant

sacred spindle
#

I'm not?

trim tangle
#

in your first code snippet

sacred spindle
#

look at the from X

trim tangle
#

from PyQt6 import QtCore, QtWidgets, QtCore

sacred spindle
#

oh!

#

typo on my end ๐Ÿ˜›

trim tangle
#

ah, I assume that's not a thing in the real code

sacred spindle
#

I'll try __all__ and see what happens

#

still same error ๐Ÿ˜ฆ

soft matrix
slender timber
#

!pip QtPy

rough sluiceBOT
#
QtPy v2.3.1

Provides an abstraction layer on top of the various Qt bindings (PyQt5/6 and PySide2/6).

sacred spindle
#

Iโ€™m familiar with QtPy. We donโ€™t use it for a variety of reasons, we may adopt it in the future instead of rolling out our own abstraction layer, but weโ€™re not there yet

sacred spindle
heady flicker
#

Do you think it's possible to use typing.overload on a property?

I.e. The property's value will change based on the generic arguments to self

hallow flint
#

oh interesting. i'd bet that no type checker handles that well if you tried

trim tangle
#

that sounds like pain yeah

#

use a method ๐Ÿ˜‰

#

I usually write a method unless I need to be compatible with some interface

heady flicker
#

I can't. I am type hinting an old library (redis-py)

hallow flint
#

maybe thereโ€™s some descriptor trick you could use

heady flicker
#

I don't think so because overload behavior is built into the type checkers

atomic idol
#

Code:

    def __gt__(self, other: Self) -> bool:
        if not isinstance(other, self):
            raise TypeError(
                f"'>' not supported between instances of "
                f"'{self.__class__.__name__}' and '{other.__class__.__name__}'"
            )
        return self.index > other.index

mypy error

error: Argument 2 to "isinstance" has incompatible type "Self"; expected "_ClassInfo"  [arg-type]

How do i use it then.

slender timber
heady flicker
#

Huh. Interesting. Will check it out.

trim tangle
slender timber
#

Do I subclass existing protocols from _typeshed in type stubs, or do I add stuff like dunder methods manually instead?

#

the stubgen have generated stuff like this:

def __add__(self, other): ...
__radd__ = __add__
def __iadd__(self, other): ...
def __mul__(self, num): ...
__rmul__ = __mul__
def __imul__(self, num): ...

so if I subclass, do I subclass from both SupportsAdd and SupportsRAdd or just the first and leave __radd__ = __add__ in the stubs?

#

it seems I should use class MyAdder(SupportsAdd[Iterable[_T], Self]): ..., as MyAdder cannot be referenced there?

soft matrix
#

Self doesnt work there either

#

unless pyright solved its graph theory issues

slender timber
soft matrix
#

are you using mypy?

#

i swear i disallowed this cause pyright cant support it

slender timber
#

i am using mypy + pylance both

soft matrix
#

!pep 673

rough sluiceBOT
#
**PEP 673 - Self Type**

Link

Status

Accepted

Python-Version

3.11

Created

10-Nov-2021

Type

Standards Track

slender timber
#

pyright preview

soft matrix
# 
class Bar(Generic[T]):
    def bar(self) -> T: ...

class Baz(Bar[Self]): ...  # Rejected```
#

oh well if it works i wouldnt complain

slender timber
#

heh

#

so is subclassing good?

soft matrix
#

its unknown

image.png
slender timber
#

huh

#

why?

soft matrix
#

i think its that at least

slender timber
#

how can i avoid this

if TYPE_CHECKING or sys.version_info >= (3, 9):
    IDEventMap = dict[int, AnyEvent]  # Pyright Error
else:
    IDEventMap = dict

class MyMap(IDEventMap): ...

Pyright: Subscript for class "dict" will generate runtime exception; enclose type annotation in quotes

trim tangle
slender timber
trim tangle
#

yep

atomic idol
acoustic thicket
#

just type them as Node

atomic idol
#

๐Ÿ‘ Worked!

spice locust
#

do i understand correctly that when type hinting callables, i must put the most specific type hint in the parameters according to the rule of contravariance? e.g. my type hint would be Callable[[Dict[str, Any]], None], so the user's function's param can be a dict, Mapping, MutableMapping, etc?

pastel egret
#

The user's function's param can be one of those yes, though you might want to make it MutableMapping, to allow yourself to change what type you pass to the function in the future if you need to.

spice locust
#

if i type my decorator's parameter as Callable[[MutableMapping[str, Any]], None], the user cannot type their function's parameter as dict

#

so i guess the solution here is to either A. explicitly specify that the object being passed around is a Dict, or B. tell the user if they want to add a type hint themselves it has to be MutableMapping

#

now, if pyright would automatically infer a decorated function's param types, the second option wouldnt be needed

pastel egret
#

They would be forced to use MutableMapping, which might be good anyway - since later you could decide to pass a different type if that's easier for your code, a ChainMap for instance. Might not be worth it though.

spice locust
#

maybe the easiest thing to do is provide a type alias that users can type-hint with for convenience

#

like

# exported from my package
HandlerInput = MutableMapping[str, Any]

# user's code
from mypkg import HandlerInput

@actions.action(...)
async def hello(payload: HandlerInput):
    return f"Hello, {payload.get('name', 'world')}!"
#

what do you think?

#

however there remains one problem. if the user wants to create a TypedDict to type what fields their function expects in the dict parameter, the types become incompatible again

image.png
pastel egret
#

You could just use a dict, if you are fine with guaranteeing that.

spice locust
#

if i change my type alias to be HandlerInput = Dict[str, Any], it is still incompatible with any user-defined TypedDict

#

so i guess the only way to allow the user to use their own mapping subclasses is to make the argument type hint Any on my side?

#

since i can't guarantee my package code will pass a value that is compatible with some arbitrary user-defined type

pastel egret
#

Well, that is quite true, it's not certain that the payload your package passes is compatible.

spice locust
#

or could i use typevars here?

pastel egret
#

You could, bound=dict, but that would just push the type-unsafety into your own code. I'm not sure, is type[T] where T is a TypedDict valid? You could have your decorator take the TypedDict, then validate it matches on your side...

spice locust
#

ideally, my code would validate that it is compatible and throw a TypeError if it isnt, without much code and no deps

#

so the user can use any dict subclass as a type hint, and my decorator would accept that and throw an error later on if the data received doesn't validate

#

something else that just came to mind: the fields in the payload param depend on the inputs defined in the decorator's params, e.g. if i add an input named "name", then the payload should contain a field accessible with payload["name"]. is there a way to generate this type from the param and use it for type hinting?

viscid spire
#

isn't TypedDict always a mapping of str to something?

#

seems so strange

pastel egret
#

Mapping[str, Any] is fine, but Dict is invariant in keys and values since it's mutable.

dull lance
pastel egret
#

If it was assignable, this would be possible:

class MyTD(TypedDict):
    spam: int

a_dict: MyTD = {'spam': 42}
alias: dict[str, Any] = a_dict
alias['spam'] = "non_int"
dull lance
#

so, (dict[str, Any]) -> None can be assigned to (Mapping[str, Any]) -> None, but not the other way round

spice locust
#

this is my decorator

    def action(self, *, name: str = "", title: str = "", inputs: Sequence[Input] = tuple()):
        """Decorator to add an action."""

        def decorator(handler: ActionHandler):
            return self.add(
                handler=handler,
                name=name,
                title=title,
                inputs=inputs,
            )

        return decorator

these are my type aliases

HandlerInput = Mapping[str, Any]
ActionHandler = Callable[[HandlerInput], Awaitable[Any]]

here's my decorator being used

@actions.action(
    title="Say hello",
    inputs=[
        Input(
            name="name",
            type=InputType.STRING,
            optional=False,
        ),
    ],
)
async def hello(payload: ??):
    return f"Hello, {payload['name']}!"

hopefully this gives us a better starting point to look for solutions

pastel egret
#

So do the Input objects correspond to payload keys?

dull lance
# viscid spire isn't TypedDict always a mapping of str to something?

TypedDict can't be assigned to Mapping[str, Any]. Consider this function:

def foo(m: Mapping[str, Any]) -> Any:
    return m['k']

Pyright allows such a function since Mapping[str, Any] lets you pass any string key. On the other hand, TypedDicts don't necessarily have a key k, so it would result in an error if you passed a TypedDict into foo.

spice locust
pastel egret
#

That's not really able to be typed as it is right now, since there isn't a way for type checkers to understand the correspondance.

#

What you could do instead is use dataclass_transform, perhaps?

#

So the user code would be something like this:

@actions.payload(
    title="Say hello",
)
class Hello:
    name: string = Input(optional=False)

@actions.action(Hello, title="Say hello")
def hello(payload: Hello):
    return f"Hello, {payload.name}!"
dull lance
#

so you can't infer the type of payload based on what is in the decorator

#

you would have to give payload a type in the input function. the most you could do afterwards would be to check that your given type for payload matches what is in the decorator

#

which you might be able to achieve via dataclass_transform

pastel egret
#

You can yes. Your field class can do whatever, so that can just be the options in Input.

#

You wouldn't need to reimplement dataclasses though - just remove the Input objects from the namespace temporarily, call dataclass() on the class, put them back (or in your own data structure).

royal lagoon
#

Hi,
I am trying to achieve completion via type hinting with pydantic models.
here is an example:

from dataclasses import dataclass
from typing import Type

from pydantic import BaseModel


class OrderReceived(BaseModel):
    order_id: str


@dataclass
class Event:
    source: str
    detail_type: str
    detail: Type[BaseModel]

OrderReceivedEvent = Event(
        source="Atlas.OrderReceiver",
        detail_type="OrderReceived",
        detail=OrderReceived,
)

what i want to be able to do is to do this and get completion for the needed parameters:
OrderReceivedEvent.detail(orde...)

Does anyone know if this is possible and if so how?

dull lance
#

so OrderReceivedEvent would be an instance of Event[OrderReceived]

royal lagoon
#

like this?

T = TypeVar("T", bound=BaseModel)

class OrderReceived(BaseModel):
    order_id: str


@dataclass
class Event:
    source: str
    detail_type: str
    detail: Type[T]

tried it but doesn't work unfortuanetly

dull lance
#

you need to make Event a subclass of Generic

royal lagoon
#

that was the missing piece, thank you!
Didn't work that much with generics in python as they still feel weird to me

spice locust
#

something like

@dataclass
class HelloPayload:
    name: Input(optional=False)

@actions.action(payload_type=HelloPayload, title="Say hello")
def hello(payload: HelloPayload):
    return f"Hello, {payload.name}!"
pastel egret
#

What dataclass_transform does is tell the type checker that your function behaves just like @dataclass, causing the checker to apply the same special casing rules.

#

The problem there is that Input isn't a dataclass.field, so you need your own function.

spice locust
#

right

pastel egret
#

You could use field(..., metadata={'actions': Input(...)}).

#

metadata just gets exposed on the field object, and is intended for this sort of thing.

spice locust
#

alright... i will look into it more tomorrow, thanks for all the help so far!

lapis swift
#

why i forgot the logic of the code that i wrote some days ago

dull lance
lapis swift
#

Thx i thought it was general

novel elbow
#

why does pycharm think a is an int but still says it expects a str

acoustic thicket
#

thats funny

novel elbow
#

mypy has the same issue

#

idk how to check the type of a for other type checkers though

acoustic thicket
#

have you checked mypy in strict mode

#

id expect it to error on a = t(5)

novel elbow
#

it does but it still thinks a is an int

#

it has the same bug as pycharm

acoustic thicket
novel elbow
#

I mean I know pycharm uses a custom version of mypy so it makes sense

novel elbow
#

these are the types it thinks the vars are

#

{'location': {'start': {'line': 14, 'column': 6}, 'stop': {'line': 14, 'column': 7}}, 'name': 'a', 'fullname': '__main__.a', 'node_name': 'NameExpr', 'annotation': 'builtins.int', 'attrs': []}

#

you can see at line 14 it thinks it's an int

oblique urchin
#

you may want to use --allow-redefinitions

novel elbow
#

even with redefinitions allowed it still errors there

novel elbow
trim tangle
novel elbow
#

ah my bad

patent saffron
#

you can forward declare it

a: Union[int, str] (or a: int | str)

fair forge
# novel elbow

Type checkers by default freeze variable type to the first declaration

#

Delete a = 1

#

To fix that error

#

In that case a is at global scope, so when you use inside the function you are overwriting the variable. If you declare first the function you will use a as local scope and dont have colision problems

novel elbow
fair forge
#

Umm idk let me check

fair forge
#

from typing libarary

#

but I don't recommend it

novel elbow
#

I'd like to do it without modifying the source code

fair forge
tranquil turtle
#

You can do -> Any

#

Or a: Any

fair forge
#

if he use Any lose types properties

tranquil turtle
#

Yes, but he is already doomed

fair forge
#

That's the same as not having types

rare scarab
#

add # type: ignore on the line of the return instead.

novel elbow
# novel elbow

but it must know that a is a str at the end otherwise it wouldnt be able to tell me it expected a str, so why would it show me that a is an int?

hasty jungle
novel elbow
#

in my case I'm making an obfuscator and type information is very useful for more advanced features so I can't rely on the customer writing quality code

hasty jungle
novel elbow
#

I mean not valid is a big word since all we're doing here is reassigning a variable to a different type

hasty jungle
#

in the sense of mypy, I said ๐Ÿ˜„

#

I agree with you, it should have been understood, if python was as smart as smth made for that kind of things

#

(but that's where I stop believing in python, personnaly)

novel elbow
#

sure but the thing is that it does actually know that a is a str otherwise it wouldnt be able to tell us the unsupported operation cuz of str + int, so why would that info not appear in the type list it generates?

#

so perhaps there is some processing done while it's checking for these issues that isn't done when generating the type list

hasty jungle
#

but wait, you talk about the runtime crash, or the pycharm warning?

novel elbow
#

the pycharm/mypy warning

upper flame
#

Is there a way to type torch.nn.Module better?

def __getattr__(self, name: str) -> Union[Tensor, 'Module']:

Currently we have this annoying problem
https://github.com/microsoft/pyright/issues/4213

Ideally we would know when we get Tensor and when Module.

oblique urchin
#

if we know that, maybe we can figure out a way to express it to the type checker

upper flame
#

It's kind of bad

    def __getattr__(self, name: str) -> Union[Tensor, 'Module']:
        if '_parameters' in self.__dict__:
            _parameters = self.__dict__['_parameters']
            if name in _parameters:
                return _parameters[name]
        if '_buffers' in self.__dict__:
            _buffers = self.__dict__['_buffers']
            if name in _buffers:
                return _buffers[name]
        if '_modules' in self.__dict__:
            modules = self.__dict__['_modules']
            if name in modules:
                return modules[name]
        raise AttributeError("'{}' object has no attribute '{}'".format(
            type(self).__name__, name))
rough sluiceBOT
#

torch/nn/modules/module.py line 1620

def __getattr__(self, name: str) -> Union[Tensor, 'Module']:```
oblique urchin
#

um yes, that looks pretty hopeless

upper flame
upper flame
#

Going to propose a patch to return -> Any

spice locust
#

can i define a type hint for a class method based on one of the constructor params (or some other means dynamically)?
the snippet below doesn't work, but hopefully illustrates what i am trying to achieve

class Ham:
  T = TypeVar("T")

  def __init__(self, type: Type[T])  # type accepts a type object, e.g. Tuple
    self.type = type

  # param's type hint is based on a value recieved in __init__ or otherwise retrieved somehow
  def method(self, param: T):
    pass

ham = Ham(Tuple)
ham.method((1, 2))  # valid
ham.method("string")  # invalid
#

i think this is supposed to be solved by pep 695, but i'm wondering if there is a way to do it in python 3.8 (admittedly any solution will probably not be very elegant)

void panther
spice locust
#

yup that idea just materialized in my head, testing it right now

#

it looks a bit cursed but type checker is not complaining

ham = Ham[Tuple](args)
spice locust
#

is there a way to type hint that an argument has to be literally a Union[int, float] object?

#

as in,

def func(param: TypeHint):
  pass

func(Union[int, float])
#

my solution right now is to put in in an enum, like so

class ParamType(Enum):
  TYPE = Union[int, float]

def func(param: ParamType):
  pass

func(ParamType.TYPE)
stray summit
#

for example pytest has a storage + storagekey concept so one can do typesafe stashing of own object by encoding a type in the storage key

spice locust
#

i want my function to accept one of 3 types as an argument: bool, str, or Number, where Number is equal to Union[int, float]. the function cannot accept int or float by themselves

stray summit
#

but one can also use something like a type[T] and a typevar

#

so the function takes something like Literal[type[bool], type[str], type[Union[int, float]]] - whats the intent there, it seems a little scetchy and confusing at first

#

(i'd like o avoid the XY-problem)

spice locust
#

the function is a class constructor, the class in question is a helper to define some JSON structure as it would appear in API usage, and the parameter is so the class instance knows what type it is

#

the API in question supports 3 types: "string", "number", and "boolean", roughly equivalent to the javascript types of the same name

stray summit
#

do you have a example of the api you envision, and is there any reason for not using something like pydantic,chattrs or one of the other schema toolkits as to me this reads like you are invention your own, and i made that invention your own as a mistake often enough to be very compelled to tell others not to repeat my mistake ^^

spice locust
#

the API already exists, this is simply a package to make it easier to define an interface with it

#

to give you the full truth here, there already exists a javascript package that does this for the API, and i'm just writing an equivalent in python

#

based on the javascript package

stray summit
#

packages like pydantic are simply there for helping with that - i strongly recomend a little discourse into tools for dealing with apis and schemas, as transcribing a javascript package to python is most likely to cause pain and suffering to users of said package

#

is the package open source/public or something internal?

spice locust
stray summit
#

this suspiciously looks like a proprietary mutilated openapi clone

acoustic thicket
#

!d typing.NewType

rough sluiceBOT
#
typing.NewType 

class typing.NewType(name, tp)```
Helper class to create low-overhead [distinct types](https://docs.python.org/3/library/typing.html#distinct).

A `NewType` is considered a distinct type by a typechecker. At runtime, however, calling a `NewType` returns its argument unchanged.

Usage:

```py
UserId = NewType('UserId', int)  # Declare the NewType "UserId"
first_user = UserId(1)  # "UserId" returns the argument unchanged at runtime
spice locust
soft matrix
rare scarab
#

I'm ashamed of what I have done. ```py
class classproperty(Generic[S, T]):
def init(self, func: Callable[[type[S]], T]):
self.fget = func

def __get__(self, instance: None, owner: type[S]) -> T:
    return self.fget(owner)

class Foo(BaseModel):
if TYPE_CHECKING:
# doesn't work well with pydantic at runtime
@classproperty
@classmethod
def bar(cls) -> list[str]:
...
else:
# mypy complains about this
@classmethod
@property
def bar(cls)-> list[str]:
return []

tranquil turtle
#

my brain melts

mortal cipher
#

is there a way to add a type parameter to a function such that it will return the given type? the equivalent of this typescript declaration in python:

declare function foo<T>(key: string): T;
#

i can do cast(T, foo(key)) of course, but definitely not elegant

soft matrix
#

That should work without the cast

mortal cipher
#

wdym?

soft matrix
#

Unless this isn't inside a class

rare scarab
#

if it knows the assignment, it will work.

mortal cipher
#

ah, so something like this?

my_value: T = foo(key)
soft matrix
#

Why are you converting to T here?

rare scarab
# 
T = TypeVar("T")

def foo(key: str) -> T:
  pass

x: int = foo("12")

Though generics are usuless if the var is only used once.

mortal cipher
#

in actual code, this is a registry which will return a class based on a key, and i need to be able to cast it to a more specific type if i want to be able to do anything with it.

rare scarab
#

Maybe something like this. ```py
class MyKey(Generic[T], str): pass

MY_KEY = MyKeyX

def foo(key: MyKey[T]) -> T: ...

value = foo(MY_KEY)

soft matrix
#

I think what you're doing is just wrong in this case, if you know the type of key then the type should be the same not T

mortal cipher
soft matrix
#

(Unless key is of type T but even then you shouldn't be casting)

mortal cipher
#

i think @rare scarab's solution is an elegant one here

#

another question: is there a way to express a protocol class itself as a type? i want to have a function which accepts a runtime-checkable protocol class and does some stuff which includes an issubclass call with that protocol, but mypy says Variable "typing.Protocol" is not valid as a type

rare scarab
#

Now if only we could have generic enum values

soft matrix
#

Best you can do is probably just type

mortal cipher
#

fair enough

#

now that i think about it, it wouldn't be of much use to know the type anyway, since the code using the type is generic

upper flame
rare scarab
#

Is there a way to get mypy to infer union types instead of the common base?

soft matrix
#

nop thats just how mypy works

#

use pyright :p

rare scarab
#

Now, how do I properly configure a default value for a typevar?

soft matrix
#

default=?

rare scarab
#

This doesn't work. ```py
from typing_extensions import TypeVar

TNode = TypeVar("TNode", bound=BaseNode, default=Node)

def foo(cls: type[TNode] = Node) -> TNode:
pass

#

mypy says

Incompatible default for argument "cls" (default has type "type[Node]", argument has type "type[TNode]")

#

it was saying the same thing before I added default=

soft matrix
#

theres no way that ik of that will work with both mypy and pyright here

#

without a type ignore

#

pyright is fine with the thing youve sent but pyright doesnt like cast(TNode, Node)

rare scarab
#

I'll just remove the default argument. I only used it in one place anyway

#

I'm down to 3 mypy type errors in my project

#

Oh, of course it's inferring the dict to be dict[str, type[payload.type]]

    kwargs = {}
    if payload.type:
        kwargs["type"] = payload.type
    if payload.dup:
        kwargs["dup"] = True
#

This feels nice.

Success: no issues found in 58 source files

#

Now if only adding a metaclass to my pydantic models didn't break pyright

#

Found a workaround for that.

#

I had my models configured like this. ```py
from pydantic import BaseModel as PydBaseModel
from pydantic.main import ModelMetaclass

class BaseModel(PydBaseModel):
pass

class MyModelMeta(ModelMetaclass):
pass

class MyBaseModel(BaseModel, metaclass=MyModelMeta):
pass

class MyModel(MyBaseModel):
foo: str

MyModel(foo="bar") # No parameter named "foo"

#

Setting the base of MyBaseModel to PydBaseModel makes it work.

#

But that's not what I want.

#

The better solution I found was to declare MyModelMeta as class MyModelMeta(type(BaseModel))

jade viper
#

Hey guys! What is the most user friendly way to type hint an object that must inherit from some class? Is it just Type[Class]? Doesn't seem very intuitive

rare scarab
#

Class works. type[Class] means it expects the class instance.

#

if Class is abstract, it should all work out

jade viper
#

Class is indeed abstract

#

Ty:)

#

But what if it wasn't?

rare scarab
#

then it would also be a valid type

jade viper
#

I'm writing a function that receives a streamable object (inherits from typing.IO) and does some parsing and uploading

#

The correct type hint then would be just stream: typing.IO?

rare scarab
#

either IO[str] or IO[bytes]

jade viper
#

So Union of those?

#

Either is acceptable

rare scarab
#

yes.

jade viper
#

Kay

#

Thank you!:)

rare scarab
#

maybe consider TextIO | BytesIO

jade viper
#

My company uses Python 3.8 ๐Ÿคก

#

Oh you mean the classes lol

rare scarab
#

use typing.Union instead of |

jade viper
#

Yeah, that's what I do haha

rare scarab
#

or use from __future__ import annotations or wrap it in quotes

jade viper
#

Ain't no way I could've used from __future__ import annotations this whole time

rare scarab
#

it doesn't work if you use typing.get_type_hints() or inspect.get_annotations()

rare scarab
#

Looks like typevar defaults broke the pydantic mypy plugin

rare scarab
#

microsoft/pyright#3907 seems to be the closest match, though I don't think it quite matches.

mighty lindenBOT
rare scarab
#

Oh well. Decorating my metaclass with @dataclass_transform(kw_only_default=True, field_specifiers=(Field, FieldInfo)) seems to be a workaround.

rare scarab
#

microsoft/pyright#5402

mighty lindenBOT
jade viper
#

Guys, how do I type hint a parameter that must be a value from an enum?

oblique urchin
jade viper
#

Simple as that?

rare scarab
#

Unless you mean any enum from any enum class

jade viper
#

Nope, a specific enum