type-hinting

[slender timber]

for simplicity's sake just forget that it exists

😄

[tranquil turtle]

what if someone has custom stdlib with typing module that serves completely different purpose?

in this case you cannot guarantee that typing.cast is noop

iirc, typing.reveal_type is not a noop at runtime, so you cannot optimize everything noop-ish from typing

maybe in the future typing.cast will be able to do type checks at runtime

[trim tangle]

that's generally undecidable beyond plain types like int, str and maybe tuples of such

you can't even check for list[str]

[tranquil turtle]

sometimes i want to write something like this: py assert isinstance(x, list[int]) to tell typechecker that x is an instance of list[int] (and perform check like isinstance(x, list) at runtime)
but mypy yells at me :(

[brazen jolt]

make a typeguard

[tranquil turtle]

too much characters to type

[trim tangle]

The thing is, you cannot verify that it's a list of ints

[thick saffron]

wouldnt a generator expression with sum() be even faster?

[brazen jolt]

I mean, it'd be O(N), but you can with a type guard, what do you mean?

[trim tangle]

It might only contain ints right now, but who knows what happens to it a few moments later.

Maybe someone else holds a reference to it and believes that it is list[int | str]

[brazen jolt]

hmm, yeah, that's true

that's so annoying, let's just implement a borrow checker lol

[tranquil turtle]

im pretty sure iterating over generator is slower than iterating over range(n) or repeat(None, n)
but getting rid of x = x.__iadd__(...) can help here

[tranquil turtle]

so, it is possible to perform isinstance(x, list[int]) check if there is only one reference to x, right?

[trim tangle]

that is true

but not very useful 😄

[tranquil turtle]

yeah

[thick saffron]

i meant:

[drifting ibex]

Anyone know how I might type hint this in the latest version of mypy or pyright:

from __future__ import annotations

from typing import Generic, TypeVar

T = TypeVar("T", int, float)

class Foo(Generic[T]):
    def __init__(self, arg: T):
        self.arg = arg

    def to_foo_int(self) -> Foo[int]:
        return Foo(int(self.arg))

    def to_foo_float(self) -> Foo[float]:
        return Foo(float(self.arg))


class Bar(Foo[T], Generic[T]):
    def bar(self):
        bar_int = self.to_foo_int()
        bar_int.baz()  # Cannot access member "baz" for type "Foo[int]"

    def baz(self):
        pass

How can I get to_foo_int to return a Foo[int] or Bar[int], depending on the self type? I essentially need a way to refer to Self but without the generic, so I can make the return type Self[int] instead of Foo[int].

pyright-playground link here: https://pyright-playground.decorator-factory.su/?gzip=H4sIALa5amQC_21Quw7CMAzc8xUWUyMFPqASDAzwAxELqqIUEhSpxFVqBvh6nKilvDxE8vnu7ItPeAVj_I1uyRkD4dpjIrAxIlkKGAchfObQvQ_xMs33LroUTgr0vXcHm4TQsJ6aaqEXCkIkBb5DS1KIU2eHAXaI1ag86kbWArjOzvMBIQYyphpc5xXYdKlBj_NcGV4xyjv4FS8dofGILKailLDc5CVHBppZnRyHi2X7xMxmUv4YlXM_rQr032xmT3Zjzi3_QdbqRsHfvC0TypbZl6GcgxMWw7dk8puzau2jejs-t19uPZ_xBIjW9gDbAQAA

[tranquil turtle]

If you are talking about latest version of pyright, you can completely get rid first three lines

It is not even correct code. It will not work at runtime

[oblique urchin]

not sure why you say that

[drifting ibex]

I only want static typechecking?

not sure what runtime has to do with it

[rustic gull]

that's because the return type of to_foo_int() is Foo[float], which doesnt have baz attr, you can try using typing.Self for the return type most probably and instead of py return Foo(...) ```py
return self.class(...)

[drifting ibex]

I don't think Self will work in this case because calling to_foo_int() on a Bar[float] would give back a Bar[float] instead of a Bar[int]

Self would work if there was some way to get that type but without generics, and then add them back on, i.e. Self[int] or Self[float]

[lament swan]

why's this complaining?

"List[Any]" not callable  [operator]mypy(error)
chunks = png.chunks()

@dataclass
class Png:
    ...

    @property
    def chunks(self) -> list[Chunk]:
        ...

class TestPng(unittest.TestCase):
    def _testing_png(self) -> Png:
        ...

    def test_list_chunks(self) -> None:
        png = self._testing_png()
        chunks = png.chunks()
        self.assertEqual(len(chunks), 3)

[dull lance]

chunks is a property, not a method

[lament swan]

ah damn

i forgot that it was a property

weird err msg, i thought it was complaining that png is List[Any], which it's not.

now i think it was complaining about chunks

makes sense

[trim tangle]

Unfortunately that's not possible. You need something called "higher kinded types"

[median ledge]

Say I have def foo(...) -> tuple[Any, ...]
If I do

bar: tuple[str, ...] = foo(some_arg)

What would the linter show as the type of bar?
I'm thinking tuple[str, ...], but it shows tuple[Any, ...].

It's also a possibility that it's due to my messy typing (overloaded self annotations, generics, ...)

[trim tangle]

What would the linter show as the type of bar?
type inference and such is generally not defined anywhere 🥲 it's up to the mercy of the type checker

pyright says it's tuple[Any, ...], mypy says it's tuple[str, ...]

[median ledge]

I'm using pylance 🥲

although I may be able to get rid of Any

[trim tangle]

pylance is in a strange relationship with pyright

but their inference should be the same

[median ledge]

yeah, ik

I think TypeVarTuple may work, but that means I have to update to 3.11 🫤

thanks btw

[oblique urchin]

no, it means you can use typing_extensions

[soft matrix]

You can use it on <3.11 with typing extensions

You just need to use Unpack

[median ledge]

don't give me reasons to procrastinate updating XD

[median ledge]

TypeVarTuple can't be bound yet 😭

[blazing nest]

Anybody else having issues with Pyight's --verifytypes marking TypedDict's with all NotRequired fields as partially unknown? It is obliterating my score and the output is close to useless

from typing import TypedDict, NotRequired

# For some reason I had to add @final to this
class Test(TypedDict):
    field: NotRequired[str]

reveal_type(Test.popitem)  # Type of "Test.popitem" is "(self: Test) -> tuple[str, Unknown]"

[wind mason]

I keep having mypy issues when I run mypy .. It fails saying module is installed, but missing library stubs or py.typed marked for my custom package. Does anyone know the cleanest way to deal with this? I wasn't sure if it would be better practice to write stub files in the module directory or just skip them in the toml file.

[tranquil turtle]

add py.typed marker

[median ledge]

Say I have a descriptor class and some subclasses:

class Foo(Generic[T]):
    def __get__(self, obj, owner) -> T: ...

class Bar(Foo[int]):
    ...

Say I have a function spam that takes a descriptor instance and returns the result of .__get__.
Is it possible to annotate this?
For example, spam(SomeClass.bar) # int

[soft matrix]

!d typing.Protocol

[rough sluice]

typing.Protocol

class typing.Protocol(Generic)```
Base class for protocol classes. Protocol classes are defined like this:

```py
class Proto(Protocol):
    def meth(self) -> int:
        ...
```  Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing), for example...

[soft matrix]

you can probably do it using this

[median ledge]

How so?

Wait, I think it clicked

Actually, it didn't 😅

[soft matrix]

from typing import Protocol

class DescrGet[T](Protocol):
    def __get__(self,  obj, owner) -> T:
        ...

def spam[T](x: DescrGet[T]) -> T:
    ...

class Foo[T]:
    def __get__(self, obj, owner) -> T: ...

spam(Foo[int]())```

something along these lines at least

you need overloads on the Foo class i think

[strange ledge]

Does anyone have a quick explanation why in PEP585 there are various types that it says to import from collectiongs.abc instead of typing, and that "importing them from typing" is deprecated. Among those is Callable, Iterator, Generator which I still see most people just import from typing?

[soft matrix]

wheres @trim tangle when you need them for the plug

!pypi flake8-pep585

[rough sluice]

flake8-pep585 v0.1.7

flake8 plugin to enforce new-style type hints (PEP 585)

[soft matrix]

that will do the conversion automatically

but the types were better suited where they are now because they make more sense as things that arent strictly typing related

[grave fjord]

Afaik flake8-pep585 won't automatically upgrade the syntax

You'll need pyupgrade for that

[median ledge]

I think that would even work even without protocols.
Is there a way to make it work with TypeVarTuple?

In reality, spam would take *args and return values in the same order.

[grave fjord]

collections.abc.Callable was broken on python 3.9.0 and 3.9.1 so that's why some people keep the old typing import

[soft matrix]

no you cant do that currently

[median ledge]

😭

Why doesn't TypeVarTuple support the options of TypeVar?

[lime fern]

Does anyone know how to type-hint multiprocessing objects? eg. multiprocessing.Value, multiprocessing.Lock.
Pylance gives me type errors when I use the associated class. Currently I'm using protocols as a workaround, but I have to explicitly set # type: ignore for all variable assignments of multiprocessing objects.

I made a similar post in #async-and-concurrency. Trying my luck here to see if any of you folks have had the same situation.

[trim tangle]

Yep, though pyupgrade doesn't catch quite everything

[trim tangle]

I am in ur walls

[soft matrix]

Because the pep was already pretty massive

A type level Map operator should be coming soonish I think

[grave fjord]

what does it miss?

[trim tangle]

I don't remember exactly, but stuff like Iterator and ContextManager maybe?..

[grave fjord]

and AbstractSet

probably doesn't do renames that need an from typing import ContextManager -> from collections.abc import AbstractContextManager as ContextManager

[rare scarab]

What's with the Abstract prefix? Isn't all the classes in collections.abc abstract?

[oblique urchin]

it's to distinguish typing.Set and typing.AbstractSet

typing.AbstractSet actually corresponds to collections.abc.Set

[next iris]

Is there a way to generate stub files from a .so file I generate via rust's pyo3?

[oblique urchin]

I think mypy's stubgen might have some support for that?

[next iris]

https://mypy.readthedocs.io/en/stable/stubgen.html - Yeah, trying to work that out

[oblique urchin]

it's this file https://github.com/python/mypy/blob/master/mypy/stubgenc.py

[next iris]

It's a damn internal module tho and we're all pyre @ Meta 😦

[oblique urchin]

not sure how good it's going to be at extracting type info

I think just invoking stubgen should do it

[next iris]

Ya, it's just building mypy that's painful internally

Think this is a tomorrow problem

[soft matrix]

Pyo3 has a pr for it iirc

[next iris]

Yeah saw that, this will be awesome!

[proven narwhal]

Is there a way to enforce constraints between generics? i.e. have a TypeVar that is dependant on another TypeVar e.g. Parent = TypeVar("Parent", ParentClassA, ParentClassB); Child = TypeVar("Child", ChildClassA, ChildClassB) where ChildClassA is only valid when ParentClassA is in use.

[soft matrix]

you can but its a pain and youll need to annotate every method's self type

[proven narwhal]

What does one call this type of constraint? i'm struggling to find the keywords to google for

[soft matrix]

conditional types i think?

[proven narwhal]

My current approach has been to if isinstance(parent): assert isinstance(child)... which seems way too hacky

[soft matrix]

you can do it at type time

but it wont apply to the body of the function cause overloads dont do anything to the runtime function body

i think the best way to deal with this is to just use 2 different subclasses

[proven narwhal]

hah, I'm actually trying to merge two classes that has 99% similar logic but operate on different class groups. from a duck typing perspective they are mostly the same, at least from the logic's point of view.

there are some small differences which I'd like to catch via type hints

[carmine phoenix]

whats the type hint for returning an instance that is a subclass of a given class?

[soft matrix]

you cant do subclass of a given class really

[carmine phoenix]

wdym?

like maybe with NewType?

[oblique urchin]

sounds like you want something like def func[T](cls: type[T]) -> T: ...?

[carmine phoenix]

nope, I have a function that returns a instance of a subclass of an ABC class (and the whole reason why the main class is abstract is because there are multiple implementations)

[oblique urchin]

then just use that ABC in your annotations?

types in annotations include their subclasses

like if you have -> int you can legally return a subclass of int

[carmine phoenix]

true, but wouldnt it be nicer if you can specify thats its not int, but a subclass of it? even more specifc

[carmine phoenix]

I suppose I could, but the abstract class has vague return types, like Iterable[str] or Collection[str], and the implementation might have list[str] or maybe set[str

[heady flicker]

I am using pyright (latest) through pylance and am a bit confused with its protocol handling. Take a look:

from typing import Any, Protocol


class P(Protocol):
    p: dict[str, Any] | None


class C:
    p: dict[str, Any]


def foo(s: P):
    return s


foo(C()) # typechecker error: dict[str, Any] is incompatible with dict[str, Any] | None

I can't for the life of mine understand why is there a problem here and how to type hint it properly. Variance should not be affected by | None but it is. Mypy has the same problem.

I'm missing something but not sure what.

UPDATE: because protocol can set it to None. Okay, got it :D

[tranquil turtle]

I think you can declare p as property returning dict|None
In this case property returning dict in subclass should be fine

If it is in fact a property and not an attribute

[heady flicker]

Oooh that makes so much sense!

I thought of doing it using TypeVars but for some reason generics are bad with protocols. Does anybody know why?

[soft matrix]

wdym bad with protocols?

[heady flicker]

Let me explain

[tranquil turtle]

Protocols can be generic over only covariant vars (or contravariant, i dont remember)

[trim tangle]

hm?

[oblique urchin]

that's not true

[trim tangle]

yeah that's not true

[heady flicker]

from typing import Any, Callable, Concatenate, Generic, ParamSpec, Protocol, TypeVar

T_val = TypeVar("T_val", bound=int | None)


class Proto(Protocol, Generic[T_val]):
    value: T_val


T = TypeVar("T")
Self = TypeVar("Self", bound=Proto)
P = ParamSpec("P")
Meth = Callable[Concatenate[Self, P], T]


def meth_decorator(meth: Meth[Self, P, T]) -> Meth[Self, P, T]:
    def wrapper(self: Self, *args: P.args, **kwargs: P.kwargs) -> T:
        reveal_type(self.value)
        return meth(self, *args, **kwargs)

    return wrapper

Here. On line with reveal_type we should see something in the lines of "T_val" or *int but we see "Unknown" in pyright.

[tranquil turtle]

Hmm... I will try.

[oblique urchin]

interesting, mypy also gives Any. Probably something to do with how bounds work

[tranquil turtle]

https://mypy-play.net/?mypy=latest&python=3.11&gist=024e73f63ed574413d478c524660fb2f

[heady flicker]

Do you think it's a common bug or do you think it's intentional?

[oblique urchin]

I doubt it's common. It's probably better if it produced int, but there might be complications

[oblique urchin]

seems like that check is a bit too strict there

if you do ```class X(Protocol[T]):
def meth(self, arg: T) -> T: ...

but I guess it enforces that unused typevars are always covariant, possibly by accident

[soft matrix]

I think if you use a covariant type var with a property for the val then you should be able to specialise the Proto class in the bound and everything should work as expected

[heady flicker]

I guess he's right. It really does look like higher kinded types

[soft matrix]

from typing import Callable, Concatenate, Protocol


class Proto[T_val: int](Protocol):
    @property
    def value(self) -> T_val: ...


def meth_decorator[Self: Proto[int], **P, R](meth: Callable[Concatenate[Self, P], R]) -> Callable[Concatenate[Self, P], R]:
    def wrapper(self: Self, *args: P.args, **kwargs: P.kwargs) -> R:
        reveal_type(self.value)
        return meth(self, *args, **kwargs)

    return wrapper```if you really need this should work

[jade viper]

Hey guys, what is the cleanest way to create a "wrapper" cached version of a function? What I mean is:

Say I have the following function:

def foo(bar: int) -> bool:
  return bool(bar)

And I want to have a cached, alternative version of this function:

from functools import lru_cache

@lru_cache(maxsize=None)
def cached_foo(*args, **kwargs):
  return foo(*args, **kwargs)

This doesn't seem very clean, but since this is part of a large project I want the cached version to take in "generic" arguments but still use the parent function type hints. Is this possible?

[chrome hinge]

I think you can do that with a ParamSpec: https://docs.python.org/3/library/typing.html#typing.ParamSpec

[soft matrix]

You can't really afaicr

There's just no way to have it work with descriptors

[heady flicker]

Are you using a prototype of that typevar PEP?

Or does pyright already support that syntax?

[soft matrix]

Its in 3.12 now and Pyright supports it yeah

[heady flicker]

Wooow. I didn't know that it has already been accepted. Nice!

[brisk hedge]

Self cannot be generic as it's a type variable already. Your snippet asks for higher-kinded types, which are not a feature of python's static type system

[heady flicker]

Thanks! Eric has mentioned that too ;)

[wild zealot]

I am building a project at my job that requires type checking and my choices are between Beartype and Pydantic. I want to use Pydantic, however, it is still in alpha version. Would it be a good idea to use it or should I use Beartype for now and come back to Pydantic after it is officially released?

[heady flicker]

Pydantic is not in alpha version. It's a full blown and very mature library. Note, however, that its main purpose is not type checking.

Read its docs for more.

[trim tangle]

What do you mean by "type checking" and why do you think you need one of these tools?

[wild zealot]

Pydantic v2 is alpha
I can't use v1

[wild zealot]

to help catch any incorrect types the user tries to input

[spring ice]

@frank glade

In the above case:

• x is an instance of class Foo.
• Foo is an instance of the type metaclass.
• type is also an instance of the type metaclass, so it is an instance of itself.

https://www.pythontutorial.net/python-oop/python-type-class/

[frank glade]

but UnionType should be special, it's instantiated implicitly via operation on types

[spiral fjord]

It still functions the same, int | str is an instance of UnionType which is an instance of type which is an instance of type which ...

[delicate swallow]

Is there a preferred way of encoding algebraic data types in Python? or is just inheriting from a base class enough?

[jade viper]

What would be the right way to type hint the following?

from functools import lru_cache

def translate_symbol():
  pass

translate_symbol_with_cache: Callable = lru_cache(maxsize=None)(translate_symbol)

Is just Callable enough?

[acoustic thicket]

you shouldnt need to type hint it at all

[grave fjord]

you'll need to add type hints to translate_symbol

[grave fjord]

lru_cache is missing ParamSpec in typeshed

[grave fjord]

should look like this: https://mypy-play.net/?mypy=latest&python=3.11&gist=de1aad3526ca1eec4d442cb4fb97d802

https://github.com/python/typeshed/issues/6987

[jade viper]

Would this fix linting? The linter doesn't recognize the original function's linting at all

[grave fjord]

Which linter are you using and what error does it say?

[grave fjord]

You'll need a totally different type annotation because translate_symbol is different than what you posted

[jade viper]

Yeah, I wanted the cached version to just "copy" the original function's parameters :/

[grave fjord]

@jade viper you can "fix" the type of lru_cache using a protocol

https://mypy-play.net/?mypy=latest&python=3.11&gist=eb3f8ce1b9f79d5e5a3f7cb10062dc6c&flags=strict

[jade viper]

That's great, thank you so much!

[median ledge]

If I know the types in an sqlite.Row object (say I selected only an str and an int column), can I annotate that, or do I have to cast or type ignore?

For example

rows = await cursor.fetchall()
return tuple(map(tuple, rows)) 

# this returns tuple[tuple[Unknown, ...], ...]
# I want tuple[tuple[*Ts], ...]

[young zealot]

Is there a way to make sure mypy understands that v is no longer optional after the if check(v): condition in this example:

def check(v: str | None):
    return (v is not None)

def required(v: str):
    print(v)

def test(v: str | None):
    if check(v):
        return
    required(v)

[grave fjord]

TypeGuard

[oblique urchin]

I think TypeGuard won't exactly work here because it doesn't narrow in the negative case

[young zealot]

yeah I tried making it def check(v: str | None) -> TypeGuard[str]: and it didn't change

[median ledge]

Is there a way to make pylance report using : list without specifying the generics?

That is, implicit list[Unknown]

Some older code is full of that 😫

[tranquil turtle]

class Truthy(Protocol):
    def __bool__(self) -> Literal[True]: ...

def check(x: object) -> TypeGuard[Truthy]: ...

def test(v: str | None):
    if not check(v):
        return
    reveal_type(v) # str
    # do string stuff

class Truthy(Protocol):
    def __bool__(self) -> Literal[True]: ...
``` does this even work?

[soft matrix]

Unions or base classes

[trim tangle]

I would pick a union of dataclasses because the type checker knows about its exhaustiveness

[primal lantern]

i'm trying to check for a runtimewarning in this code and I'm not sure how to exactly implement it.

    global x_sp
    global y_sp
    global source
    global y
    global l_b, r_b
    y_sp = sp.sympify(new)
    new_np = sp.lambdify(x_sp, y_sp, 'numpy')
    new_eq = new_np(x)
    y = new_eq
    source.data = { 'x': x, 'y': new_eq }
    y_init = get_coeff(y_sp, a_slider.value)
    taylor_coords.data = { 'x': x, 'y': y_init }
    get_ROC(y_sp)
    compute_error(x, y_init, new_eq)
    if l_b is None and r_b is None:
        l_b.visible, r_b.visible = False, False

I know that for example, if new equals x**x, there will be a Run. warn. exactly in this line:
new_eq = new_np(x)
so how can i check for it? Thanks

[austere rapids]

Hi people ! I made a class that will raise TypeError at runtime based on type annotations : https://github.com/6r17/madtypes ; I'm looking for feedback and I hope this package might bring some sanity when needed !

[soft matrix]

typing that actually raise TypeError

will raise ValueError

what are these lies 😉

[austere rapids]

are they ? : o did i misunderstand something ? 😮

[trail kraken]

I think... the question boils down to: In what situation does it raise TypeError as described? Because your example saysit raises ValueError instead of TypeError.

[austere rapids]

Ohh my bad !; I only raise TypeError ; it's a set overwrite that checks for type and raises TypeError whenever the type mismatch the annotation

Thanks both of you ! I just realized

[trail kraken]

Hi. Are there common ways to generate type stubs for an API implemented in Boost.Python?

I see mypy and pylint can work with dynamic libraries, if I understand correctly.. This particular API is statically linked into an application, and only available from inside the embedded interpreter. I would like to make static type checking available outside of it if possible.

I have access to the source code. As a fallback, I am thinking of compiling a dynamic library from it. I would like to exhaust alternatives first, because the build system is very convoluted.

As a fallback to the fallback, I can spend hours doing it by hand, I suppose...

[slender timber]

how can i do this correctly?

if TYPE_CHECKING:
    STRUCT = c.ExprAdapter[str, str, str, str](...)

STRUCT = c.ExprAdapter(
    c.GreedyString("ascii"),
    lambda obj, *_: obj.rstrip("\0"),  # type: ignore
    lambda obj, *_: obj + "\0",  # type: ignore
)

this works

if TYPE_CHECKING:
    STRUCT: c.ExprAdapter[str, str, str, str]
else:
    STRUCT = c.ExprAdapter(
        c.GreedyString("ascii"),
        lambda obj, *_: obj.rstrip("\0"),  # type: ignore
        lambda obj, *_: obj + "\0",  # type: ignore
    )

would have to make do ig

[tranquil turtle]

you dont need # type: ignore inside of if not TYPE_CHECKING:

[slender timber]

uh yea

[slender timber]

PEP696 when?

[viscid spire]

!pep 696

[rough sluice]

**PEP 696 - Type defaults for TypeVarLikes**

Link

Status

Draft

Python-Version

3.12

Created

14-Jul-2022

Type

Standards Track

[soft matrix]

Next year

Oh that version should probably say 3.13

[young zealot]

Are there any pros/cons/gotchas lists of type hinting "in code" vs "in comment"? Like def test(a: int) -> str: vs def test(a): # type: (int) -> str?

[oblique urchin]

the latter is basically deprecated

we'll probably drop support in mypy at some point

[young zealot]

Well that’s an important one 😅 at the moment I kind of like the commenting better, but I can see why it would be deprecated

[viscid spire]

Why would you like commenting more?

The typehints then don't show up documentation, and they aren't picked up by some type checkers no?

[tranquil turtle]

does it mean that you also will deprecate type comments (such as #type:ignore) ?

[oblique urchin]

# type: ignore has to stay

but there's no good use for Python 2-style type comments any more

[tranquil turtle]

are there any standard error names that can be passed in #type:ignore[...] ?

[heady flicker]

Nope as far as I know. You could consider mypy's set a standard though.

[slender timber]

pyright doesn't use that set

[median ledge]

I use pylance and tyoe: ignore[generalTypeIssues] works at least

[slender timber]

should __new__ be type hinted to return the class itself or Self?

[soft matrix]

Self

[lethal solar]

is this type of type hinting "ok" or should it be avoided ?
tmp = set[str]() instead of tmp: set[str] = set()

[buoyant swift]

i don't think typecheckers understand the first one

[lethal solar]

they do

[buoyant swift]

seriously?

[lethal solar]

at least pyright do

[buoyant swift]

oh yeah it does

seems mypy does too

[lethal solar]

it looks like a cool trick to avoid duplication (set[...] = set(...)) but not sure if it is a good practice

[soft matrix]

I like using it for things that aren't literals

[trail kraken]

I think I would use it, if pre-3.9 support is not necessary.

Now I kinda wonder if they compile to the same bytecode.

[trim tangle]

no, they are very different

Python always has to assume that you can redefine set with a completely arbitrary object

So set[foo](bar) will first index the global/builtin set with foo and then call the resulting object with bar. But set(bar) just calls the global/builtin set with bar

if that's what you meant

[trail kraken]

Mm. Right. There has to be an extra lookup, because set can potentially mean something different.

I didn't notice that possibility.

So this means there is a slight runtime trade-off with that syntax?

[trim tangle]

yes

though, of course, it is very minor

[lethal solar]

well, if it's not considered as a bad practice, this looks pretty useful (even if it is only a minor detail)

[lethal solar]

how to correctly overload this function ?

@overload
def extract(
    query: T,
    choices: Mapping[K, T],
    processor: Callable[[T], P] = ...,
    scorer: Callable[[P, P], int] = ...,
    limit: int = ...,
) -> list[tuple[T, int, K]]: ...
@overload
def extract(
    query: T,
    choices: Iterable[T],
    processor: Callable[[T], P] = ...,
    scorer: Callable[[P, P], int] = ...,
    limit: int = ...,
) -> list[tuple[T, int]]: ...
def extract(
    query: T,
    choices: Iterable[T] | Mapping[K, T],
    processor: Callable[[T], P] = ...,
    scorer: Callable[[P, P], int] = ...,
    limit: int = ...,
) -> list[tuple[T, int]] | list[tuple[T, int, K]]: ...

If choice is a dictionnary, the return type is a list of 3-tuple but the last element is the Key
If it's only an iterable that is not a mapping, the return type is a list of 2-tuple

but I get this :
Overload 1 for "extract" overlaps overload 2 and returns an incompatible type

btw it's this function that I try to type:
https://github.com/seatgeek/thefuzz/blob/04deff5cc280e034d0344dc9f8731399ecd207cd/thefuzz/process.py#L124

[rough sluice]

thefuzz/process.py line 124

def extract(query, choices, processor=default_processor, scorer=default_scorer, limit=5):```

[oblique urchin]

This is not type-safe because a Mapping is an Iterable

so you cannot type this safely

[fading temple]

how i should typehint a classmethod? with from __future__ import annotations or from typing import Self?

[lethal solar]

yes that's the point
so there is no way ? that's annoying, but ok I will deal with it

[oblique urchin]

the interface is inherently unsafe so it makes sense that type checkers will complain

your best bet is to type ignore it, but you'll hide a real type unsafety

[oblique urchin]

those are different tools and both can be useful

I would use Self where it makes sense

[acoustic thicket]

from typing import TypeVar
class A:
    def f(self, x: int) -> int:
        return 2

T = TypeVar("T", int, tuple[int, int])
class B(A):
    def f(self, x: T) -> T:
        if isinstance(x, int):
            return 2
        return (1, 2)

pyright gives me this error

Method "f" overrides class "A" in an incompatible manner
  Return type mismatch: base method returns type "int", override returns type "T@f"
    Type "int* | tuple[int, int]*" cannot be assigned to type "int"
      "tuple[int, int]*" is incompatible with "int"

but this is type safe right..?

[fading temple]

not sure if use a classmethod tho

class Autor(models.Model):
    # ...
    @classmethod
    def obtenir_o_crear(cls, nom: str, cognom: Sequence[str]) -> Autor:
        slug = slugify(nom + " ".join(cognom) if cognom else "")
        try:
            autor = Autor.objects.get(slug=slug)
        
        except Autor.DoesNotExist:
            autor = cls(nom=nom, cognom=" ".join(cognom) if cognom else None)
            autor.save()
            
        return autor```

[oblique urchin]

yes, that seems like a false positive to me

[oblique urchin]

Self looks incorrect there as the method does not necessarily return an instance of cls

[fading temple]

oh! i got the difference

then my choice is correct

anyways, it is correct to use classmethod there?

i never used a classmethod before and i wanted to try it

[oblique urchin]

looks like a reasonable use for a classmethod yes

[fading temple]

okay thanks!

[bitter maple]

hello everyone! could you help me please... dont know how to solve this problem (from aiogram import Bot
ModuleNotFoundError: No module named 'aiogram')

[gloomy hawk]

install the aiogram library using pip

[bitter maple]

i've already installed the library, but the error is still unresolved

[gloomy hawk]

weirdly enough, I had the same problem with aiogram, your program will still work just you'll have that warning/error

[bitter maple]

okay, thank you!

[lethal solar]

can Protocol allow to set direct attributes or set a property ?

class Foo(Protocol):
  foo: str

class Bar:
  @property
  def foo(self) -> str:
    pass

here Bar is incompatible with Foo

this seems logic but.. Bar().foo is a str as well

[heady flicker]

Try property with a setter

Or try using the property in the protocol as well

[lethal solar]

oh yes it works thx

[rustic gull]

Is it possible to get ide autocompletion on a config class that reads an .env file?

or do i have to manually add type hints

[soft matrix]

you probably have to do it manually

though if you were feeling cool you could probably just use a regular python file for it and it would probably autocomplete correctly

[rustic gull]

i wanna set "required" variables though, so thought it made more sense to make a class vs spamming/bloating if statements

the class is prob now more lines if i gotta add type hints

I just remembered someone making a .yaml reader class that had autocompletion without type hinting - prob my bad memory

[slender timber]

Can't typing have some default TypeVars exported? So I can avoid writing this all the time

T = TypeVar("T")

Would also like a default ParamSpec exported as P

[viscid spire]

https://peps.python.org/pep-0695/

[slender timber]

This syntax isn't backwards compatible

[viscid spire]

Neither would adding exports

[slender timber]

Exports can be added to typing_extensions

Nothing special really

[viscid spire]

there are type variables already in typing but they are commented that you shouldn't import them I believe

so it seems that is already a decision that has been made

[slender timber]

If its not exported it's an implementation detail and it can change any time

So better export it from typing_extensions

[viscid spire]

I'm not sure what you mean by export tbh

[oblique urchin]

typing.T actually exists and we're not going to remove it

https://github.com/python/cpython/blob/595ffddb33e95d8fa11999ddb873d08e3565d2eb/Lib/typing.py#L2479

[rough sluice]

Lib/typing.py line 2479

T = TypeVar('T')  # Any type.```

[trim tangle]

use AnyStr 😎

everything is a string anyway

[oblique urchin]

that reminds me, I want to propose formally deprecating AnyStr

[trim tangle]

Why?

Ah I guess it's kinda redundant with the new syntax

[oblique urchin]

TypeVars with constraints are a confusing thing that I don't want to expose people too much to

AnyStr is mostly useful for Python 2/3 compat

which we no longer need

plus with PEP 695 you shouldn't be using "reusable" TypeVars

[trim tangle]

I guess there's re.Pattern and IO stuff

But yeah, reusable typevars

Spooky

whereas I am just a ducky

[oblique urchin]

the name is also confusing, people think it's a union

[trim tangle]

The value of reuse is often exaggerated tbh

There's always a balance of reuse and coupling (it's not always clear whether two things actually refer to the same concept) but maybe some other stuff

This might also apply to non-code things like standards. e.g. PEP484

But don't quote ducks

[grave fjord]

Do you know why PEP 695 deprecated TypeAlias but not TypeVar and ParamSpec?

[oblique urchin]

TypeVar is still being used

[soft matrix]

will the old form ever be deprecated though?

[grave fjord]

Why would you use the old form when you can use the new form

[oblique urchin]

The new syntax literally creates instances of typing.TypeVar

But yeah maybe we should deprecate support for the old syntax in type checkers at some point

I don't know why Eric chose to deprecate only TypeAlias, maybe because it's still relatively new and less pervasive than manually creating TypeVars

[grave fjord]

How do you use it with NewType?

[oblique urchin]

use what?

[grave fjord]

Oh you can't use generics with NewType anyway https://github.com/python/mypy/issues/3331

[grave fjord]

Generics with NewType

[lethal solar]

from dataclasses import dataclass

class Parent:
    pass

class Child(Parent):
    pass

@dataclass
class Foo:
    bar: list[Parent]

@dataclass
class ChildFoo(Foo):
    bar: list[Child]

how to correctly type this, because Child is incompatible with Parent here

[acoustic thicket]

you cant use list, will tuple work for your usecase

[lethal solar]

yes, ok

[soft matrix]

or just sequence

[slender timber]

https://github.com/pygobject/pygobject-stubs/issues/144#issuecomment-1566689835

seems like a bug in mypy?

[jade viper]

What are the differences between types.FunctionType and typing.Callable?

[oblique urchin]

the former is specifically the type for Python functions (created with def), and it's not very useful for type hinting

the second refers to anything you can call (so also e.g. builtin functions, types, objects with __call__)

[jade viper]

I see, thanks! So what would be the correct type hint in this case: translate_symbol_with_cache: SomeTypeHint = lru_cache(maxsize=None)(translate_symbol)?

[oblique urchin]

something like Callable[[SomeArgument], SomeReturnType]

if you also want to expose the methods that lru_cache adds (like cache_clear or so) it gets more complicated

[jade viper]

I don't, my actual problem is that with either type hint my linter (pylance on vscode I think) is recognizing it as a variable and not as a function :/

It recognizes the return type for some reason

But not the parameters

So I was trying to type hint it as a function to force the linter to "look" for the parameters, if that makes sense?

[soft matrix]

You can use a callable protocol and cast the function to that type but that also has problems

This whole aspect of pep 612 is a bit of a mess and no one's sure how exactly to fix it

[jade viper]

I see

I think there may be something wrong (as far as type hints go) with lru_cache, since even when using it as a decorator my linter will show "lru cache wrapper" instead of the actual parameters

[soft matrix]

It's because the choice to type it went with adding the important methods like cache clear to the method hence why it returns an instance of a private type time only type

If you really care you can probably just change the signature as a temporary solution to F: Callable -> Callable[[F], F] or something

[jade viper]

That's alright

Thank you for everything guys!:)

[earnest tundra]

Why is typehint used for?

[trim tangle]

Check these links out
<#type-hinting message>

[wet pilot]

heyo friends, if anyone can take a look at #1112842519705833663 message that'd be grand, it's a bit of a headache figuring this out so I'd appreciate the help :)

[young zealot]

Is there a way to make mypy report assigning attributes in a class that are not declared in __init__ without manually maintaining __slots__?

[tranquil turtle]

It should report it by default

I think

[young zealot]

doesn't seem to for me 🤔

[pure raptor]

pylance/pyright has been really slow for me recently, does anyone have any idea why

like I save the file / make a change it takes like 5-10 seconds for it to register and pyright to either stop complaining or complain

and vsc bar just shows "N file to analyze..." while that is

[rare scarab]

maybe its your anti-virus?

[pure raptor]

mayyybe? but how would I check?

[rare scarab]

add a scan exception for your project directory.

[pure raptor]

hmm dont think that did anything

[grave fjord]

Can you create a https://mypy-play.net demo?

[young zealot]

class Test:
    def __init__(self) -> None:
        self.a = 1

    def test(self) -> None:
        self.b = 2

if I add __slots__ = ['a'] it will complain about self.b = 2

[grave fjord]

https://mypy-play.net/?mypy=latest&python=3.11&flags=strict&gist=78cac5e2f118f7b81d5daaf2cbdcb1e4

Interesting

@young zealot one way to do it is to use a slotted dataclass

https://github.com/python/mypy/issues/10552

[young zealot]

What is a "slotted dataclass"?

oh just like @dataclass(slots=True)?

in theory, is there any downside to just like making everything a dataclass? I have not dug into them much other then using them for simple cases

[cunning plover]

it is good idea to use more pydantic BaseModel as very often data often carrier. (as more feature rich alternative to dataclass)
but it is not fitting all usage cases. because sometimes u need just simple variables
sometimes u need custom classes and u don't need all of the pydantic functionality

it depends on meaning u wish to translate with your code basically and logic

[rustic bay]

wow, a whole channel dedicated to the thing I joined this discord to ask about lol that is awesome..
Disclosure, I'm really new to python, but I'm trying to do one more refactor of an open source project I'm wanting to release in the next day or two. I welcome any thoughts / suggestions, especially if I'm doing something silly.
After a couple hours of arguing with ChatGPT and Google mostly giving me ancient (10+ years in tech...) information,I simply don't know if this is possible, reasonable, or even a good idea.

I'm importing a function from a 3rd party library that has a rather complex Union type for the return value (it "loads" a thing). When I assign that return to a variable, I get all the type hint info as expected. However, I'm passing that into another function (does some "processing" on the thing). That processing function should only accept the exact type returned by that load function. This processing function is the one most likely to change or be updated by other contributors, so I'd like to get the best type hinting possible in there, including in VS Code. Is there any way to "capture" that return type definition to use as a type hint in my processing function parameter definition? It seems like the prototype of the loading function should be available?

TLDR: Can you get/use the type information from one function as part of the definition of another's parameters? (for VS Code AND runtime)

---

As an additional note, the "loading" function is only called once, but the "processing" function might get called 10's of thousands of times and represents about 90% of the overall runtime. So I'm fine with something like runtime type checking anywhere except in the processing function, need to keep it as performant as possible.

[soft matrix]

i can think of a few ways to do it

1stly you might just able to convince the 3rd party libraries developers to export the return type for the function in the form of a type alias

2nd idea if that doesnt work involves a bit of decorator magic with something to the effect of ```py
def ensure_correct_input[LoadReturnType, ProcessReturnType](load_function: Callable[..., LoadReturnType]) -> Callable[[LoadReturnType], ProcessReturnType]: ...

type SomeMassiveUnionYoullBeWarnedIfOutOfDate = int | str

@ensure_correct_input(3rd_party_function_loads)
def processor(arg: SomeMassiveUnionYoullBeWarnedIfOutOfDate) -> ProcessedType: ...

but this isnt as good imo

[rustic bay]

Hmm so in that 2nd version, would the actual decorator code get executed each time the function was called? Unfortunately changing the library doesn't seem realistic right now.

The more I look at it, I'm not sure exactly what your example is doing actually... Is the idea to just replicate the same type that the loading function returns manually then check if that type matches what the loading function is actually spitting out?

And I really appreciate the thoughts and suggestions by the way.

[soft matrix]

no the ensure_correct_input should just return the inner function in place

the actual impl looks like

def ensure_correct_input(load_function):
  def inner(processor_function):
    return processor_function
  return inner

and hence should have no runtime impact

[rustic bay]

Dang, ok, so clearly I don't understand decorators (especially in python) all that well. I was heading down a somewhat similar path before, but doesn't this still mean I'd need to import the types and define the union type manually? I guess I was hoping to be able to do something like type LoadingReturnType = magic_way_to_get_return_type_from_function(actual_loading_function)

[young zealot]

Isn't it just simply a type alias?

type LoadedThing = int | float | str | MyClass
def load(...) -> LoadedThing:
   ...
type ProcessedThing = SomeClass
def processor(thing: LoadedThing) -> ProcessedThing:
   ...

[rustic bay]

I'm afraid I might have over complicated and over simplified different parts of my question, going to mock up a better example, one sec

Ok, try to ignore some of the prior explanation, I think this distills down where I'm struggling a bit better. Also can ignore the union type portion, this is just a more basic (hyper simplified) example:

from myloader import load_ai_model_wrapper
from something import millions_of_widgets

def do_the_processing(model, widget):
    # do something with model and widget, but I want
    # these to be typed properly in this function
    pass

if __name__ == '__main__':
    model = load_ai_model_wrapper()
    for widget in millions_of_widgets:
        do_the_processing(model, widget)

[young zealot]

In myloader define the type that load_ai_model_wrapper returns so it can be imported, then use that type in the processing func? Something like:

# In myloader
type Model = ModelA | ModelB | ModelC
def load_ai_model_wrapper() -> Model:
    ...

# In other script
from myloader import load_ai_model_wrapper, Model
from something import millions_of_widgets, Widget

def do_the_processing(model: Model, widget: Widget):
    # do something with model and widget, but I want
    # these to be typed properly in this function
    pass

if __name__ == '__main__':
    model = load_ai_model_wrapper()
    for widget in millions_of_widgets:
        do_the_processing(model, widget)

[rustic bay]

Thanks, I think this is basically the approach I'll go with for now. I actually am a little surprised there isn't a clean way to pull that information off the function prototype/signature. Guess I was using TypeScript for a bit too many weird things before.

[young zealot]

Is there a way to define a type that is something that conforms to multiple protocols?

[trim tangle]

class Intersection(Protocol1, Protocol2,  Protocol):
    pass

[young zealot]

Oh I meant type hint*. For example this scenario: https://gist.github.com/mypy-play/d80f7fa6c4c7361a5cb8da672ca3f039. I was hoping for something more like myVar: Intersection[Protocol1, Protocol2]

[soft matrix]

doesnt exist no

[young zealot]

dang

[cunning plover]

Sounds like u need guarding typing to narrow down union object to exact type

Protocol is option too

I would say u need Protocol and guarding narrow down perhaps

[cunning plover]

https://docs.python.org/3/library/typing.html#typing.TypeGuard
Use type guard, to simplify third party union to some single type object

And then input into your own protocol

It will make protocol working best to verify, u a assigning into it object that truly satisfies protocol

Protocol could be not even used at this point, but it should be used to minimize effect of third party code ripples across your code

With protocol u will see incompatibility errors in one place

Obviously... Best to isolate functionality of this dangerous third party, into your own class that inputs and outputs only your app code types

That will isolate code ripples from this dangerous object to single class, that u can easily replace with mock-spies if necessary for unit testing

Technically, if u isolate it into your own class, protocol will be already not necessary may be.

[celest iron]

Is there a way to type a generator that returns objects of types in a specified order, similar to a tuple?

def my_generator():
  yield 1
  yield "two"
  yield 3

i'm wondering if I can type exactly int -> str -> int

likely, no but worth a shot heh

[trim tangle]

sadly no

[celest iron]

are there any suggestions on what that type could look like

the use-case for generators beyond just iterators is pretty nifty, with even the standard library agreeing with @contextmanager

def meth():
  resp = yield transform(req)
  yield transform(resp)

this pattern is interesting to me and for me to get perfect typing, the better way would be to return as the last value and maybe decorate the generator so that u don't have to try and catch..

[rare scarab]

httpx uses that to support sync and async auth handlers

you can yield as many times as you like.

[celest iron]

i wonder what they did for their type then

[rare scarab]

Generator[Request, Response, None]

[celest iron]

do u have the file, i'm trying to find it

https://github.com/encode/httpx/blob/dedd37b9ce5b68182120c47225a823c4c7202a3f/httpx/_auth.py#L133
found it

[rough sluice]

httpx/_auth.py line 133

def auth_flow(self, request: Request) -> typing.Generator[Request, Response, None]:```

[celest iron]

whoa, cool bot

[slender timber]

I see a crime here

[rare scarab]

Who?

[slender timber]

Import from typing directly, don't import typing

[celest iron]

why?

[slender timber]

your friendly flake8 bot

[rare scarab]

httpx supports python 3.7

[celest iron]

they imported for typing.TYPE_CHECKING

[slender timber]

mm, so?

[rare scarab]

collections.abc before 3.9 doesn't support generics

[celest iron]

oh, i see

[slender timber]

Use sys.version_info to determine where to import from simple

[rare scarab]

but typing is alised to collections.abc

[slender timber]

if sys.version_info >= 3.9:
    from collections.abc import Generator
else:
    from typing import Generator

[celest iron]

the whole typing -> collections.abc is annoying since intellisense still suggests import typing instead of collections.abc atleast

[rare scarab]

typing collections may be deprecated, but they won't be removed any time soon

[celest iron]

something depreciated in 3.3 caused a ton of breakage in 3.7/8, i forgot what but it was a mess for me to deal with

[slender timber]

or be a chad, import everything from typing_extensions instead

[celest iron]

i think it was from collections import x instead of from collections.abc import x

[rare scarab]

well typing can be converted with pyupgrade (or ruff)

[oblique urchin]

yes, we're not actually going to remove the typing ones any time soon

[oblique urchin]

typing_extensions doesn't export everything

(yet; we'll change that in the next feature release of typing-extensions)

[slender timber]

wair is it 3.9 for generic collections.abc?

[oblique urchin]

yes

!pep 585

[rough sluice]

**PEP 585 - Type Hinting Generics In Standard Collections**

Link

Status

Final

Python-Version

3.9

Created

03-Mar-2019

Type

Standards Track

[rare scarab]

at least we get TypeVar in typing_extensions with the default kwarg

[slender timber]

i think i ended using it up for my 3.8+ lib, and it surprisingly passed the CI?

[trim tangle]

!pypi flake8-pep585

[rough sluice]

flake8-pep585 v0.1.7

flake8 plugin to enforce new-style type hints (PEP 585)

[celest iron]

i think that's enabled in ruff

[oblique urchin]

it might work in annotations, if you use from __future__ import annotations

[trim tangle]

minor plug 🐽🔌

[trim tangle]

is it?

[slender timber]

Uh yea i did that

[celest iron]

from __future__ import annotations this is superseded now tho, right?

[slender timber]

No

[rare scarab]

unless you use it as a base class (as guido intended)

[oblique urchin]

kind of, yes. 3.13 will ship with a different behavior (based on PEP 649)

[trim tangle]

what in particular

[rare scarab]

pyupgrade

[slender timber]

is dataclass_transform finally supported by mypy now?

[trim tangle]

ruff doesn't seem to complain about typing.List

[rare scarab]

because you don't have the UP rule enabled

and have your target_version set

[oblique urchin]

yes

[slender timber]

nice

time to relearn how it actually works

[rare scarab]

you're basically configuring your own @dataclass decorator or base class.

[trim tangle]

how do I do this?

[slender timber]

sometimes type hinting gets so complicated, it actually looks more complicated than the code itself

i thank typing.Any for coming to my rescue

[celest iron]

i suppose i'll have to liter some Any or type: ignore for this.
return transform(resp) is still tricky bc the try..catch pattern is ugly and I can decorate but it'll start snowballing in complexity

[rare scarab]

in pyproject.toml ```toml
[tool.ruff]
target-version = "py310"
select = ["UP"]

[celest iron]

understandably, i wouldn't even know what a type of that would even look like
Generator[[Request, Response], Response, None]?
perhaps

[trim tangle]

ah I see, now it does detect that. neat

[rare scarab]

it's even fixable with --fix

[trim tangle]

doesn't complain about Iterable though

oh wait, it does

nvm

[rare scarab]

using the ruff vscode plugin?

much better than flake8

[trim tangle]

nice, it does actually work

[rare scarab]

if you have the isort plugin, you may want to disable it or disable import organization on save

[umbral stump]

I enabled basic type checking on vscode I started to see this
"auth" is not a known member of module "requests"

when I was using like this

class SomeAuth(requests.auth.AuthBase):```
any idea?

[oblique urchin]

import requests.auth explicitly

[umbral stump]

nice, I was under the impression that the solution was related to how it should be imported. Now, why?

[oblique urchin]

presumably because https://github.com/python/typeshed/blob/main/stubs/requests/requests/__init__.pyi doesn't import auth

[umbral stump]

I was under that impression too. So basically, there's not a magic answer if this happens again with another lib. I basically need to dig into source, right?

[oblique urchin]

or just always write import a.b if you're using a.b.c

no need to look at the source directly

[umbral stump]

👍 thank you @oblique urchin

[umbral stump]

I'm generating stub files for a package. I see it creates stubs inside out directory but I still see vscode complainig that can't find stub file for the package. Should they be moved somewhere or what am I missing?

[tranquil turtle]

You should point typechecker to stub directory
Or install stubs globally (i dont know how that works)

[celest iron]

imo

var: cast[Type] = my_function(...)

ofc, this only works in non-callable situations.

• fall-back on typing.cast if u really need it in a complex expression
• it's basically a crutch as python typing system matures (when your forced, like with my generator typing issues [1])
• this is equivalent to turning off static typing for a line, but it's better than type: ignore which usually causes annoyances downstream in ur code.
  • which is what people end up doing anyways (see my later question)..

[1]:

def my_generator():
  yield 1
  yield "two"

gen = my_generator()
item: cast[int] = next(gen)
# 1: int

item2: cast[str] =  next(gen)
# "two": str

...
# current
gen = my_generator()

# imo, ugly and error prone due to ordering, + runtime
item = cast(int, next(gen))
item2 = cast(str, next(gen))

[celest iron]

not sure what i'm doing wrong here with my typing (forgive the casting lol)

from typing import Iterator, cast, Generator


def my_func(n: int) -> Generator[int | str, int | str, None]:
    x: int = cast(int, (yield))
    yield x + 10

I realize i have odd unions here but i don't understand where this None came from

[chrome hinge]

You're not yielding a value.

you're declaring your generator as yielding int|str, getting an int|str every send, and returning None. You're violating the first one by doing yield without a value.

[celest iron]

ah, coroutines have to yield a value, gotchya

that makes sense

hense why coroutine styled generators have to be "booted" with next(gen)

[prisma fiber]

Whre is the correct placement for a type var, next to the function, at the top of the code, next to the constants?

[slender timber]

I have a feeling this doesn't work

IndexPath: TypeAlias = Dict[int, Union[List[int], "IndexPath"]]

[vast olive]

why do you think so?

recursive types?

[slender timber]

yes

yup it doesn't

"__getitem__" method not defined on type "int"

[vast olive]

IIRC, with the new type alias syntax that might be supported

[brisk hedge]

Sounds like it worked properly and you just have other issues

[trail kraken]

Are there documentations on type parameters for types in collections.abc? I find myself searching typing for them. Particularly parameter orders for Generator.

[soft matrix]

one day 😢

i was gonna work on it but im busy procrastinating revising for my exams

[trail kraken]

OneDay[_TM]

[soft matrix]

fwiw https://github.com/python/cpython/issues/104869

[trail kraken]

Ah, I see. Circularl dependecy. Didn't think of that.

Hm.... Is that a problem?

[soft matrix]

it hopefully wont be in 3.13+

[trail kraken]

Ah. TypeVar potentially having alternatives?

[soft matrix]

but i can probably add them to the 3.12 docs

i just need to make an issue in sphinx about it

[trail kraken]

Eh. Quite the chain of issues.

[soft matrix]

no i think itll just be all rewritten in c

like typing._type_check and other things that are currently in python

[trim tangle]

huh, it implicitly imports typing? what happens if I hack in my own typing module somewhere in the chain 👀

[pastel egret]

Well you can make the results of some of the methods/operators do silly things. But you shouldn't be able to do anything beyond exceptions and whatnot.

[tranquil turtle]

Where will it be placed?
types.TypeVar or _typing.TypeVar?

[pastel egret]

_typing, that module has actually already existed for a little while.

Until 3.12 it just contained an identity function used to speedup NewType.__call__().

[trim tangle]

lol

identity function in C?

peak python

[pastel egret]

Yeah, literally just staticmethod(lambda x: x). Defining the method in Python caused an unacceptable slowdown compared to the original function version of NewType. But that one wasn't introspectable.

https://github.com/python/cpython/pull/27262

[trim tangle]

🤡

-- here's a function that literally does nothing
-- too slow

[soft matrix]

typing.TypeVar is already in c :)

[lunar dune]

Revising for your exams sounds like something you should be doing, so that definitely doesn't count as procrastination 😉

[rare scarab]

In UK, students revise for their exams. In US, teachers revise their exams.

[soft matrix]

does anyone know what the intended behaviour for accessing magic methods on a GenericAlias is the types and typing versions seem to disagree
as it stands i have a class defined as py class ID(Generic[TypeT], metaclass=abc.ABCMeta): ... i have a bit of code which uses a pre-specialised type alias to avoid unnecessary object construction py IndividualID = ID[Literal[Type.Individual]] and then somewhere in a subclass of ID i do py IndividualID.__init__(self, user.id64, type=Type.Individual) this breaks unless i add py class ID(Generic[TypeT], metaclass=abc.ABCMeta): __class_getitem__ = classmethod(GenericAlias) cause it tries to get typing._GenericAlias.__init__ which seems like incorrect behaviour to me

but im also not entirely sure

[warped lagoon]

hey Alex! we should definitely work more on HKTs PEP this summer, haha

[celest iron]

omg, there's workstreams for HKTs in python 😳

[cerulean pecan]

anyone know how to fix this error

i am using the mypy extension as well as poetry with mypy installed

[cerulean pecan]

fixed with ignore_missing_imports_per_module = true

[prisma fiber]

should i use type[T] or typing.Type[T]?

T = TypeVar("T")  # Defines a generic type variable "T"


def validate_input(input_prompt: str,
                   error_message: str,
                   input_type: type[T] = int) -> T:

[lunar dune]

typing.Type is deprecated on Python 3.9+. So you should prefer to use type[T] if you're only supporting Python 3.9+ or you're able to use from __future__ import annotations. But we won't be removing typing.Type any time soon, so you also don't need to worry too much about it 🙂

[prisma fiber]

oh okay, i thought they only changed the list in python 3.9

[lunar dune]

There's a full list of all the typing aliases that were deprecated in Python 3.9 here: https://peps.python.org/pep-0585/#implementation

Quite a few 🙂

[prisma fiber]

is there anything new to use instead of TypeVar?

[lunar dune]

On Python 3.12, you'll be able to use PEP-695 syntax, so you won't need to manually declare TypeVars anymore

But Python 3.12 isn't released yet

It's still in the beta period; it'll be released in the autumn

[prisma fiber]

i'm using python 3.10.8

[prisma fiber]

oh ok

thanks for the help :))

[lunar dune]

no worries!

[lunar dune]

types.GenericAlias and typing._GenericAlias differ from each other in quite a few ways. In some ways, it would be nice to see if we could get typing._GenericAlias to inherit from types.GenericAlias. But that would be a very risky project that would be a lot of work and might not markedly improve things for users...

[soft matrix]

isnt typing._GenericAlias gonna be removed at some point, wouldnt that be the first step towards removing it?

[lunar dune]

The specific difference in behaviour here, I think, is that types.GenericAlias delegates all attribute access to the __origin__ class, except for a hardcoded list of attributes here: https://github.com/python/cpython/blob/06893403668961fdbd5d9ece18162eb3470fc8dd/Objects/genericaliasobject.c#L591-L604. (This behaviour has led to a fair few bug reports! 🙂 )

typing._GenericAlias doesn't defined a __getattr__ method in the same way at all, however

[rough sluice]

Objects/genericaliasobject.c lines 591 to 604

static const char* const attr_exceptions[] = {
    "__class__",
    "__origin__",
    "__args__",
    "__unpacked__",
    "__parameters__",
    "__typing_unpacked_tuple_args__",
    "__mro_entries__",
    "__reduce_ex__",  // needed so we don't look up object.__reduce_ex__
    "__reduce__",
    "__copy__",
    "__deepcopy__",
    NULL,
};```

[lunar dune]

oh that's a pretty cool bot

[lunar dune]

I'm not aware of any plans to do that, but yeah, unification would be nice

Just... very risky 🙂

[soft matrix]

https://github.com/python/cpython/blob/06893403668961fdbd5d9ece18162eb3470fc8dd/Lib/typing.py#L1136-L1144 it does also appear to already attribute forward, just only a few things

[rough sluice]

Lib/typing.py lines 1136 to 1144

def __getattr__(self, attr):
    if attr in {'__name__', '__qualname__'}:
        return self._name or self.__origin__.__name__

    # We are careful for copy and pickle.
    # Also for simplicity we don't relay any dunder names
    if '__origin__' in self.__dict__ and not _is_dunder(attr):
        return getattr(self.__origin__, attr)
    raise AttributeError(attr)```

[soft matrix]

just not dunders

[lunar dune]

Interesting

If you have a use case for forwarding dunders, I suppose you could open a bug report. But relaying all dunders by default has led to some unexpected corner cases for types.GenericAlias, so I'm not sure that's a great idea tbh.

[soft matrix]

idm either way cause i can just use object.getattribute id just prefer if things didnt break under my feet

typecheckers seem to think that it should just forward all attributes afaict

anyway ill add unification of the 2 to my list of things to do

[slender timber]

if TYPE_CHECKING:
    class StructEventBase(EventBase[c.Container[Any]], UserDict[str, Any]):
        ...
else:
    class StructEventBase(EventBase[c.Container], UserDict[str, Any]):
        # Class definition here

how do i make this look nice?

if i use the above, vscode darkens my entire class definition

[soft matrix]

class StructEventBase(EventBase[c.Container[Any] if TYPE_CHECKING else c.Container], UserDict[str, Any]):```

not sure why exactly though youre not subclassing with the generics yourself at runtime

fwiw c.Container if thats collections.abc.Container doesnt need to be strictly inherited from, you can either use the register decorator at runtime or just leave it out and it will still work iirc

[lunar dune]

I don't think you need the .register decorator or inheritance from Container, since UserDict already has Container in its __mro__:

>>> import collections
>>> class Foo(collections.UserDict[str, str]): ...
...
>>> import collections.abc
>>> isinstance(Foo(), collections.abc.Container)
True
>>> Foo.__mro__
(<class '__main__.Foo'>, <class 'collections.UserDict'>, <class 'collections.abc.MutableMapping'>, <class 'collections.abc.Mapping'>, <class 'collections.abc.Collection'>, <class 'collections.abc.Sized'>, <class 'collections.abc.Iterable'>, <class 'collections.abc.Container'>, <class 'object'>)

[slender timber]

ok TIL, UserDict wasn't generic until 3.9 too

I ended up using TypeAliases at the end

class StructEventBase(EventBase[AnyContainer], AnyDict):

where

(type alias) AnyDict: Type[UserDict[str, Any]]
(type alias) AnyContainer: Type[Container[Any]]

and, no, Container is not from stdlib

[lunar dune]

and, no, Container is not from stdlib

Ah, that changes things

[slender timber]

I simply hate when this stuff breaks compatibility and I end up making a botched release

[soft matrix]

simply dont support <3.9 😎

[rare scarab]

do this? ```py
if TYPE_HINTING or sys.version_info >= (3, 9):
BaseClass = UserDict[str, Any]
else:
BaseClass = UserDict

[slender timber]

yea i did that

[rare scarab]

Use tox or github actions to run your test on all supported python versions

[slender timber]

i have a separate types submodule in my library for all these typing shenanigans and its increasing its LoC by the day

[acoustic thicket]

shadowing the stdlib module name 👀

[slender timber]

i do, but since my test workflow is very fickle, I don't want to end up depending on the successful completion of tests for the publish workflow to happen

just because sphinx detects a dead link in my docs, i don't want an annotated tag to go wasted

[slender timber]

remove the one from stdlib 🗿

[warped lagoon]

hm, we need to define how the type-checking process would work

because it's not simply using bounds and storing the current type

[rare scarab]

What does sphinx have to do with tests?

[slender timber]

i build my docs in my test workflow (i run tox in my workflow, which runs sphinx linkcheck)

[warped lagoon]

that's... a bit weird of a workflow

perhaps you should separate the two

[rare scarab]

put that in your "mkdocs" task

Makefile, shell script, whatever

[slender timber]

make a separate workflow for it you mean?

idk why i never thought about that

maybe because its in the single tox file

and running tox from ci is just super convenient

[rare scarab]

docs are unrelated to your tests, so yes

Most people wouldn't even consider updating the docs a real update

[slender timber]

i could just exclude the docs subsection of tox to not run in tox when on CI

[slender timber]

yea me neither, i just added it once when i had skill issues with sphinx

[tranquil turtle]

some guy in #internals-and-peps supports only >=3.12 🗿

[rare scarab]

understandable

[soft matrix]

based

[junior dock]

I have a factory method that returns an object. It can either be from a base class A or an inherited class B that has additional methods.
Now when I try to use a method from the inherited class I get the following type error from Pylance, although I have specified a union type.
Is there a way to fix this or maybe a smarter approach?

Here is the example code:

class A:
    def alpha(self):
        pass


class B(A):
    def bravo(self):
        pass


def factory() -> A | B:
    return B() if is_b() else A()


obj = factory()
obj.alpha()
obj.bravo()

[rare scarab]

Well A doesn't have bravo, so you need to do a type check before calling it

[soft matrix]

how does the function decide which to return?

you might able to use overloads

[oblique urchin]

just at the basic level, if you do something with a Union, that something must be valid on all members of the union

so you need some way to tell which member of the union you're dealing with

[rare scarab]

A | B is effectively A

[junior dock]

This is a database connection. It will check which variant your system supports and then return either the basic one or the advanced one.

it is based on python's dbm.open, which also returns a different object based on which DBM variant is supported

[oblique urchin]

that's true here yes, since B is a subclass of A

[oblique urchin]

for that one we just say it returns a fake class that represents the common interface of all the databases. https://github.com/python/typeshed/blob/9456f5501b564b333891fb474f44f7172c6d5174/stdlib/dbm/__init__.pyi#L94

[rough sluice]

stdlib/dbm/__init__.pyi line 94

def open(file: str, flag: _TFlags = "r", mode: int = 0o666) -> _Database: ...```

[junior dock]

ah thanks. yeah that could be a solution

[umbral stump]

I'm using an external lib and I seeing this when calling a method of an object of that lib. What are my options?

  Type of "external_method" is "(tree: Any | None = ...) -> Unknown"```

I'm using it like:
```external_object.external_method(data)```

[young zealot]

Can you make a protocol that is like a "subclass" of a protocol? Like is this valid:

from typing import Protocol

class A(Protocol):
    def a(self) -> None:
        ...

class B(A, Protocol):
    def b(self) -> None:
        ...

class C(B):
    def a(self) -> None:
        pass

    def b(self) -> None:
        pass

[oblique urchin]

yes, but only of another Protocol

so in your example, A is a protocol that requires a, B is a protocol that requires a and b, and C is not a protocol

[young zealot]

great, thanks

[young zealot]

If a function takes list[str | int] as a parameter, if you pass a list[str] to it you get an error about incompatible types. Is there a way around that since they should technically be compatible types?

[lunar dune]

list is invariant. For parameter annotations, you can avoid this issue by trying to use covariant types such as Sequence or Iterable wherever possible

https://mypy.readthedocs.io/en/stable/common_issues.html#invariance-vs-covariance and https://mypy.readthedocs.io/en/stable/generics.html#variance-of-generics give more details

[young zealot]

ah, ok makes sense, thanks

[rare scarab]

try [*lst]

note: makes a copy, which may not be what you want

[young zealot]

if I have no control over the function definition I will, but for ones I control I do prefer using Sequence like Alex mentioned

[celest iron]

Is there a pep that relates to typing generator sequences?

# These types assume PEP 696 for simplicity
# Though, it would work w/o it.

# this generator yields a ints forever
Generator[int]

# this generator yields an int then str
Generator[[int, str]]

# this generator yields an int, str, int, str, ... (forever)
Generator[[int, str, ...]]

Currently, we are forced to do this, forcing us to add # type: ignore OR cast calls.

Generator[int | str]

[oblique urchin]

no, I'm not aware of any serious proposal around that

[celest iron]

Do you have any recommendations on where I can start a conversation about this?

[oblique urchin]

https://github.com/python/typing/issues

[exotic glacier]

This sounds like an anti-pattern. What is the use case?

[oblique urchin]

I should add that I'm not convinced this pattern is common enough to justify the cost of a new type system feature. You can start a conversation if you like but in my view it may not go anywhere.

[celest iron]

I'm mostly inspired by some ideas presented here: http://www.dabeaz.com/finalgenerator/

I've been using resp = yield req to implement some sans-io patterns. It's been very helpful for me to abstract some buisness-logic, esp middleware components in an elegant way.

IMO, these sort of generator as non-iterators patterns are weird to see bc they aren't well-supported within the typing system. they don't feel robust since u can't precisely type them.

The standard library already leverages the yield pattern for @contextmanager which is a use-case of generators beyond just iteration.

# From the above link
import tempfile, shutil
from contextlib import contextmanager

@contextmanager
def tempdir():
   dirname = tempfile.mkdtemp()
   try:
       yield dirname
   finally:
       shutil.rmtree(dirname)

the type of the generator here could enforce only having one yield (a second yield statement or one that doesn't yield anything would cause a typing error). Generators beyond iteration I think are really neat, but the typing system makes it a bit difficult to treat it robust-ly.

[exotic glacier]

Yes but generating something where the types alternate seems like an anti pattern. You can never know where the next consumer of the generator will be in the series

[celest iron]

the main purpose of the suggestion was for bounded sequences, perhaps adding the ... for infinite alternating sequences is a mistake but it was for completions-sake.

it's also not that hard to know as you iterate through it.

though, i might be missing something on implementation side

[brisk hedge]

I would be interested in the flexibility of generators being explored a bit further than just for homogeneous sequences

Since there's a lot you can do with the continuation stuff

[trim tangle]

I think the thing yall are looking for is called "session types"

[brisk hedge]

of course generators are kind of de facto locked in at this point

[celest iron]

https://github.com/fremartini/SessionPy like this?

[trim tangle]

no idea

too big brain for my small brain

I just remember smart words

(smart words make me look smart)

[celest iron]

My usecase was for middle-ware type things..

def transform() -> Generator[
  [Request, Response],
  [Request, Response],
  None
]:
  req: Request = yield
  resp: Response = yield convert_request(req)
  yield convert_response(resp)  # or return

[trim tangle]

but me tiny ducky

quack

[celest iron]

this transformation logic can be chained together w/o any of the transformation logic needing to perform the actual request (ala. sans-io)

def transform(req: Request) -> Generator[Request, Response, Response]:
  resp: Response = yield convert_request(req)
  return convert_response(req, resp)

here's a form that currently is typed correctly (though, it seemingly looks like it would go infinite instead of just limited in scope, similar to contextmanager).

[young zealot]

I'm working with tkinter, and I want to make a function that will take "any window" type as its parameters. The main windows are subclasses of the Tk type, and sub-windows are subclasses of Toplevel type. How can I define this AnyWindow type to accept any of my windows?

[trim tangle]

Maybe also something like this? py @chain_nones def sequence(): foo = yield foo_nullable() bar = yield maybe_make_bar(foo) return bar.baz() where each yield returns a value depending on what you yield

Kinda like await, I guess

[celest iron]

i'm using the generator to hold context, this logic was previous done via recursive functions where the req was held in a call-stack.

what's @chain_nones?

[trim tangle]

it's like do notation in Haskell in the Maybe monad if you know what I mean

[oblique urchin]

we actually use "heterogeneous" generators very heavily with https://github.com/quora/asynq. To typecheck it I just had to write my own typechecker. It's really hard to imagine a way to do this correctly in the standard type system.

[trim tangle]

def sequence():
    if (foo := foo_nullable()) is None:
        return None

    if (bar := maybe_make_bar(foo)) is None:
        return None

    return bar.baz()

[trim tangle]

To typecheck it I just had to write my own typechecker
ah yes. so trivial, how didn't I think of that

[celest iron]

what, where was this when I was blasting #async-and-concurrency about sans-io

the whole obj = yield value pattern is really nice but kinda awkward to iterate through manually but that's another discussion

[oblique urchin]

yes, we never iterate manually through asynq coroutines, it's always driven by the framework

[trim tangle]

quora seems like a nice place to work at
(though maybe a bit of NIH as it seems from the outside)

[oblique urchin]

it's pretty good! asynq is not something I would recommend we build today, but it's hard to move away from in a codebase with like 25k asynq functions

[trim tangle]

the website is kinda annoying though lmao

no offence

[celest iron]

why so? is the abstraction layer needing u to teach like a new "language" to use async internally or is it to DSL-esque?

[oblique urchin]

mostly because it's better to use tools that are widely available and used across the industry (I think that's what @trim tangle meant with "NIH" too)

[celest iron]

what does NIH stand for lol

[trim tangle]

yeah, you literally you have your own type checker

[oblique urchin]

e.g. if we used asyncio instead we could use lots of existing libraries that work with it

[oblique urchin]

not invented here

[celest iron]

ah

[trim tangle]

https://en.wikipedia.org/wiki/Not_invented_here

[celest iron]

this sort of pattern could be used with asyncio, trio etc? can't you integrate something like anyio

my reasoning for this setup is to make it easy to maintain a codebase with both sync and async apis

maybe asynq is a bit more involved tho

[oblique urchin]

possibly, not too familiar with anyio

one thing about asynq is that it's primarily for batching, not asynchronous IO

making it so that instead of 10 calls to memcache we can make just one

[celest iron]

anyio basically duplicates the trio concepts of nurseries and task management from a generalized POV so u can use whichever backend. it's less neccesary since some trio concepts are arriving in asyncio but it allows u to use whatever framework u want afaik.

[celest iron]

i'm looking thru ur asynq library, so many ideas i've been having about unique cases for generators are explored here, pretty neat stuff.

[brisk hedge]

I guess you can "fake" it by using await instead of yield and just writing a paper-thin "async" runtime 😄

[brisk hedge]

like, this typechecks for example

from __future__ import annotations
from typing import Generator, Generic, TypeVar, TypeVarTuple, reveal_type

Ts = TypeVarTuple("Ts")
Head = TypeVar("Head")
Tail = TypeVarTuple("Tail")

class Stream(Generic[*Ts]):
    def __call__(self) -> Requester[*Ts]:
        ...

class Responder(Generic[*Ts]):
    def __await__(self: Responder[Head, *Tail]) -> Generator[None, None, tuple[Requester[*Tail], Head]]:
        ... # maintain internal state

class Requester(Generic[*Ts]):
    def __call__(self: Requester[Head, *Tail], req: Head) -> Responder[*Tail]:
        ... # maintain internal state

class Request1: ...
class Response1: ...
class Request2: ...
class Response2: ...

# used with e.g. a @streamable decorator that 
# initializes internal state and does the actual work.
# that one doesn't need to be typed outside of its api
async def heterogenous_ish(stream: Stream[Request1, Response1, Request2, Response2]):
    _ = stream()
    _, resp = await _(Request1())
    reveal_type(resp) # Response1
    _, resp = await _(Request2())
    reveal_type(resp) # Response2

[celest iron]

oh man, using await as a sugar / abstraction is even more dubious

[brisk hedge]

yep >:3

that's what you get from someone who lives in #esoteric-python

[celest iron]

from typing import Protocol

class Req1: ...
class Resp1: ...
class Req2: ...
class Resp2: ...

class MyProtocol(Protocol):
    def __call__(self, req: Req1 | Req2) -> Resp1 | Resp2:
      ...

Is it possible to use TypeVar in some way that makes the guarantee that if req is Req1 then the return is Resp1 and same with Req2 -> Resp2?

i was thinking about @overload but that seems wrong

or do I just define two protocols here and then union them?

[oblique urchin]

seems like exactly what overloads are for

[celest iron]

oh, i didn't realize it worked with protocols like that neat

[fluid jay]

i'm currently learning type hinting and checking it with mypy

should I accept what mypy say as a rule or is it better to treat what error it spew at me like a guideline

[oblique urchin]

you can always ignore errors with # type: ignore if you want. However, it's good to aim to get your codebase to have no mypy errors, because that means that if you make changes later and get mypy errors, you know they're due to your changes and not due to something that was already bad.

[fluid jay]

if thats the case, i'd like some help with this snippet of code, just as a primer of what i should expect:

def userinput() -> str:
    try:
        entry = input("Package/module name: ")
    except KeyboardInterrupt:
        print('Exiting...')
        time.sleep(1)
        exit()
    try:
        pkg, method = entry.strip().split('.')
        main(pkg, method)
    except ValueError:
        main(entry)

mypy said:

main.py:60: error: Missing return statement  [return]

in a condition where I don't need the function to return anything, what should I do?

[oblique urchin]

change the return type to -> None if you don't want it to return anything

[fluid jay]

aaaah

So -> None doesn't mean a it returns a nonetype object (iirc)

[oblique urchin]

it means it returns None, and functions with no return statements return None

[fluid jay]

got it
thank you

[fluid jay]

i have this function:

def fsearch(var: str, names: List[str]) -> Tuple[None, None] | Tuple[str, Optional[List[Tuple[str, int]]]]:
    cutoff: float = 0.55 #tentative
    candidates: Dict = {}

    for i in names:
        score: float = ratio(var, i)
        if score > cutoff:
            candidates[i] = int((score*100))
            
    candidates_sorted = sorted(candidates.items(), key=lambda x:x[1], reverse=True)

    if candidates and candidates_sorted:
        return max(candidates, key=lambda x:candidates[x]), candidates_sorted
    else:
        return None, None

another part of the code uses it like this:

...
else:
  matches, closest = fsearch(method, names)
...
  else:
  print(closest[0:5])

and i'm getting this from mypy:

main.py:64: error: Value of type "Optional[List[Tuple[str, int]]]" is not indexable  [index]

what did i miss?

[viscid spire]

Optional is the same as a Union of the type and None. None is not indexable/subscriptable, so Optional[...] is not indexable/subscriptable

which means you have to check for is not None in some way before subscripting

[fluid jay]

aaaah

got it ill do that

honestly i was lost

[viscid spire]

it is the same as when you do a Union, that you can only use methods and access attributes common to both types in the Union

(by typing system, of course you can write valid code that fails 😄)

[fluid jay]

im really still new to actually applying typehinting, ive just been reading things

i think one advantage for typehinting is, everything feels more rigourous? since python is dynamically typed

[viscid spire]

it is also very optional

you can typehint what you want, and not typehint what you don't want (although I recommend typehinting most things 😄)

but your code isn't gonna fail because you didn't put a typehint, like types in languages like Java or C++

[fluid jay]

i used C# briefly in the past and i kind of liked the idea of declaring variable types before assigning values into it

[viscid spire]

you can do that in Python too but generally not recommended 🤷‍♂️

I do it for classes sometimes

[fluid jay]

yea i wont
at least the spirit lives in type hinting for me

[viscid spire]

class LotsOfAttributes:
  x: int
  y: int
  abc: str
  arr: list[str | int]
  
  def __init__(self):
    self.x = 2
    self.y = 3
    self.abc = "abc"
    self.arr = []

[fluid jay]

i was thinking of doing soemthign like that

[viscid spire]

(yes I know that 4 is not "Lots" xdd)

[fluid jay]

i tried it and it works, but is it the way to do it

[viscid spire]

in essentially every other case I won't do this tho

sometimes you need it tho

one specifically is unpacking

x, y = 0, 1

[fluid jay]

oh yeah

[viscid spire]

you aren't allowed to put typehints on this line

so you must do them above

[fluid jay]

i did try that as well

pylance just screamed at me

[viscid spire]

x: int
y: int
x, y = some_function_which_returns_ints()

although most typecheckers just grab the type from the function's return type

[fluid jay]

it was some annotation errors

[viscid spire]

so not even a necessary annotation imo

[fluid jay]

i'd prefer this as well

but i guess it doesn't hurt

[viscid spire]

3 lines tho 🥶

doesn't look great

[fluid jay]

honestly like
while learning i do not really pay attention to amount of lines, just as long as it makes sense to me and no one find it offensive its cool

[viscid spire]

it's important to know that you can do this I guess

but ultimately it's choice based

[fluid jay]

yes definitely

learning what can be learned and later discard maybe some redundant stuff

Success: no issues found in 1 source file

and mypy seems happy

[viscid spire]

typehinting is primarily a tool to help you and others who may read your code. Doing the bare minimum for your own benefit is fine imo

[frigid jolt]

hi guys; i have a bot.py file from where i'm importing a Context class, but in the file where Context is defined i need to import something from bot just for typing things (that's causing circular imports)... now what i had done before to avoid the circular import was:

# bot.py
from __future__ import annotations
from utils import Context

class MyBot(...):
  ...
  def get_context(..., *, cls = Context) -> None:
    ...

...

# utils.py
from __future__ import annotations
from typing import TYPE_CHECKING

from discord.ext import commands

if TYPE_CHECKING:
  from bot import MyBot


class Context(commands.Context[MyBot]):
  ...

now that's not working: i get an error saying that MyBotis not defined...

hence my question arises, is it possible to pass things imported under TYPE_CHECKING to Generic(s)?

[tranquil turtle]

Use "MyBot"

[frigid jolt]

i tried that too and got TypeError: <class 'utils.context.Context'> is not a generic class

shouldn't the __future__ import handle that? convert typings to strings and store them in __annotations__ to not evaluete them at runtime

[chrome knot]

``import random
import string
import requests
import time

mum = 10

for i in range(10):
code = "".join(random.choices(string.ascii_lowercase + string.digits + string.ascii_uppercase, k = 16))
print(code)``

I have this simple code

But I have a question

How can I make it write random choice between 16 to 22
Not only 16

@frigid jolt
Can you help me please

[frigid jolt]

this is not the right channel; go ask in #python-discussion or open a post in #1035199133436354600

[chrome knot]

I can't make post

Please help me here

[frigid jolt]

i can't help you here because it's against the rules...

[trim tangle]

If you are unable to make a post, you should contact @still stone

[blazing nest]

On a Python 3.11 project, am I supposed to use typing.AsyncContextManager or contextlib.AbstractAsyncContextManager?

[oblique urchin]

the latter

[slender timber]

ah, yet another collections.abc pulled off successfully

[young zealot]

I have a function that takes an enum, and the return type is a different class based on the enum case. Is there a way to typehint this function/enum so I get the concrete return type based on the enum case passed to it?

[heady flicker]

typing.overload

[rare scarab]

Should I be doing isinstance(typ, types.UnionType) or typing.get_origin(typ) is typing.Union? I somehow ran into this where Sequence[str] | dict[str, str] is a _UnionGenericAlias but it isn't UnionType.

[young zealot]

hmm I'll look into it, thanks! any good links for reading?

[heady flicker]

The first one, yes.

[heady flicker]

See mypy docs or python typing official docs

[rare scarab]

🤔

>>> isinstance(typing.Sequence[str] | typing.Mapping[str, str], types.UnionType)
False

works with list and dict, but not this

Ah, works when used with collections.abc

I think if I want to support typing collections and python <3.9, I have to use a function like this. ```py

def _is_union(typ: Any) -> bool:
return isinstance(typ, types.UnionType) or get_origin(typ) is Union

[rare scarab]

sorry, not support python <3.9. Support libraries written for <3.9

[soft matrix]

Another reason should probably merge GenericAlias with _GenericAlias

Or make _UnionGenericAlias a subclass of UnionType

[young zealot]

I'm upgrading a legacy codebase with packages to use full type hinting, but now I'm struggling with cyclical imports. Are there any good rules of thumb to follow? Any good links for reading up on how to manage this?

[soft matrix]

Are you using typing.TYPE_CHECKING?

[young zealot]

I have for some, but I'm going to go through and do more

if I do if TYPE_CHECKING: from a.b import c, do I need to do my_var: 'c' instead of my_var: c?

[rare scarab]

Also do from __future__ import annotations

[young zealot]

What does that do?

Ah i think I found it, it’s the postponed evaluation thing?

[viscid spire]

Lazy evaluation of typehints

[hallow flint]

see also https://mypy.readthedocs.io/en/stable/runtime_troubles.html

[lunar dune]

There's actually a bit of code in functools in the stdlib that deals with exactly this problem:

https://github.com/python/cpython/blob/9d35a71a76cb033598ce136ea655d9e452fe3af0/Lib/functools.py#L842

@oblique urchin should we maybe add a typing.is_union function?

[rough sluice]

Lib/functools.py line 842

def _is_union_type(cls):```

[oblique urchin]

I wish we could unify types.UnionType and typing.Union

Not sure that's possible while preserving compatibility

[lunar dune]

Yeah

I think the issue is that we'd have to implement ForwardRef in C if we wanted to allow forward references in types.UnionType, and Guido wishes ForwardRef never existed, so doesn't want any part in that

[oblique urchin]

there are ways around that

[lunar dune]

Say more?

[oblique urchin]

the same way TypeVar.__or__ currently returns a typing.Union

https://github.com/python/cpython/blob/9d35a71a76cb033598ce136ea655d9e452fe3af0/Objects/typevarobject.c#L100

[rough sluice]

Objects/typevarobject.c line 100

make_union(PyObject *self, PyObject *other)```

[lunar dune]

Right, so I guess we might just have to flag in the docs that if you use a forward reference in a types.UnionType, it might end up being a bit slower as it would call into Python code to create a typing.ForwardRef object when it encountered a string

Though "it might be a bit slower" is probably better than the current situation of "it will fail"

So maybe we wouldn't even have to document the slowness

[oblique urchin]

we wouldn't necessarily have to make | accept forwardrefs

can only gives types.UnionType a __class_getitem__ that works like typing.Union

though admittedly that's not the most intuitive behavior

[lunar dune]

That feels like it would unify the implementation details a little bit without unifying the semantics for the end user

The latter is what I'd like to see personally

[oblique urchin]

Sure, but we can't do that completely as the types have to implement __or__. E.g. Union["A", "B"] works but we can't make "A" | "B" work

unifying the implementations is a good first step towards unifying the semantics, in any case

[lunar dune]

(Not a serious suggestion) we could add str.__or__ 😈

Add __or__ to everything

The typing community needs it

[oblique urchin]

we should also make int.__or__ return a Union then

[lunar dune]

As I say, the typing community needs it

The Steering Council has said they want typing more integrated into the core language, who are we to disobey

[lunar dune]

On a more serious note again, I do think it would be good to unify semantics "as much as possible", even if we can't do it completely. Multiple users have been confused by the fact that int|"Foo" doesn't work; I think it's pretty clear by now that it's unintuitive for users upgrading from typing.Union to types.UnionType

It's probably more obvious to users that "Foo"|"Bar" is not going to work

[oblique urchin]

here's a first implementation: https://github.com/JelleZijlstra/cpython/commit/8c8b1dd785bff474b190f46fabc6ea84b6bf4de6. Works for simple cases: ```>>> from typing import Union

Union["int", "str"]
ForwardRef('int') | ForwardRef('str')
Union["int", "str"] | float
ForwardRef('int') | ForwardRef('str') | float
_.args
(ForwardRef('int'), ForwardRef('str'), <class 'float'>)

There's a bunch of test failures in `test_typing.py`, mostly for silly reasons (the repr() changes)

[soft matrix]

Doesn't this break .origin?

[lunar dune]

I guess the people who like Optional won't love Optional["Foo"] being evaluated at runtime to ForwardRef("Foo") | None 😄

[oblique urchin]

yes will have to add that back

[slender timber]

hey typing people
I need to make this work

class Str:
    def __class_getitem__(cls, item: tuple[str, int]):
        return Annotated[str, *item]

class TestStruct:
    b: Str

I expect an error here, but I get nothing from pylance

expected use is an (encoding, length) tuple

b: Str["utf-16", 50]

pls tell me this is possible

[lunar dune]

__class_getitem__ is an implementation detail of how typing works at runtime. Type checkers aren't expected to support any custom definitions of __class_getitem__

Not sure it's possible to do what it looks like you're trying to do here, unfortunately :/

[slender timber]

I am making a declarative dataclass like thing but for C types

so I get type checker support by using dataclass like field declarations

and i can annotate all the metadata cleanly

[slender timber]

is there any other way i can achieve the above? :/

[brisk hedge]

you can set Str to be Annotated when inside a TYPE_CHECKING block and a regular class otherwise

not sure it's possibly to partially bind it in an alias though

[slender timber]

How can I make Str an annotated?

Str = Annotated[str, ???]

[trim tangle]

You can make a generic type alias

hmmm actually idk if you can

[slender timber]

Like Str[Encoding, Length] (where Encoding and Length are TypeVars)?

[trim tangle]

nope no you can't

maybe it will be possible with PEP695 but not sure

[slender timber]

!pep695

[rough sluice]

PEP 8

PEP 8 is the official style guide for Python. It includes comprehensive guidelines for code formatting, variable naming, and making your code easy to read. Professional Python developers are usually required to follow the guidelines, and will often use code-linters like flake8 to verify that the code they're writing complies with the style guide.

More information:
• PEP 8 document
• Our PEP 8 song! :notes:

[slender timber]

[trim tangle]

lmao

!pep 695

[rough sluice]

**PEP 695 - Type Parameter Syntax**

Link

Status

Accepted

Python-Version

3.12

Created

15-Jun-2022

Type

Standards Track

[slender timber]

uh yea how did i forget this one

[slender timber]

is there a TypedDIct or type hint for __dataclass_transform__?

[soft matrix]

dont think so

[regal summit]

Wrote a script that turns json data like example.jsoninto (properly) typehinted python classes, https://paste.pythondiscord.com/ujudurafak. Thought it was kinda cool

The code is very gross and very inefficient but you only really gotta run it once so its fine

[brazen jolt]

noqa is for flake8

with pyright, you can only do # type: ignore

mypy does have some specific error types you can choose to ignore

but pyright only gives you a wildcard ignore for all errors on that line

[summer berry]

It tells you the name of the error. It's reportGeneralTypeIssues

Unfortunately I think that is a very broad error

You can disable it with a comment. https://microsoft.github.io/pyright/#/comments

[brazen jolt]

huh, that's interesting, I've never seen those before

but yeah, ig it's because pyright errors are usually in the same category

[summer berry]

I think with something as broad as reportGeneralTypeIssues you may as well do type: ignore. But maybe ignoring just the former still buys you something? Not sure.

[soft matrix]

Their are a couple different ones but a lot of the common ones fall under the same name

[brazen jolt]

you could go with a type-cast instead

[soft matrix]

I try to avoid having more than required on a line with a type ignore much like a try except

[brazen jolt]

I tend to prefer those to type ignores, as while they also sort of bypass the type-checker, at least they're much more explicit about what you're doing

[oblique urchin]

then you might as well not run pyright

assert foo is not None before the await?

[summer berry]

The comment only disables it for one line, not for the entire code base.

[summer berry]

Oh I'm mistaken, that comment is for the whole file. For the line it would be # pyright: ignore [reportGeneralTypeIssues]

Can a type stub file be used to overwrite that?

[halcyon gull]

Is it ok/pythonic to add a

if not isinstance(func_arg, MyCustomClass):
    raise TypeError("func_arg must be of type MyCustomClass")

in a procedure, if I know that func_arg being of any other type than MyCustomClass makes no sense? Or, considering Python is dynamically-typed, should I leave that error message out and just allow the interpreter to automatically fail when I try to access a field of MyCustomClass? Not sure if the second way is more pythonic, since it avoids dealing with types, but is less user-friendly.

[summer berry]

Runtime type checks are typically not considered pythonic. Pythonic is duck typing.

If you check this argument, then are you going to check every other argument? Are you going to extend that to other functions, or to the rest of your program? Where do you draw the line?

[halcyon gull]

This would be a single instance as far as I can predict. It is in the constructor of an abstract class many classes inherit from, so the call to super().__init__(...) should handle that in this case, I think.

[summer berry]

You can add such check if you think it would be really easy to mistakenly pass a different type, and/or if passing a different type would result in an error or behaviour that's really difficult to debug otherwise

[young zealot]

is there a way to get the concrete type of a generic's type? for a basic example, how to make this work: https://gist.github.com/mypy-play/6de9884a8a9e19272da70e3738296cf1

[summer berry]

Are you looking to implement a type guard? https://peps.python.org/pep-0647/

Hmm I guess that doesn't help with generics either way

Ah actually they have an example that does something similar py def is_set_of(val: Set[Any], type: Type[_T]) -> TypeGuard[Set[_T]]: return all(isinstance(x, type) for x in val)

Oh, right. That is passing the type explicitly but you want to get it from a type var

[young zealot]

yeah. and my end goal is not just a basic check like a TypeGuard check, thats just the contrived example

[summer berry]

I think the answer to that is no because Python does not track the concrete type of a typevar.

[young zealot]

yeah, I'm thinking of just adding the Type[T] to the initializer (like this: https://mypy-play.net/?mypy=latest&python=3.11&gist=226bd5d73cad133b47f7c8059a4acbff)

[warm ermine]

how i check if a property is of type of Generic[T] isinstance(self.value,T) don't work

[trim tangle]

Generally that's impossible. Instances don't "know" T at runtime, it's all in the mind of a type checker

Why do you want this? Could you share more code maybe?

[warm ermine]

i want raise a Exception if a value not is instance of a my generic Type

[trim tangle]

How would you perform the check if T is Callable[[int], str]?

[soft matrix]

If only bound could be ~Callable ;(((

[trim tangle]

how would you check an Iterator[int] 🙂

[soft matrix]

Or recursively exclude them

And that but allow anything beyond a Collection

[tranquil turtle]

class Boom(Collection[T], Callable[..., T]): ...

it is collection, but you cannot ensure that it is a Callable[..., T]

[slender timber]

what is the most appropriate type hint for all types of file paths, readable + seekable binary streams, memoryviews, bytes, bytearray, Sequence[int] etc.?

[soft matrix]

collections.abc.Buffer | str | bytes | ACustomSeekableReadableProtocol | Sequence[int]

maybe make str | bytes a type alias cause its technically covered by Sequence[int]

also are you actually sure that any Sequence[int] is accepted?

cause being able to just pass [1,2,3,4] seems a bit odd

[slender timber]

To make it a bit less ambiguous I chose to use separate factory methods

class Parser:
    @staticmethod
    def from_file(file: c.FilenameType) -> Parser: ...
    @staticmethod
    def from_stream(stream: BinaryIO) -> Parser: ...
    @staticmethod
    def from_buffer(buffer: SupportsBytes) -> Parser: ...

collections.abc.Buffer is nice, but 3.12 :/

[soft matrix]

typing_extensions backports it

[slender timber]

it does?

[soft matrix]

p sure

[slender timber]

i already got 4.6.3

and there's no Buffer in it

[soft matrix]

https://github.com/python/typing_extensions/pull/125

says its in 4.6

[slender timber]

i see it in typing_extensions.py in my venv, but pylance says unknown import

[soft matrix]

probably not in a typeshed release yet

[slender timber]

:/

SupportsBytes fwiw rn

[oblique urchin]

it caused some problem in pyright

might need to work more with eric to figure it out

[lunar dune]

https://github.com/microsoft/pyright/issues/5243

[raven prairie]

Is there some way that I can enforce correct type hints in an evaluated context?

[soft matrix]

no, typehints are not enforceable at runtime

[raven prairie]

Even if I control the execution? i.e. I'm using eval to run the script with a custom environment?

[soft matrix]

i think if you have total control and its standard python it might be possible

but uhh thats really not worth the effort cause it seems hellishly hard to implement

[trim tangle]

There are libraries like typeguard that let you run the checks for some cases (like int or tuple[str, ...]) but for more complex stuff it's not possible
https://decorator-factory.github.io/typing-tips/faq/runtime-checking/

[raven prairie]

Context is that I am considering rewriting an in-house build tool in Python and the current scripting language of choice has types.

so it seems like it'd be nice for the build scripts to have enforced typechecking before running.

[soft matrix]

wait hmm?

python has types

[raven prairie]

Indeed.

[soft matrix]

its just all static use a tool like mypy or pyright for this then no?

[raven prairie]

Right, so I can use mypy/pyright from python to check an arbitrary .py file on load before evaling it?

[soft matrix]

yes

[raven prairie]

fantastic

[soft matrix]

misunderstood what you were asking sorry

[raven prairie]

no worries, no offense taken 🙂

[nocturne dew]

Python focuses on a principle called "duck typing," which emphasizes the behavior and capabilities of objects rather than their specific types, maybe try using Decorators and Custom Wrappers to check type during runtime, use assert statements

[rare scarab]

pydantic has an experimental @validate_arguments decorator for that.

https://docs.pydantic.dev/latest/usage/validation_decorator/

[nocturne dew]

how are you going to do it? iterating over whole ast tree to get args and return value and then matching it with custom environment you passed which has types

[slender timber]

https://paste.pythondiscord.com/vijaxijeyi
I am looking for a way to share the common code here, I think of mixin class but that doesn't play well with type hinting

[primal sapphire]

what about the typeguard library?

https://typeguard.readthedocs.io/en/stable/userguide.html

it sounds like exactly what you'd want

[trim tangle]

Oh huh, TIL typeguard only checks the first collection item by default

[slender timber]

This is nice, but I fixed my issue by making a protocol, then inheriting it in the mixin and finally inheriting the mixin in my actual class

[cedar sundial]

I need a little help understanding this change in pyright 1.1.312: Behavior Change: Changed the matching of an unpacked dictionary passed as an argument to a callable when keyword arguments are present and have default values. Previously, the types of all such parameters were not checked against the unpacked dict type (under the assumption that the default argument would satisfy the parameter in this case). The new behavior matches that of mypy and assumes that the unpacked dict may supply arguments for all otherwise-unmatched keyword parameters even if they have default argument values.

I've now a bunch of errors that I'm not 100% sure how to fix. Example:

from collections.abc import Mapping

class Test:
    def __init__(self, a: str = "test", b: int = 1, c: bool = False) -> None:
        ...

kwargs: Mapping[str, str | bool | int] = {"a": "testing", "c": True}
t = Test(**kwargs)

---
t.py
  t.py:8:12 - error: Argument of type "str | bool" cannot be assigned to parameter "a" of type "str" in function "__init__"
    Type "str | bool" cannot be assigned to type "str"
      "bool" is incompatible with "str" (reportGeneralTypeIssues)
  t.py:8:12 - error: Argument of type "str | bool" cannot be assigned to parameter "b" of type "int" in function "__init__"
    Type "str | bool" cannot be assigned to type "int"
      "str" is incompatible with "int" (reportGeneralTypeIssues)
  t.py:8:12 - error: Argument of type "str | bool" cannot be assigned to parameter "c" of type "bool" in function "__init__"
    Type "str | bool" cannot be assigned to type "bool"
      "str" is incompatible with "bool" (reportGeneralTypeIssues)
3 errors, 0 warnings, 0 informations

I can fix this with a TypedDict, but when using classes from other libraries with many parameters, this seems not feasible. Is there a better way to fix these issues?

[soft matrix]

dont use a dict to pass kwargs 😉 (theres nothing im aware of)

[cedar sundial]

What should be used? I've decided to use a TypedDict with total=False which seemed to do the trick as long as I cast the dicts to the TypedDict class. Whether that's the best play, idk

[soft matrix]

that doesnt seem like the best idea for long term maintainability because it involves a lot of code duplication

is there a reason why you arent just directly passing the kwargs?

[cedar sundial]

For my use case, we are applying different parameters based on different ML models, so we're updating a param dict with the parameters and then creating an instance of the model which reduces duplication in our code base

[soft matrix]

hmm maybe you want Unpack[TypedDict] for the stuff you have control over but i dont think theres anything better for the stuff you dont have control over

[cedar sundial]

Yeah, can only see it being used in callable signatures which I don't have here. I'll have to stick with the total=False and cast for the time being

[soft matrix]

is init not a callable signature 🤔

[cedar sundial]

Yes, but I can't do Unpack[TD] on it since the model is from a 3rd party library 🙃

[soft matrix]

oh ic

[slender timber]

how do i type this?

def basic_repr(cls: type[T]) -> type[T]:
    def __repr__(self: T) -> str: ...
    cls.__repr__ = __repr__
    return cls

[soft matrix]

thats fine no?

[hallow flint]

i'd type ignore the assignment to get rid of "cannot assign to a method". for the "incompatible types" your options are type the monkey patched method as taking object (or don't type it at all) or type ignore again

[soft matrix]

when i do something similar i dont get any complaints```py
class _IDComparable(Protocol):
id: Any

TypeT = TypeVar("TypeT", bound=type[_IDComparable])

def impl_eq_via_id(cls: TypeT) -> TypeT:
def eq(self: _IDComparable, other: object, /) -> bool:
return self.id == other.id if isinstance(other, cls) else NotImplemented # type: ignore

def __hash__(self: _IDComparable) -> int:
    return hash(self.id)

cls.__eq__ = __eq__
cls.__hash__ = __hash__
return cls

[slender timber]

Is there a type hint for "not None"

I need it for an overload

[frigid jolt]

bool

[slender timber]

?

no

[frigid jolt]

yes

!e print(not None)

[rough sluice]

@frigid jolt :white_check_mark: Your 3.11 eval job has completed with return code 0.

True

[frigid jolt]

that is bool

[soft matrix]

just use overload order

itll probably work

[slender timber]

huh?

nvm i figured it out

[cyan glen]

How do you type hint a dict of str key value pairs, but also can be empty?

[soft matrix]

You can't encode the emptiness part because mutability is a thing