type-hinting

[muted iron]

It's like a plug in system I suppose. run doesn't know what it may run.

[trim tangle]

The extra types would be useful if you did py foo = run("f") but if I understand you correctly, you're doing ```py
for row in csv_rows:
fname = row["fname"] # fname: str
foo = run(fname)

[muted iron]

run may return tuples and other arbitary types. The names of the functions may come from a csv, some are hard-coded. run is used in multiple places.

[trim tangle]

In the hardcoded places, why not just use f() directly?

[muted iron]

DW about it. It's a rabbit hole :)

[viscid spire]

I think you need to go down the rabbithole

[trim tangle]

You could do something like ```py
class Key(Generic[T]):
def init(self, name: str) -> None:
self._name = name

@property
def name(self) -> str:
    return self._name

class Store:
def init(self):
self._store = {}

def get_by_name(self, name: str) -> Callable[[], object]:
    return self._store[name]

def __setitem__(self, key: Key[T], fn: Callable[[], T]) -> None:
    self._store[key.name] = fn

def __getitem__(self, key: Key[T]) -> Callable[[], T]:
    return self._store[key.name]

F_KEY: Key[int] = Key("f")
G_KEY: Key[str] = Key("g")

store = Store()
store[F_KEY] = f
store[G_KEY] = g

h = store[G_KEY] # h: Callalbe[[], str]

But I have a feeling that you have overcomplicated something that could've been simpler

You cannot do this with string literals without writing all the overloads out

definitely not with globals()

[muted iron]

You cannot do this with string literals without writing all the overloads out
Yeah, that would make a big mess, since the f's are in 10+ files. They also have different paramspecs too.

definitely not with globals()
That was just me simplifying it for discord.

They're methods on a big class.

[trim tangle]

You can type it if you explicitly restructure it to a mapping like the above demo^

But there's no way to link a string literal and an attribute on a class

[rustic gull]

I'm a bit confused with the weakref.ReferenceType class. I can subscript it as ReferenceType[T] (since it implements __class_getitem__ but calling it I get an expected return type of Any | None when I'd expect T | None. Is there a specific reason for this? Shouldn't the class just be generic over some T with __call__ -> T?

[tranquil turtle]

this looks fine: https://github.com/python/typeshed/blob/main/stdlib/_weakref.pyi#L23-L30

[rough sluice]

stdlib/_weakref.pyi lines 23 to 30

class ReferenceType(Generic[_T]):
    __callback__: Callable[[ReferenceType[_T]], Any]
    def __new__(cls, __o: _T, __callback: Callable[[ReferenceType[_T]], Any] | None = ...) -> Self: ...
    def __call__(self) -> _T | None: ...
    def __eq__(self, __value: object) -> bool: ...
    def __hash__(self) -> int: ...
    if sys.version_info >= (3, 9):
        def __class_getitem__(cls, item: Any) -> GenericAlias: ...```

[solemn sapphire]

You should try it in the playground

[topaz parcel]

yeah

[obsidian oar]

https://github.com/CarliJoy/intersection_examples
it's possible? Can anyone tell me if it was already approved, if it will be released in a later version or if it was rejected?

[soft matrix]

its not been approved yet no

its not rejected either though

[tranquil robin]

Been digging Mypy! One issue I've been having is that it seems to really not like dictionaries with arbitrary contents though? I have a bunch of helper functions for manipulating dicts that contain ECS config stuff - is there a way to stop Mypy from freaking out about them?

Without manually doing it for every function lol

I also don't want to have to use TypedDict's for everything

[trim tangle]

If you don't have to use mypy, I'd recommend trying pyright. It integrates with VSCode (as Pylance, make sure to set the type checking setting to basic or standard), vim, emacs, sublime and many other editors

[trim tangle]

Do you have an example maybe?

[tranquil robin]

Like anything that uses the Dictionary Manipulation functions from this (one of my favorite packages!) https://toolz.readthedocs.io/en/latest/api.html#dicttoolz

[trim tangle]

oh yeah that's not typeable in python

best you can do is dict[str, Any]

[tranquil robin]

I'm using a mypy plugin for VS Code. Which is convenient, but also splatters all its complaints on my static files lol

[trim tangle]

If you're using VSCode:

1. disable/remove the mypy plugin
2. make sure you have Pylance installed
3. go to settings -> find "type checking" and switch it from "off" to "standard" (or "basic" if that's not an option)
   you will get errors as well as autocompletion and types on hover

[tranquil robin]

Ah, neat, thanks!

[trim tangle]

If you want to type these, you can write a mypy plugin. I have an example here: https://github.com/decorator-factory/mypy-plugin-attempt, but it's very poorly documented and it's really hard in general

[tranquil robin]

Neat, thanks!

I might bug a package we're a customer for to do this. Mypy freaks out whenever we use some of the optional args on Prefect tasks & flows.

[trim tangle]

if you're in some corporate setting where mypy is already extensively used in CI, it's probably not worth picking another type checker

[tranquil robin]

It's not, at least not for my team! I just use it cuz on-balance it saves me more headaches than it causes lol

[fierce ridge]

you can do better with TypedDict or at least being more precise about the value types

[trim tangle]

I meant that you cannot describe these generic functions that manipulate dicts

[muted iron]

from collections.abc import Iterable

class Position(Iterable[int]):
    def __init__(self, x: int, y: int) -> None:
        self.x = x
        self.y = y

    def __iter__(self) -> Iterable[int]:
        yield from (self.x, self.y)

x, y = Position(1, 2)  # "Position" is not iterable  "__next__" method not defined on type
print(x, y)

This runs, but Pylance doesn't like it.

Bug?

[brisk hedge]

try annotating the iter return type as Iterator instead of Iterable

[muted iron]

You had me at Iteator. That was it. Thanks.

Ok, next issue.

from collections.abc import Iterable, Iterator

class Position(Iterable[int]):
    def __init__(self, x: int, y: int) -> None:
        self.x = x
        self.y = y

    def __iter__(self) -> Iterator[int]:
        yield from (self.x, self.y)

pos = Position(1, 2)
x, y = pos     # yes
x, y, z = pos  # no

Can I teach it somehow to understand that it's always 2 elements long? Like, calling tuple(position) should be tuple[int, int] instead of tuple[int, ...]?

[soft matrix]

coulld probably just subclass tuple[int, int]

[brisk hedge]

Sounds like a good use for namedtuple!

[trail kraken]

I remember reading an article about not using it.

I don't remember the reasoning though... Immutability being surprising? Might have been something else.

[trim tangle]

It adds stuff like length, indexing etc. which you might not need

So for a generic "Record" it might not be the best

Immutability is not surprising if you have a namedtuple, and IMO stuff should be immutable unless you explicitly need mutability

[trail kraken]

That's true. I think I would expect tuple behaviour if I look for namedtuple.

Not doubting the recommendation in this case. I think it makes sense for Position unless there are other requirements.

[muted iron]

coulld probably just subclass tuple[int, int]
Sounds like a good use for namedtuple!
I agree. That's what I started with, a typing.NamedTuple.
However, I am trying to serialize this using the built-in json library, and all sub-classes of tuple are serialized as lists, so I lose the type after deserializing it.

I need something that is treated like a tuple by the typing stuff, but as object by json, so I can use a custom JSONEncoder with object hook.
That method doesn't fire for sub-classes of tuples.

[muted iron]

I settled for

@dataclass(frozen=True)
class Position:
    x: int
    y: int

    def as_tuple(self) -> tuple[int, int]:
        return self.x, self.y

and using as_tuple in a million and one places.

[glass lily]

If I'm creating a custom iterable class which works similar to a list

Should I use Iterable or Sequence

[tranquil ledge]

Probably the latter, assuming "similar to a list" means you can index it and get its length.

[glass lily]

Rockin

I discovered how to use Cython to implement a private dimension within an API

Which is awesome - because privacy is both sane and essential

And it opens the door for immutable sequences

With custom behaviours as needed

[ruby ember]

I have some code that uses SQLAlchemy, and I want to typehint session.fetchall(). I know the table / return types, so am wondering if rows : list[tuple(int, int, str)] = session.fetchall() would be how this is typehinted ? It's actually a list of sqlalchemy.engine.row.Row, but that's pretty vague, and I can't seem to find something clear via google... It seems I can either do

list[Row] - typehint it's a row , but say nothing of its structure, or
list[tuple(<types>)], type hint the structure of the row, but lose information about it actually being a list of SQLA objects

[bleak imp]

The SQLAlchemy docs for fetchall have it as method sqlalchemy.engine.CursorResult.fetchall() → Sequence[Row[_TP]]. The docs for the Row type say "The Row object seeks to act as much like a Python named tuple as possible." Row itself can be subscripted, so why not list[Row[int, int, str]]? If not, and you want specific info, you should use a namedtuple for your typing since Rows have methods like Row._asdict, which a tuple doesn't have so you would get an error/warning in your type checker.

[eager vessel]

Maybe use orm models? 🤔

They'd give a lot more context, e.g. Book is better than Row[tuple[int, str, str, int]]

[ruby ember]

In this context it'd be returning a subset of the books columns via a raw SQL query

[eager vessel]

Well, wrap everything into a namedtuple or dataclass yourself

You can also return a subset of fields using orm, but there's no way to indicate that using type hints

[heady marlin]

Hi folks, I am using cffi to interface with some Win32 APIs and have the following ignore set on my import of the library itself:

from .winffi import (
    crypt32,  # type: ignore[attr-defined]
    ffi,
    raise_windows_error,
)

Below that, I am accessing a function on the library which mypy won't know exists:

...
    context = crypt32.CertCreateCertificateContext(
...

Despite the ignore on the import, mypy is still reporting the issue:

oriel\windows\certificate.py:56: error: "Lib" has no attribute "CertCreateCertificateContext"  [attr-defined]

Any ideas?

I have also attempted to put all imports on the same line and ignore them, also have tried # type: ignore and none of them seem to work. Using a # type: ignore[attr-defined] on the respective line of code does work, but using it at the import level would be way cleaner as I call many functions on the library throughout the module.

[tranquil turtle]

crypt32: Any after imports should fix the problem

[heady marlin]

thanks, let me give that a try 🙂

sadly no pickles,

oriel\windows\certificate.py:8: error: Name "crypt32" already defined (possibly by an import)  [no-redef]
oriel\windows\certificate.py:58: error: "Lib" has no attribute "CertCreateCertificateContext"  [attr-defined]

I suspect my only option may be to do what I'm doing and ignore each line that makes a function call. 😦

[tranquil turtle]

is crypt32 an instance of Lib?

[heady marlin]

Interestingly, I found another library that uses cffi that also does this the way I do
https://github.com/yifeikong/curl_cffi/blob/main/curl_cffi/curl.py

[heady marlin]

Yep it is indeed 🙂

crypt32 = ffi.dlopen("Crypt32.dll")

[tranquil turtle]

if TYPE_CHECKING:
   crypt32: Any
else:
  from ... import crypt32
``` try this

[heady marlin]

ah nice, will do

also, this seems to work
crypt32: Any = ffi.dlopen("Crypt32.dll")

i'll try your solution too 🙂

[tranquil turtle]

i believe the issue is fixable by adding Lib.__getattr__ annotation, if it is possible

[heady marlin]

Ah yes, and by the way, thank you, your solution also worked!!

🙂

the truth is that it's very hard to type check cffi libs, even if I annotated getattr, as there are many functions available and none are typed

but I may see if I can try to add type hints somehow 🙂

[tranquil turtle]

yeah, this kind of code is very dynamic and doesn't play well with typechecking

[heady marlin]

yah totally, this is why I wrapped all this functionality into nicely typed Python functions and have plenty of unit tests to cover them

[tranquil turtle]

def __getattr__(self, attr: str) -> Any: ...
``` this should do the trick
if you are changing .py code, then you should merge these annotations into existing `__getattr__` method signature (im pretty sure it exists)

[heady marlin]

thanks heaps 🙂

[tranquil turtle]

https://github.com/python/typeshed/blob/main/stdlib%2Fctypes%2F__init__.pyi#L70

[rough sluice]

stdlib%2Fctypes%2F__init__.pyi line 70

def __getattr__(self, name: str) -> _NamedFuncPointer: ...```

[tranquil turtle]

this is how ctypes did it for their libs

[heady marlin]

beautiful, thank you!

[heady marlin]

Okies I've just got one more question if I may. I'm using the win32 library which has types defined in the typeshed repo via the types-pywin32 package. Unfortunately, some of the annotations in typeshed are incorrect. I hope to soon send a PR to attempt to correct them, but in the meantime, is there an easy way to override a type definition for a function.

e.g. https://timgolden.me.uk/pywin32-docs/win32security__GetSecurityInfo_meth.html

This should be:

def GetSecurityInfo(handle: _win32typing.PyHANDLE | int, ObjectType: int, SecurityInfo: int) -> _win32typing.PySECURITY_DESCRIPTOR: ...

But it is defined as follows in typeshed:

def GetSecurityInfo(handle: int, ObjectType, SecurityInfo) -> _win32typing.PySECURITY_DESCRIPTOR: ...

I'm passing a handle to the function and of course getting a mypy warning about a mismatching type.

I might just spend the time now and try to create the PR 🙂

[tranquil turtle]

you can find the place where these type stubs are located and fix them manually
when mypy will look for them again it will see new updated version

it is not a permanent solution, of course: after that you should either create an issue or create a PR that fixes it

[heady marlin]

yeah understand, thanks heaps

I'll modify them in the virtualenv just to test that my updates are correct and submit a PR

just going through the contribution guidelines for typeshed atm

[tranquil turtle]

stubs are located somewhere in yourpython/lib/site-packages/

if lib is called X, then stubs module is usually called X-types

[heady marlin]

yeah I have indeed found them there, thanks so much

[round estuary]

With Pydantic, I want the json schema to include the description field. How can I make this work?

    def my_function(self, query: str = Field(description="This is a query string")):
        return do_something()

print(TypeAdapter(Simple().my_function).json_schema())

This outputs:

But I'd like the description to be included.

[pliant vapor]

i want to use this from typeshed
is the recommended way to pip install typeshed_client and import it from there?

[rough sluice]

stdlib/_typeshed/__init__.pyi line 85

SupportsRichComparisonT = TypeVar("SupportsRichComparisonT", bound=SupportsRichComparison)  # noqa: Y001```

[muted iron]

class C:
    def __init__(self, optional: int | None = None) -> None:
        self.optional = optional

o = C(1)
match o:
    case C(optional=None):
        pass
    case C(optional=value):
        print(value + 1)   # Operator "+" not supported for "None"

Pylance can't narrow "value" down to int inside a match. Any reason why?

[trim tangle]

Sounds like a bug

[lunar dune]

You can pip install useful_types if you need a version you can use at runtime (it's a package maintained by typeshed maintainers)

[supple lily]

I'm using Pylance in vscode and it works great. there's a bunch of "benign" errors i can't be bothered to fix, but for the class of errors that will break my code (like missing a function arg), I'd like my program to run a type check programmatically and crash if it fails. Is there an an easy way to do this with Pylance?

[muted iron]

What I am doing is just setting python.analysis.diagnosticMode to "workspace", and then pressing F8 before attempting to run anything.

[hollow bane]

Hi. isort is making this changes idk why suddenly it's happening. I'm using black + isort + ruff + mypy

here is my pyproject.toml https://github.com/alienrobotninja/bee-py/blob/main/pyproject.toml

[rare scarab]

What you're talking about is static runtime checking.

You could run mypy on it before running

e.g. mypy . && python main.py

pydantic is another option.

as a replacement for type-safe dataclasses that is

and maybe functions

I don't recommend it though. There's a huge performance drawback to doing runtime checking

[trim tangle]

You can run pyright from a CLI as well

(pylance is based on pyright but is closed source)

[pliant vapor]

nice thx

[vivid ore]

Does anybody know why this doesn’t work?

from typing import Generic, TypeVar

T = TypeVar("T")

class X:
    pass

class Y:
    pass

class A(Generic[T]):
    pass

class B(A[X]):
    pass

class C(A[Y]):
    pass

def f(a: T) -> A[T]:
    if isinstance(a, X):
        return B()
    elif isinstance(a, Y):
        return C()
    else:
        raise TypeError

the error is

22: error: Incompatible return value type (got "B", expected "A[T]")  [return-value]

[soft matrix]

the is one of the rare times you should use typevar constraints

def f[T: (X, Y)](a: T) -> A[T]:
    if isinstance(a, X):
        return B()
    elif isinstance(a, Y):
        return C()
    else:
        raise TypeError
```works

[vivid ore]

Is there a way to emulate this pre-3.12?

Does [T: (X, Y)] behave like bound=Union[X, Y]?

because that doesn’t seem to work

[soft matrix]

yeah make a new TypeVar("U", X, Y) and use that

[vivid ore]

now it’s complaining about B()

Incompatible return value type (got "B", expected "A[Y]")

[soft matrix]

probably a mypy bug

works with pyright

maybe youll just have to use overloads

[hollow bane]

can anyone help me with this ?

[oblique urchin]

It's sort of off-topic for this channel since it's not about type hinting, and it's not clear from your screenshot what's wrong. Seems like a perfectly normal isort change

[trim tangle]

that font 💀
for some reason it feels non-monospace but it actually is monospace

[hollow bane]

isort is kind of breaking the multi_line_output = 3 settings

[unborn anchor]

hinting

[vivid ore]

I don’t see what’s so bad about it

[trim tangle]

serifs

[rustic gull]

cursed code

[fierce ridge]

why is issubclass(dict[str,int], dict) False?

[viscid spire]

!e

print(issubclass(dict, dict))

[rough sluice]

@viscid spire :white_check_mark: Your 3.12 eval job has completed with return code 0.

True

[viscid spire]

does seem to be a bit strange yeah, because a type is regarded as a subclass of itself

[supple flax]

https://docs.python.org/3/library/stdtypes.html#types-genericalias

<class 'types.GenericAlias'>
>>> type(dict)
<class 'type'>```

[fierce ridge]

oh, that makes sense

unfortunate, but makes sense

[supple flax]

it be how it is 🙂

[viscid spire]

if issubclass worked for generics, then that might open up to working with isinstance which we don't really want

as that incurs heavy runtime costs

so if it can't work well with isinstance, it probably shouldn't work with issubclass

[trim tangle]

Well no, it's just not possible

If you consider dict[str, int] in the typing sense, and not just in the "a dictionary object that happens to have strings as keys ans ints as values right now"

[viscid spire]

not possible in general I guess

you can loop through each item to determine that they match the types

(which sucks)

[trim tangle]

def foo(x: object):
    if isinstance(x, dict[str, int]):
        x["bar"] = 42

empty: dict[bool, list[int]] = {}
foo(empty)

[viscid spire]

ok fairnuff

but that is a philosophical question also

[trim tangle]

philosophically, you cannot determine the "type" of a dict/list just by inspecting it in runtime

it's also not quite clear where this check would be useful

[viscid spire]

I'm talking about like "every x in this room is y" when there are no x's in the room

and whether that is considered a true or false statement

[trim tangle]

I think either choice leads to problems if you do this kind of runtime check

[viscid spire]

yes

I am not saying that it isn't problematic

[trim tangle]

there are more examples btw, like saving that dict for later use

class ReliesOnDict:
    def __init__(self, things: MutableMapping[str, int]) -> None:
        self._things = things

def foo(x: object):
    if isinstance(x, dict[str, int]):
        return ReliesOnDict(x)
    raise TypeError

hmm: dict[str | int, bool] = {"a": True, "b": False}
r = foo(hmm)

[trim tangle]

Suppose you want to do something like typeguard and basically automate typing checks at the library boundary. At the boundary you'll probably have Iterable or Mapping instead, not just dict[str, int]

Of course you could special-case a dict check since that's the most common Mapping you recieve

[hallow flint]

(runtime checks also just not workable for many kinds of generics. think about Callable or Generator)

[trim tangle]

Yeah I have a whole page on this weirdly specific topic
https://decorator-factory.github.io/typing-tips/faq/runtime-checking/

[rustic gull]

How do I un-stringify the type inside a TypeAliasType object in python 3.11?

Value = TypeAliasType("Value", int | list["Value"])

# I want this.
Value.__value__ == int | list[Value]

For functions, classes, etc we have the get_type_hints function which can do this for them but not for typing_extensions.TypeAliasType

[rustic gull]

def get_type_alias_value(obj: TypeAliasType) -> object:
    globalns = getattr(sys.modules.get(obj.__module__, None), "__dict__", {})  # type: ignore
    localns = dict(vars(obj))
    return typing._eval_type(obj.__value__, globalns, localns)  # type: ignore

This is evil

[trim tangle]

https://github.com/microsoft/pyright/releases/tag/1.1.345 😦

Removed support for Union[*Ts] and Union[*tuple[...]]. This functionality was included in an early draft of PEP 646 but was dropped in the final spec. The functionality can still be used in pyright if useExperimentalFeatures is set to true, but it will likely be removed entirely in the future.

[viscid spire]

wait what

that wasn't spec??

!pep 646

[rough sluice]

**PEP 646 - Variadic Generics**

Status

Accepted

Python-Version

3.11

Created

16-Sep-2020

Type

Standards Track

[trim tangle]

Hmmm

[trim tangle]

import random
from typing import TypeVarTuple

Ts = TypeVarTuple("Ts")

def foo(bar: tuple[*Ts]):
    x = random.choice(bar)
    reveal_type(x)

What the hell is the type of x here?

I think it's going to cause the same "phantom type" problem, where a type exists in the type checker's head but there's no way to spell it

Like with intersections

if isinstance(thing, Foo) and isinstance(thing, Bar):
    reveal_type(thing)  # pyright reveals this as "intersection of Foo and Bar"
                        # but there's no way to annotate it

[trim tangle]

right now pyright reveals it as Union[*Ts] btw

[viscid spire]

tbf, shouldn't you choose tuple[T, ...] in the random scenario?

or well ok... tuple[Any, ...]

[trim tangle]

Perhaps, but this could come up in a different scenario. Like if you have a class already parameterized by *Ts

granted, this is pretty fringe

[viscid spire]

yeah

I don't know why they wouldn't have included it in the pep

[strange tree]

Hi all! I've noticed that making a class inherit a protocol e.g.

class A(Protocol):
    def a(self): ...

class B(A):
    pass

often my LSP (pyright) wouldn't report an error.
Ofc protocol is not ABC, in fact I find "inheriting" a protocol rather confusing. Indeed at runtime B does have a method A AFAIK?
Has a "type" Implements[P] or similar been considered to mark that a class should verify a protocol?

class B(Implements[A]):
   pass  # error

[trim tangle]

I do get an error with pyright https://pyright-play.net/?strict=true&code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAK4MYAxmADYBQVpFAhgM6NQCCAFEWCeRQJQAuKlBFQAJgFNgUeu0YSKwQVAB0amnSYsAQu1aDhohFqA
I think it's not enabled on "basic", you'll have to turn some option on

tbh I find the basic settings to be baffling, they're missing some very basic stuff like incompatible overrides

[viscid spire]

imagine if we got an implements kwarg to the class header, kinda like metaclass

surely would fit in a "typed python" re-implementation

[strange tree]

What'd be the advantage of a class kwarg in your POV?

Looking at the PEP (https://peps.python.org/pep-0544/#explicitly-declaring-implementation) I believe that the main advantage of protocols (avoiding inheritance) was disregarded when designing explicit declaration

[wheat lynx]

hi guys, I have this code snippet:

from typing import Any, Generic, TypeVar


A = TypeVar('A')


class Ham(Generic[A]):
    def a(self) -> A:
        ...


HamT = TypeVar('HamT', bound=Ham[Any])


class MyType:
    def foo(self) -> str:
        return 'foo'


class HamA(Ham[MyType]):
    ...


class Spam(Generic[HamT]):
    _ham: HamT

    @property
    def ham(self) -> HamT:
        return self._ham

    def do_ham(self):
        return self.ham.a()


my_ham = Spam[HamA]().do_ham()

the problem here is I want to annotate the return type of do_ham of Spam. I can clearly see that it returns the ReturnType of Ham.a, which is A, however A can't be annotated as it's unbound, and I can't find anything that seemed fit

my auto complete can still understand my_ham has the type MyType even with no do_ham return type annotation (it suggested method foo, which only available if it knows my_ham is of MyType). reveal_type(my_ham) returned Any instead

I could work around this case by adding A as a generic type param in Spam, however I see it a bit inconsistent as A is not compatible with A in HamT

I would like to ask for the correct way to handle this case, as well as finding out the correct type annotation for do_ham. in case I misunderstood something conceptually, please enlighten me. TIA.

[paper salmon]

this is something i've only encountered once myself, and i tried handling it with the same workaround of propagating the type vars down, but since i had 3 type vars that needed to be present in multiple classes, it was very verbose and confusing and i wish i didn't write it

i believe it's the lack of higher-kinded types that makes it difficult to more easily represent it as def ham(self) -> HamT[A]

[heady flicker]

Yeah. I think, one of sobolevn's libraries can do higher kinded types using mypy. Probably 'classes'

[wheat lynx]

@paper salmon thank you for referring me to the correct thread. As for now I tried do not directly tackle this issue by juggling some parts of my code, but I clearly see the future problem if my code base gets more logic to be implemented
I would like to hear more opinions about my situation and hopefully could see some approach of fixing this, possibly using HKTs so that I could learn something aside from just dig in those long threads/documentations
So, given the least of supporting from mypy (merely just 1.8.0 for now), how could I annotate the do_ham to satisfy mypy, or should I rewrite the code in a different way somehow?

[muted iron]

class C:
    @functools.singledispatchmethod
    def f(self, idx_or_lst):
        raise NotImplementedError

    @f.register
    def _(self, idx: int) -> int:
        return idx + 1

    @f.register
    def _(self, lst: list) -> list:
        return lst + [None]

reveal_type(C().f(2))  # Any
reveal_type(C().f([2]))  # Any

I feel like type checkers should be able to infer the types here. Why doesn't it work?

[pastel egret]

It's possible, but singledispatch is doing a unique thing here, the type checker would need to have a plugin or other special support to recognise it.

Most likely be converting that into an overload.

[muted iron]

Yeah, overload works, but it's becoming quite a monster.
singledispatch a stdlib thing, so one would expect it to behave smoothly with other stdlib things.

[pastel egret]

Mypy has a plugin for functools.singledispatch, but not singledispatchmethod it seems.

There's some discussion here about the difficulty with supporting checks: https://github.com/python/mypy/issues/11727#issuecomment-993150228
In particular, there's the problem that you can call register() from anywhere, meaning importing other modules might change the signature of a function. So it's hard to determine which order modules need to be checked in...

[muted iron]

Thanks for the link, looks interesting.
Yeah, I understand the issue better now.

[odd horizon]

what was the official reason again for not having type hints directly in python3, but instead in typeshed?

[oblique urchin]

The initial reason was that types were experimental and there was a strong desire to avoid affecting the rest of the language and standard library

Nowadays the system works fairly well and there is a lot of complexity involved in using types directly from the standard library instead of typeshed

[odd horizon]

makes sense, I remember something about not wanting to force people to use types too (that it is optional) since it is a dynamic language

[oblique urchin]

Sure, but putting types in the stdlib wouldn't force anyone outside the stdlib to use types. However, especially initially there was a lot of resistance among the core developers to typing, so that definitely played a role

[hallow flint]

unfortunately it would still be a bad idea, here's a summary of some of the issues: https://discuss.python.org/t/type-annotations-in-the-stdlib/21487/7

[fierce ridge]

is there a way to define a protocol for any type that implements a "total ordering"? ideally it would check all valid combinations of __le__, __eq__, etc.

or is there maybe a mypy plugin that can do it?

[soft matrix]

have you checked _typeshed/useful-types? id be surprised if it doesnt already exist

[fierce ridge]

i haven't, i'll take a look

https://github.com/hauntsaninja/useful_types/blob/951d36b2aaaee6ab9fa1efcb8c4f1e2733247ff5/useful_types/__init__.py#L29-L60

that looks good for me

somehow i expected it to be more complicated

[vivid ore]

Was pathlib.Path.walk() introduced or phased out in a recent version? I don’t seem to have it

[oblique urchin]

https://docs.python.org/3.12/library/pathlib.html#pathlib.Path.walk

new in 3.12

[vivid ore]

thanks!

[shadow jasper]

is there a way to enforce immutability?

in classes, and data structures in general

[bleak imp]

For making something immutable, no. You can make it very hard to change the value of something, but even tuples can have their values changed if you try hard enough. For most applications making __setattr__ raise an error, or using names starting with two underscores __ for automatic name mangling, or using dataclasses with frozen=True is more than enough, since if someone really wants to bypass it and breaks things, that's their fault and not your responsibility. The entire discussion is really good, but see this post in particular https://discuss.python.org/t/immutability-in-python-is-really-hard/2536/16
As for enforcing immutability, I'm not aware of anything that exists, but it would be an extremely bad idea, since you would basically have to store a duplicate of every single immutable object and check any new objects against that set.

[shadow jasper]

yeah I mean, just using whatever will make me not accidentally mutate things is enough for me, I don't necessarily need literal immutability

thanks for the reference!

any of you guys try to implement a bit of functional programming concepts in Python?

[bleak imp]

This channel is for type hinting related things. If you want to talk about general python stuff, please use #python-discussion (though I will say that type hinting functional style code in python is an absolute pain)

[shadow jasper]

might not be obvious but a lot of functional programming stems out of making proper types

especially e.g. in haskell

[bleak imp]

The issue is type hinting those proper types. Some examples that I recently researched are that there is currently no standard way to type hint an Intersection or Get the type from a variable or Higher-Kinded TypeVars especially.

[reef yacht]

I have a pyi stub file. It provides stubs for a file that has a dataclass in it. If I don't supply the dataclass in the stub file, pycharm screams at me. But if I do, it screams at the instantiations of that dataclass ("invalid arguments"). Anyone know what I should do?

[viscid spire]

how did you write your dataclass in the stub file?

[reef yacht]

I've tried a few things, but right now it's just copy-pasted :P

I think it's getting confused by it not having a constructor.. but I would also expect it to handle that if it's got the @dataclass decorator

[viscid spire]

I believe that is what you should do. What is PyCharm screaming about specifically?

[reef yacht]

If I try to make an instance of the dataclass, all the (keyword) arguments are marked as "invalid argument"

If I delete the pyi file then it's fine

[viscid spire]

I assume you are using dataclasses.KW_ONLY?

[reef yacht]

I'm not, but I mentally think of it like it. Positional arguments are evil in general so...

[viscid spire]

ok then you are using the kw_only kwarg of the dataclass decorator?

[reef yacht]

adding kw_only=True doesn't make a difference

[viscid spire]

ok well, if you were, those were only added in 3.10, so they aren't spec'd for stub files

I forgot which version stub files are supposed to be in (although tbf many typecheckers will understand a stub file in a version higher than <whatever version stub files are in>)

[reef yacht]

In any case, the normal mode is for arguments to be "normal" parameters if you don't specify

[viscid spire]

yeah yeah

so pycharm is just stupid I guess or something

(which I wouldn't put past it, I have heard about some issues with its typechecking)

[reef yacht]

hmm, yea ok. I guess I have to file a bug. This really puts a damper on my little project :/

Ok, so.. can I just not include my dataclasses in the pyi and have it not be angry at that?

[viscid spire]

you can lie about them being dataclasses maybe

[reef yacht]

You mean write the constructor?

[viscid spire]

yes

[reef yacht]

hmm.. that's... horrible heh

[viscid spire]

yeah it's horrible

[reef yacht]

I am generating this programmatically btw

[viscid spire]

just giving anything suggestion-wise xd

[reef yacht]

AST manipulation and all that

I guess I can do this, but it's gonna be ugly :/

[viscid spire]

at least it's a fun programming challenge, even if it doesn't work

[reef yacht]

A bit annoying heh. I am making a hack to prove a point about my idea from my blog https://kodare.net/2023/02/02/names-can-be-so-much-more.html

Basically that I could have a dict of names->types globally in a project instead of annotating every little usage. It would be cleaner code, and help nudge towards more consistent naming. My hack is to generate stub files based on this logic.

It works extremely well... except for data classes (and pydantic schemas.. for the same reason)

[reef yacht]

got it to work btw, thansk

[bleak imp]

If when you run code through a type checker it becomes type checked, what would you call this operation and the resulting code? Name checking and name checked?

[rare scarab]

type checked

fully typed?

a type checker performs type checking

[trim tangle]

They meant the thing from boxed

with names

[reef yacht]

I doubt that.

[trim tangle]

hm?

[reef yacht]

Although... it does use the word "name" a lot

[reef yacht]

Are you talking about my thing?

[trim tangle]

I understood the question as "how would you call the operation of checking the names in the aforementioned way?"

[reef yacht]

Maybe. Anyway it's not a name checker. It's the normal type checking you already know. Just that you can specify mapping centrally instead of every place in the code.

Oh and you can still override locally

[signal brook]

does anyone know if pydantic can change the type of a key based on the value of another key?

also is there anything better than pydantic or is that like the king

for hinting and validation

[bleak imp]

Maybe I'm not understanding your blog post then? To me it sounds like the names of variables result in type hints, so if the inferred type works but doesn't match the name you'd get some sort of name error.

[fierce ridge]

fancy functional programming languages have something like that ("functional dependencies" in types) but in python you can't do much of anything in the way of type-level computation. the only exception is overloads of functions, where you can have things like

@overload
def f(x: float, y: str): ...

@overload
def f(x: int, y: bool): ...

but there's no mechanism to "overload" attributes of a class, only function parameters and return types

[fierce ridge]

i wouldn't say it's the "king", but pydantic has the most features and is very popular

there are a handful of other options with various tradeoffs and benefits. but pydantic is probably the easiest to use in real code

[signal brook]

oh yeah pydantic docs even recommend typeddicts for model defining if you're focusing on speed too

thats not related to my question, just thought it was interesting

im mainly wondering about the syntax of other libraries. pydantic docs are kinda too loaded and confusing too

[fierce ridge]

The documentation is pretty bad I agree

The syntax is similar in other frameworks

Historically we had the Marshmallow schema validation framework and the Attrs framework for defining classes with tidier syntax

The latter was eventually adopted in reduced form in the standard library as dataclasses

But there was never really a good way to connect classes with their input validation other than writing it all out manually

several attempts were made to either attrs and/or dataclasses with marshmallow, or create a new input validation layer on top of those

you can of course manually validate inputs with attrs and dataclasses, but that doesn't help you if you want to do something like integrate with jsonschema

examples include mashumaro, cattrs, and desert

Pydantic was the first and so far only framework that does both, seamlessly, and in a unified way

replicating that functionality with Attrs and Marshmallow for example requires at least one extra dependency to get Marshmallow to emit jsonschema, and either DIY glue code or a relatively unknown library called Desert

mashumaro has decent adoption in some places eg inside Airflow, otherwise all of the competitors/alternatives are essentially unknown to the broader python community. and generally lack features compared to pydantic

pydantic also has the advantage of fastapi tightly integrating it, as well as sqlmodel and odmantic. so at this point it's picked up quite a lot of momentum

so people put up with its quirks and bad documentation because there isn't really an alternative that doesn't require a lot of fuss and has a decent developer community

i think pydantic v2 is kind of wacky and the verbosity went through the roof, which kind of defeats the original purpose imo. so i think there's room for something new that does the same job but better. but that doesn't exist yet. at the same time, v2 solves some of the problems that v1 had and in particular provides much more control over the generated jsonschema, which is important for web dev.

they're not going to get everything right, in a sense they are first movers exploring a new design space

thank you for my TED talk, I should write a blog post next time 😆

[undone saffron]

msgspec does a very good job of being fast, lightweight, easy, and validates types

I definitely prefer to it over pydantic

[fading thicket]

Would I be able to get some assistance in type hinting when it comes to a dynamic initialization process like getattr()?

For my specific problem, want to have a dynamic instantiation of the ccxt (cryptocurrency exchange library) exchange object by name, but also provide type hinting for the IDE! If I create a function and use the -> ccxt.Exchange, won't this just be a base version (meaning each subclass would inherit from that base class, but have other methods not available to others, I'd like to be able to see in my IDE).

Right now I just initialize them with exchange_class = getattr(ccxt, exchange_id)(), and add that to a dictionary of objects, but when accessing the exchange_class object I get no hints of inside that specific exchange class.

[fierce ridge]

i forgot about msgspec. i should try it. bonus points if it can be made to work with fastapi

[fierce ridge]

getattr returns Any, so you can annotate exchange_class: type[BaseExchange] = getattr(...) and it should work

however the type checker is 100% trusting you there, no protection against a typo in the attribute name

[reef yacht]

That would be ideal I think. But not what my current hack implementation does.

[red hamlet]

Any tools for people who want to be reminded to catch exceptions like Java? I want to be forced to explicitly state the exceptions a function can throw and then be reminded to catch them at some point. Is there such a tool? (PyCharm)

[reef yacht]

That doesn't really make sense. I think you will write very bad python if you try to do that.

[red hamlet]

I'm curious as to why you think so 🙂

[tranquil turtle]

this idea requires enormous amount of effort to become even possible
otherwise all existing code (including stdlib and builtins) will be non-exception-annotated and not be usable with your clever code

[undone saffron]

everything can raise BaseExceptions

[tranquil turtle]

dont forget about KeyboardInterrupt

[undone saffron]

pretty sure that's included there

checked exceptions are exceptionally awful to deal with. Catch what you know how to handle, let the rest propagate, on their own.

[tranquil turtle]

yes, it is
but for most other exceptions we can at least find places where they cant happen, while KeyboardInterrupt can occur everywhere

MemoryError also can occur almost everywhere

[red hamlet]

I see, thank you for your input! I'm always open to learn more. It's a bit hard for me, I moved over from Java a while ago. I just catch myself forgetting to catch exceptions sometimes. How do you deal with that? Or do I just have to git gud?

[tranquil turtle]

as wise man said:

Catch what you know how to handle, let the rest propagate, on their own.

also top-level try-except is nice for catching all uncatched exceptions

[red hamlet]

Alright, I guess I just have to be a bit more careful and learn to deal with it.

[reef yacht]

Let the code crash. Have a way to monitor crashes like Sentry. If you need to catch stuff, add that later.

[undone saffron]

Most libraries document what can raise and not, but those will be limited to things the library intends to raise rather than things going wrong in an unanticipated way. There's probably a way to use static analysis or ast analysis to collect things that are raised by a library explicitly, but putting it into the type system could become problematic, and a python ast based method wouldnt cover extension code.

[reef yacht]

It's MUCH better to crash by default than to catch too much and hide errors.

[undone saffron]

having a crash or top level exception handler (especially a default handler in async code) should let you find what's your problem (reasonable to catch and handle vs things like memory errors)

[red hamlet]

I am primarily talking about my own custom exceptions that I just wanted to be reminded to catch if not done, yet

thanks for letting me know about sentry

[fierce ridge]

Python doesn't have checked exceptions. There have been requests for it occasionally but it's so controversial that i doubt it will ever get first party mypy support

[reef yacht]

(it's also widely believed to be a mistake in Java, and was definitely decided a mistake in C++ and I think removed)

[trail kraken]

What is? Exceptions?

[trim tangle]

checked exceptions

Actually I have no idea how exceptions work in C++

[reef yacht]

It's horribly complicated. Of course ;)

[trail kraken]

Eh. Didn't know what a checked exception is, until just now.

I am not aware of this being a thing in C++.

There is nodiscard for return values. Nothing similar for exceptions as far as I know.

[oblique urchin]

https://learn.microsoft.com/en-us/cpp/cpp/noexcept-cpp?view=msvc-170

[trail kraken]

noexcept just means exception handling can be turned off. Exceptions are still allowed.

It goes straight to terminate if there is an exception.

[oblique urchin]

that's an interesting definition of "still allowed"

[trail kraken]

I thought its problem in Java is the language itself forcing you to check it.

If anything... it is the complete opposite. Like, you are not even allowed to handle it.

I think I am just missing the original point completely 😅

[oblique urchin]

ah this is the syntax for C++ checked exceptions, was hard to Google for: https://stackoverflow.com/questions/1055387/throw-keyword-in-functions-signature

[trail kraken]

Wow. Completely forgot about that part of the language.

IIRC, it essentially acts as a runtime filter for allowed exceptions.

[oblique urchin]

I had vague memories of it, thought the syntax was except(...) instead of throw(...)

[trim tangle]

I am mostly unsure how they do the type checking

Is it something like the Any trait in rust?

[trail kraken]

How does it work in Rust?

[trim tangle]

I have no idea lmao

[bleak imp]

Looks like there is from my 5 seconds of googling. From my sparse rust knowledge, most things are handled by result enums of Result[Val, Err], which might be a better pattern for whatever chris was doing. I think panic! is like a throw, but I think they are almost never meant to be caught, since you should be resulting instead.

[trim tangle]

I didn't mean it was like error handling on panics in Rust

I meant: how do you do py try: ... except Foo: ... except Bar: ... in C++, if there are no runtime types? (and I remembered that Rust has std::any, but then I remembered that I don't know how it's implemented in the compiler)

[oblique urchin]

I think for exceptions the type is stored at runtime. You can only throw things that derive from std::exception, right?

[trim tangle]

Nope, you can throw anything https://stackoverflow.com/q/53629837/10295729

[oblique urchin]

wow. but I guess when the compiler sees a throw statement, it knows what type is being thrown, so it can insert the appropriate logic? I don't know how that would deal with subclasses though

[reef yacht]

If you look long enough into the abyss, the abyss looks back into you.

Suffice it to say that checked exceptions in C++ are the worst. Just horrible.

[trail kraken]

Exception handling piggy backs off of RTTI in C++. It basically means injecting extra data into vtables.

[trim tangle]

In Rust, there's a thing called std::any::TypeId, which is a 128-bit integer number representing a type. The compiler supposedly has one of those values for every possible type

[trail kraken]

I am not sure how that works for non-virtual types being thrown though...

[oblique urchin]

well I got a segfault trying it out 😄 https://godbolt.org/z/vKz4f7ceh

oh that's just the uncaught exception

more interesting version: https://godbolt.org/z/j98oYYcdM. I threw a B but a catch(B) didn't catch it.

So seems like it only uses the static type of the throw statement

[bleak imp]

So you are saying I can't have more than 340 undecillion types in rust? garbage language.

[oblique urchin]

let me tell you about how terrible Python 3.6 was https://docs.python.org/3/whatsnew/3.7.html#:~:text=In Python 3.7 this restriction,12844 and bpo-18896.)

More than 255 arguments can now be passed to a function, and a function can now have more than 255 parameters.

[reef yacht]

I fear the code base that triggered this.

[trim tangle]

And that is how FastAPI was born

The signature of FastAPI.__init__ spans more than 750 lines

although it's mostly docstring spam

I feel like this is the single best counter-example to accepting PEP 727, directly from its author

[bleak imp]

I wonder if unpacking a list with more than 255 elements into a function would cause an error on that. Very glad I didn't program in 3.6

[trim tangle]

I worked on 3.6 code last summer

[trail kraken]

cries in Python 3.5

Can't say I have seen a function with more than 100 parameters.

[reef yacht]

That is bonkers!

[oblique urchin]

I feel like it's more the function itself than PEP 727. The alternative would be a docstring that is nearly as long and not as structured

[trim tangle]

I can easily press ⬇️ in my editor and hide the docstring though

[oblique urchin]

fair

[trim tangle]

There are already standards for describing parameters in the docstring, which do work in editors when you actually want to docs

[reef yacht]

Ah. But only 35 arguments. That typing stuff is quite ugly too.

I think my name to type idea would help here a bit

[trim tangle]

Yeah, there might be a bit of

[bleak imp]

I wonder how you would name something like py servers: Annotated[ Optional[List[Dict[str, Union[str, Any]]]]
Is there some point at which a name would become too convoluted to actually help? And at what point would that be if it exists? I can't seem to come up with a good name for a list of dicts, where the dicts have specific info. Maybe it's more of a case where the dict should be a NamedDict, and it would just be list_of_server_details, but a lot of code won't have that abstraction.

[reef yacht]

"servers" seems like an ok name to me.

Names are domain bounded so that's fine.

[bleak imp]

I suppose the question would be how much type information does/can a name contain? Since servers gives List[Any], while something like list_of_server_details gives List[SomeServerDetailClass]. That's probably something that would be dependent on a case by case basis.

A better inference would be servers -> List[Server] objects, which could either be good or bad.

[reef yacht]

Yea it's per project. You would have a mapping file per domain like:

name=str
count=int
user=something.User

[reef yacht]

Btw I have actually built this as a rough prototype and tried it on my work codebase. It's as nice as my hack allows (I use pyi files)

[tranquil turtle]

there is similar thing in C++: typeid(x) returns reference to type of x

i imagine exception matching works like this:
throw x; knows which type it is throwing, so it can add typeid(x) pointer as additional information
catch (X x) can compare type of raised exception with typeid(X) and do stuff if they match (equal or raised exception is subclass of X (class inheritance info is available at runtime, so it is possible))

[valid river]

Does anyone know if this is a bug?

a: str = "foo\nbar"
b: list[str] = a.splitlines()
# Expression of type "list[LiteralString]" cannot be assigned to declared type "list[str]"

vscode

[oblique urchin]

that's certainly unpleasant and worth reporting to pyright as an issue

the outcome may be that the maintainer says it's a necessary outcome for how the types are declared

[undone saffron]

that's actually working as specified actually, nvm, the explict annotaion on a...

[valid river]

Humm, is it correct to say that str is a subtype of LiteralString? Even it is necessary, it is not correct, as in

a: str = input()
a = "foo"
b: list[str] = a.splitlines()
# Expression of type "list[LiteralString]" cannot be assigned to declared type "list[str]"

a assumes a supertype

[undone saffron]

LiteralStr is more specific than str, so no, it would not be correct, you've got the relation backwards

[valid river]

Ah you are right, but yeah the annotation, this should probably not be happening

[undone saffron]

yeah, pyright should probably be respecting the annotation on a in the first example.

second example seems fine to me, but not ideal either.

[oblique urchin]

yeah it's inferring a more precise "local" type than the annotation. This is useful in some cases but causes problems here

(example: you could declare x: str | None = "x" because you want to set it to None later. It's useful if the type checker doesn't flag an operation on the next line that assumes x is currently a str.)

[undone saffron]

normally it's fine, but LiteralStr is really a refinement type without refinement type support in the system and could probably benefit from some additional language to help specify the behavior that's desired for consistency. I think Literals might be the only case a more precise local type would be a problem rn

I might find some time for that after the stuff with intersections wraps up if nobody else does.

until then, id still raise that as an issue with pyright though, more language for spec good, but this one seems like something they may want to be aware of

[valid river]

https://github.com/microsoft/pyright/issues/7004
Seems the issue is typeshed

https://github.com/python/typeshed/issues/10887

Although in my opinion it should still respect the signature and use the str overload instead

[trim tangle]

I kinda dislike the fact that pyright ignores the explicit annotation if it thinks it knows better

If I write a: str = "foo" there's probably a reason I don't want the type to be inferred

[soft matrix]

i think it probably does 90% of the time

[trim tangle]

wdym

[soft matrix]

im trying to find examples of where it breaks

[trim tangle]

#type-hinting message

[soft matrix]

i meant the other way round

like including the annotation is worse

[trim tangle]

oh

[oblique urchin]

#type-hinting message

[soft matrix]

but i think my issue was more with the annotation not binding the type strictly enough lol

[trim tangle]

Yeah, if you have a union but only one branch is actually used

[soft matrix]

i will admit i have a lot of things that would be nicer if bidirectional inference on it went off

[trim tangle]

In a perfect world... there'd be some kind of linter, like flake8 but having type information

and it would find foo: str | None = "bar" (foo is never reassigned) and flag it as questionable

[bleak imp]

There would also be the same for boxed’s idea.
Also, something that is obviously worse:
x: Any = "a"
Or object.

[undone saffron]

I copied my initial thoughts on this over to that issue, thanks for finding it.

[tranquil turtle]

i guess we need either static reference count analysis or borrow checker for issues like this to disappear

[trim tangle]

Why is it obviously worse?

Maybe I got some nasty union with 10 options, and I just want to treat it as Any

[reef yacht]

I built a hack to try out my idea btw https://github.com/boxed/ivrit/

imo it works quite well

[reef yacht]

Yea, the IDE won't look inside the code of decorators and execute that code to find out the types. That's not how static typing works in python.

From the code you've shown it looks like maybe you just overengineered this and don't need such complicated code.

[undone saffron]

lift the class definition out to the same scope as the decorator, still use it in the decorator, leverage generics and possibly paramspec

[reef yacht]

or in other words: "you just overengineered this and don't need such complicated code" 🤣

maybe, but Sinbads advice is actually the same as what I said :P

Nesting class creation inside a function, inside a function is complex. Probably not a good idea. I say of course probably because we don't have the full context. And even if it's a bad idea, fixing it might be too big of a job of course.

[undone saffron]

not sure it really is [the same advice]. you can do what I said without uncomplicating the code at all, it just places the type information in a way that the type checker can see it in the scope it needs.

I'd probably also consider if a decorator is really the right pattern rather than a mixin based on seeing the usage of that decorator.

[reef yacht]

*coughmultipleinheritacecough*

[undone saffron]

cause all this appaears to be is a decorator that applies a mixin rather than just using the mixin directly

[reef yacht]

At least you should be able to move some of it outside the decorator

[delicate rapids]

Copied some code from https://www.redblobgames.com/pathfinding/a-star/implementation.py and am not understanding the type issues my IDE is complaining about since I'm not very familiar with generics

Location = TypeVar("Location")

class SimpleGraph:
    def __init__(self):
# Type variable "Location" has no meaning in this context [reportGeneralTypeIssues]
        self.edges: dict[Location, list[Location]] = {}
    def neighbors(self, id: Location) -> list[Location]:
# Expression of type "list[Location]" cannot be assigned to return type "list[Location@neighbors]"
#   "list[Location]" is incompatible with "list[Location@neighbors]"
#     Type parameter "_T@list" is invariant, but "Location" is not the same as "Location@neighbors"
#     Consider switching from "list" to "Sequence" which is covariant [reportGeneralTypeIssues]
        return self.edges[id]
```  Could someone explain what these issues are?

[oblique urchin]

you need to make the class generic over Location, otherwise the TypeVars are interpreted as independent

[delicate rapids]

class SimpleGraph[Location]?

[oblique urchin]

yes, in 3.12+

[delicate rapids]

When you say independent, that is saying that the usage of Location in __init__ is different than neighbors? It doesn't matter that they're using the same global variable? How would I note that in 3.11?

Even if independent, isn't dict[Location, list[Location]] still meaningful to say "the keys and the lists of values should match"? Or would even those two usages of Location be independent?

[trim tangle]

nope, that's not a thing

[oblique urchin]

I mean that they're treated as separate TypeVars, so the type checker doesn't know the two usages have anything to do with each other

[trim tangle]

You cannot expess a dict with a relation between the key and the value

[oblique urchin]

Not exactly.

[delicate rapids]

Okay, it looks like class SimpleGraph(Generic[Location]): resolved my issue. If this was a protocol, I'd do class SimpleGraph(Protocol, Generic[Location]):?

And list any other additional generic types as multiple inheritances too?

[trim tangle]

class SimpleGraph(Protocol[Location] as a shortcut

for multiple variables, inherit from Generic[A, B, C]

[delicate rapids]

Is there reasoning behind why that isn't something I'd want to do? Even just saying that all keys are the same type seems like a meaningful thing

I feel like I'm missing something fundemental

[trim tangle]

Any dict satisfies the property "all the keys are of the same type"

Because every key is an object

[delicate rapids]

That makes sense, but then whats the point of:

U = TypeVar('U')                  # Declare type variable "U"

def second(l: Sequence[U]) -> U:  # Function is generic over the TypeVar "U"
    return l[1]```
verses just `l: Sequence`?

[trim tangle]

The point of a typevar is to link two types together. So if you call second([1, 2, 3]) you will get an int, and if you call second("hmm") you get a str

If you have something like "a list of pairs, where each pair has elements of the same type", you can't really do anything with this information

[delicate rapids]

So in that example, the types have to be exact, not just one being a subtype of the other.

[trim tangle]

No

In your case the output will be derived from the input type

But if you had this (pyright-play) ```py
from collections.abc import Sequence
from typing import TypeVar

T = TypeVar("T")

def contains(seq: Sequence[T], value: T) -> T | None:
...

hmm = contains([1, 2, 3], "foo")

if you hover over `hmm` you'll see that it's `int | str | None`

[delicate rapids]

But I couldn't do def second(l: Sequence[U]) -> U: with an int output and bool input or int input and bool output? even though bool is a type of int? It's more akin to type("input") == type("output") (what I meant by exact) then isintances(output, type(input)) or vice-versa?

[trim tangle]

When you call a function with generic parameters, the type variables (in this case U) are "solved" to some particular type. Then the return type will be computed based on that

The type variables are only solved based on the arguments

(because you can't change how the function works by changing the destination type in some way)

[thick quiver]

Anyone use get_type_hints with PEP 695 generic classes? It fails for me...

[oblique urchin]

https://github.com/python/cpython/issues/114053

though maybe you just commented there 🙂

I haven't looked too hard but it may not be easily fixable. In Python 3.13 we'll likely get PEP 649 which will provide a better solution.

Until then, might be better to just not use from __future__ import annotations

[thick quiver]

Caught me 🙂

Gotcha, I'll try that as a workaround.

[signal brook]

TIL the second Client type hint is important

class ExtensionUtils(Extension):
    def __init__(self, bot: Client):
        self.bot: Client = bot

before it was a mixed bag. would show the type in autocomplete, but not anywhere else

[trim tangle]

No, it's not needed

Can you provide some more context? Where do Client and Extension come from?

[signal brook]

it fixed the typehinting and autocomplete for me

[trim tangle]

That sounds like a bug in the type checker 🙂

to be clear, by the second type hint you mean self.bot: Client?

[signal brook]

yes

[trim tangle]

and what do you mean by mixed bag?

[signal brook]

just the screenshot

[trim tangle]

Can you show more code?

[signal brook]

why

[trim tangle]

Because i don't know what code you have

This should not happen: pyright-play
so either there's a bug in the type checker, or you have another issue somewhere

[signal brook]

its an interactions.py bot and thats the extent of the extension in the code above

https://github.com/interactions-py/interactions.py

[paper salmon]

thats with pylance right? occasionally i find myself having to run the restart language server command to fix those unusual bugs

[trim tangle]

yeah that is also a thing

[dusky tinsel]

class UserInfo(BaseModel):
    data: UserSettings
    items: Optional[UserFiles] = None
    
    async def update(self, data: dict) -> 'UserInfo': 

How do i type hint the class name?

[viscid spire]

exactly as that, or use from __future__ import annotations at the top of your script

[dusky tinsel]

Alright, thanks.

[rare scarab]

Also typing.Self

[viscid spire]

Ah true

I might prefer that if the function literally return self

(I know it is valid for returning ClassName[*Ts] too)

[rare scarab]

or cls()

though a @classmethod returning Self is weird if your class is generic.

Why isn't Self itself generic?

[viscid spire]

Self means the same type as self

basically Self == cls[*Ts]

I guess none of that answers "why" tho

[soft matrix]

why what?

[soft matrix]

its not unless ClassName is final

and that might not even be validated by type checkers properly

[viscid spire]

what

[trim tangle]

because the python type system doesn't have higher-kinded types

[viscid spire]

the question that unalivejoy asks literally above my messages

[soft matrix]

you cant have a function () -> Self: return ClassName()

[viscid spire]

I don't know what you mean by "ClassName" being final

I also said ClassName[*Ts] as a type

[viscid spire]

seem this is wrong tho, as Gobot is saying

[soft matrix]

if the class is decorated with @typing.final its final (cannot be subclassed in the eyes of a type checker)

[viscid spire]

that's strange behaviour

how does final = not subclassable?

[rare scarab]

True no subclasses is ```py
def init_subclass(*args, **kwargs):
raise TypeError("Cannot subclass")

which maybe final does. idk

[soft matrix]

its a common thing in other languages

[soft matrix]

it doesnt

[viscid spire]

just the word doesn't really make sense 🤷‍♂️

[soft matrix]

no such thing as a truly unsubclassable type :P

[viscid spire]

if you relate final with Final

[soft matrix]

https://docs.python.org/3/library/typing.html#typing.final

i didnt name it

[viscid spire]

reminds me of Optional

not exactly but 🤷‍♂️

[viscid spire]

reading up, typing.final is a decorator kinda like typing.override

basically a noop, besides adding a attribute sometimes

def final(f):
    """Decorator to indicate final methods and final classes.

    Use this decorator to indicate to type checkers that the decorated
    method cannot be overridden, and decorated class cannot be subclassed.

    For example::

        class Base:
            @final
            def done(self) -> None:
                ...
        class Sub(Base):
            def done(self) -> None:  # Error reported by type checker
                ...

        @final
        class Leaf:
            ...
        class Other(Leaf):  # Error reported by type checker
            ...

    There is no runtime checking of these properties. The decorator
    attempts to set the ``__final__`` attribute to ``True`` on the decorated
    object to allow runtime introspection.
    """
    try:
        f.__final__ = True
    except (AttributeError, TypeError):
        # Skip the attribute silently if it is not writable.
        # AttributeError happens if the object has __slots__ or a
        # read-only property, TypeError if it's a builtin class.
        pass
    return f

[trim tangle]

That's also the terminology Java uses

final variables, final classes

...they probably wanted to reuse the keyword

[rare scarab]

imagine sealed classes

[trim tangle]

not sure we really need those

Python already has unions

[fast ether]

Dose anyone know how to make an .os software?

[viscid spire]

this channel is for typehints and typechecking, not for general trying to get someone to do work for you

[fast ether]

I'm sorry they just I don't really know what type of painting is in I'm very confused about coming I'm sorry

[keen socket]

whats the recommended way to validate types of arguments?
like ```py
@post('/save')
def f(x: int, y: float) -> Response:
if x_or_y_have_unexpected_types():
return Response(400)

save_to_db()
return Response(200)

[wooden bay]

if you're using fastapi, it'll handle it for you
else,

    if not isinstance(x, int) or not isinstance(y, float)```

[keen socket]

oh, is there no better way than checking it manually for every arg?

[wooden bay]

i mean you could write a decorator

[trim tangle]

There's a difference between parsing data you get from HTTP and validating values within your program

For HTTP validation, there are already existing solutions in frameworks (like pydantic in FastAPI, and whatever litestar uses for validation)

If you want to ensure that your own functions don't pass around incorrect values, you should use a static type checker like mypy or pyright.

[tranquil turtle]

there are runtime typecheckers too, like typeguard or beartype

iirc, beartype has cool feature that allows you to automatically wrap all functions in your package in @beartype decorator, ao you dont have to change your code

[reef yacht]

You shouldn't crosspost btw

[gilded mango]

Hello. Can I ask questions regarding re here?

[trim tangle]

That doesn't seem related to type hints, so maybe make a help thread in #1035199133436354600 ( #❓｜how-to-get-help )

[gilded mango]

Okay, thank you.

[icy falcon]

im a bit confused of the syntax here: what does ```py
Type[SomeType]

correct me if im wrong but does it mean anything that inherits SomeType?

[bleak imp]

It depends on the type. With something like list[type] or Sequence[type], it's the type of the contents. You also have dict[key_type, value_type] or Callable[[arg_1_type, arg_2_type, etc], return_type]. You can implement it on any type using __getitem__, since it's the same syntax as normal getting.

[oblique urchin]

For type specifically, it means a subclass of SomeType as you suspected

(You can use type[SomeType] in 3.9+, typing.Type[SomeType] if you need to support earlier versions)

so if a function parameter is annotated as type[SomeType], you can call it like f(SomeType), or with a subclass of SomeType

[icy falcon]

okay thank you i think i understand now.

[rose mist]

Greetings guys,

Hope you are well. I wanted to ask what is the best way to annotate iterables in a generic manner?

Like how would you annotate this code :

def define_distance_matrix(self) -> Iterable[Iterable[float]]:
        """ Function to define the distance matrix for the given coordinates.

        Returns
        -------
        `Distance_matrix` : Iterable[Iterable[float]]
            The distance matrix.
        """
        # Distance Matrix
        Distance_matrix = []
        temp_set = []

        # Function for calculating the euclidean distance
        def distance_between_points(point_A, point_B):
            return np.sqrt((point_A[0] - point_B[0]) ** 2 + (point_A[1] - point_B[1]) ** 2)

        # Generating the distance matrix
        for i in range(len(self.coordinates)):
            for j in range(len(self.coordinates)):
                temp_set.append(distance_between_points(self.coordinates[i], self.coordinates[j]))
            Distance_matrix.append(temp_set)
            temp_set = []

        return Distance_matrix

[reef yacht]

The annotation supplies less information about types than is always there. List[List[float]] is actually the real return type, and will always be the real return type. Why isn't it your annotation?

[rose mist]

I actually can't say, hence why I asked hehe.

I would like to know what is the best way of doing it.

I would also want to know how it would be appplied to N dmensional lists.

[reef yacht]

There are many other issues with this code btw. The for loops are wrong. The name "temp_set" is weird. "Distance_matrix" should be "distance_matrix".

Using numpy with sqrt seems.. weird?

[rose mist]

Ohh yeah I'll get to those, but this is some old code I just copied to ask how to use the iterables generic.

[reef yacht]

for a in self.coordinates:
    distance_matrix.append([distance_between_points(a, b) for b in self.coordinates])

That's what those two for loops should be, and then you can delete the temp_set variable

[reef yacht]

I mean.. I guess that usage would be fine if generic iterables was the returned type. But in this case it should be list of list of float imo as that's the truth.

[rose mist]

Exactly, I would like to know how to represent these generally, and then to use that instead of List.

[reef yacht]

but.. that makes no sense in this context

If the type is list and will ALWAYS be list, then type it as list!

[rose mist]

I know, I am saying how to represent a list generally.

[reef yacht]

The code you showed distracted a ton from that question as that code should absolutely not use Iterable[]

[rose mist]

Well I apologize if the code is distracting, are you now clear on what my question is?

[reef yacht]

Nope. Because you showed code that clearly used it incorrectly, I am left wondering if you understand at all why you'd use Iterable :P

[rose mist]

I do not, hence why I asked it here.

[reef yacht]

Ok, so think of a function that takes an iterable and returns the first item:

def foo(x):
    for y in x:
      return y

[rose mist]

You mean y?

[reef yacht]

sorry

[rose mist]

Issokay

I get the idea.

Please continue.

[reef yacht]

thanks, fixed

Yea, so in this case you can send that function a list, or a dict, or a dict.keys() or a dict.values() right?

Or any custom class that implements the iterable protocol for that matter.

Here you'd want def foo(x: Iterable)

[rose mist]

Ok, so Iterable is the most generic way of doing this?

[reef yacht]

Yea, it's basically saying "an object with the __iter__ method"

[rose mist]

I was told it's not, as it for instance apparently doesn't support __len__.

And to use MutableSequence instead.

[reef yacht]

Yea, iterable doesn't support len, that's correct

[rose mist]

So what is the best way of doing this?

[reef yacht]

But foo doesn't use len so that's fine

[reef yacht]

You keep saying "doing this" but you haven't defined "this"

You're pointing to nothing and asking me how to do it :P

[rose mist]

Annotating and working with a generic iterable.

[reef yacht]

mutablesequence is much more specific. MUCH more specific.

[rose mist]

How so?

[reef yacht]

But it's ENORMOUSLY context dependend. Like crazy much.

[reef yacht]

"mutable" for one

[rose mist]

I don't really follow.

[reef yacht]

(1, 3, 5) is a tuple. It's iterable and supports len, but it's NOT mutable

If you use MutableSequence for that, your code is broken

[rose mist]

So you would suggest Iterables instead generally speaking?

[reef yacht]

no, not at all

Look. If you just say everything it iterable, now you can't do len on anything. Chaos.

[rose mist]

I see.

[reef yacht]

You have to use whatever is appropriate. You can't just say "always use Iterable", that's nonsense.

How do you know what is appropriate? Well then you have to look at each specific situation!

[rose mist]

Ok, so if I would like to use sth like List but in a way that is conventional and standard, and would allow for __len__, different data types, and basically be usable like List, what should I choose?

I would like the scenario where it is Mutable.

[reef yacht]

MutableSequence seems fine. But also List!

[rose mist]

So it's one or the other?

[reef yacht]

In this specific case, List 100%

[rose mist]

Ok, so I should use List for this case?

[reef yacht]

Yes. It's the real type. You know it's a list always, so put that type there.

[rose mist]

I see. A follow-up question is how would one represent an N dimensional list?

[reef yacht]

First let's be specific: there's no such thing in python. There are only lists. Those lists can contain other lists.

But in the general case if N is not known when you write the code, you can't I'm pretty sure.

[rose mist]

N is known.

[reef yacht]

Someone can correct me if I'm wrong, but the type system in Python isn't that advanced.

[rose mist]

N is known, and dtype is known.

[reef yacht]

well then it's List[List[List[x]]] where N = 3

[rose mist]

Well that's not very scalable, is it?

[reef yacht]

I mean.. code like that isn't very practical in any case so 🤷‍♂️

If you have more than List[List[]] you have bigger problems

[rose mist]

I see. Anyways, thank you so much, my apologies for the bad questions.

I wish you a blessed day ahead!

[reef yacht]

That's why we are here :P

[gilded anchor]

hello. in the current state of typing support, is there a way to generate a class with types dynamically, in a way typecheckers would understand and possibly raise issues? I tried make_dataclass function, but mypy doesn't care when I assign str to int field, so that doesn't meet the requirements...

[reef yacht]

That doesn't really make sense logically. Static typing is static, ie on the code level without the code being executed. You seem to be talking about types that aren't known until runtime.

[gilded anchor]

Well, what I have in mind is to read JSON/TOML file and be able to return class-like structure with proper typing based on what is found in file. It could be in theory done with user specifying the schema for file (like pydantic does) but I wanted to avoid that part

[reef yacht]

If you want static guarantees, I guess you'll have to do code generation. From some JSON/TOMPL spec to .py files with pydantic declarations

[fierce ridge]

this sounds kind of circular to me. you want to infer a schema from untyped data, and then use that inferred schema to typecheck the data that you inferred it from?

there's a legitimate use case here, for things like jsonschema. but my concern is that this is not that

[reef yacht]

I guess you could have a system like this where you let any deviation from the inferred schema crash and look at the data in sentry, fix it and then deploy the updates. Sort of like an iterative approach to approaching an otherwise undocumented web API. But it seems like a last resort :P

[gilded anchor]

TOML has built-in types that can be translated to Python types, so I am not sure what you mean by "untyped data"

[paper salmon]

are you intending to only do this once and then hardcode the resulting classes into your program? or do you really mean reflecting the file's current structure into your program at runtime? because the latter is quite hard for static type checkers to analyze

[reef yacht]

How would you type

[foo]
a=3

? Ie the key a must be integer?

[gilded anchor]

the latter. I wanted to make a well-typed TOML library

[reef yacht]

Yea, then it's what I said above. You are confused about what python type checking IS.

[gilded anchor]

class foo:
    a: int 

[gilded anchor]

well, I know what Python type checking is, I am just not up to date with all bells and whistles, was hoping that something like that is now possible

[reef yacht]

Well no. You keep saying AT RUNTIME. Which is logically incompatible with static typing. It's like saying you want a dry glass of water.

It's not a matter of finding a fancy glass, it's logically impossible.

[tranquil ledge]

To my knowledge, there are a only few pieces of the standard library that dynamically create classes that popular type-checkers (e.g. mypy, pyright) understand, specifically because the type-checkers have special cases for them. collections.namedtuple and dataclass-like implementations made with typing.dataclass_transform come to mind. Anything more custom than that isn't supported.

You'd have to use a separate tool to verify those.

[wet sequoia]

I think code generation is really the only solid way to do this sort of thing. It's not necessarily a bad idea - having a library that takes a toml (or any other config format) as input and spits out the inferred dataclass could be handy in some situations. If you don't want to check in the code you could run the generation as part of setting up type checking + IDE support.

The closest thing I know of to dynamic magic here is F# type providers, which can do things like infer a type off of a SQL table schema, but under the hood I think the way that works is really code generation inside the compiler, and it's unlikely to happen in Python partly because there are so many type checkers, as opposed to the one F# compiler.

In theory any one type checker could add something like this but I don't think it would be easy to get into the spec.

[delicate siren]

hello

i need help in typehinting inheritance of tuple, i wonder how i should do it, heard of tupletypevar

basically my appraoch currently is class a(tuple[T_co, ...])

i wonder if it's fine

please ping me

[tranquil ledge]

What's your use case? What do you want to do with this class?

[delicate siren]

make a subclass of tuple

[tranquil ledge]

Then yeah, if the inside of your tuple subclass is always going to be homogenous in terms of type, inheriting like that should be fine.

[delicate siren]

i've heard there's tupletypevar

not sure how to use it of if i should use it

[oblique urchin]

It's TypeVarTuple. It might be useful, but it's a more advanced feature

It depends on how your subclass of tuple will be used

If it's only for homogeneous tuples (all values are of the same type), your existing approach is fine

[delicate siren]

uh i want it to support various types

i mean i would like the most to be able to do the same syntax as tuple

eg a[T_co, ...] or a[int, float]

[oblique urchin]

then I think you want to make it generic over a TypeVarTuple

[delicate siren]

how would u do that for example

[oblique urchin]

https://typing.readthedocs.io/en/latest/spec/generics.html#using-type-variable-tuples-in-generic-classes

[delicate siren]

def get_shape(self) -> tuple[*Shape]:

but the signature of the function looks like this

    def __getitem__(self, __key: slice) -> tuple[_T_co, ...]: ...

thats tuple signature

tuple[_T_co, ...]

[oblique urchin]

tuple itself is pretty heavily special-cased in the type system

[delicate siren]

🤔

[oblique urchin]

Possibly we could write it using TypeVarTuple in typeshed, but the stubs there predate TypeVarTuple and we haven't had a reason to change them

[delicate siren]

i see, well thanks

[undone saffron]

There's existing stuff here that will (de)serialize to toml to an expected structure https://jcristharif.com/msgspec/api.html#toml

[fierce ridge]

untyped in the sense that you don't know know what's a list, string, etc without actually looking at the data. i should maybe say "dynamically typed", like python itself

[fierce ridge]

I think what you haven't clarified is your intended use case. you're asking about tooling, but we're not sure what you're actually trying to do, so it's hard to recommend anything in particular

you can definitely generate that class from an example toml file

you keep implying that isn't what you want, and neither is manually writing out a schema of some kind. but you aren't really clarifying what you do want

[hallow flint]

you might be able to make a mypy plugin, but that will then only work with mypy

[fierce ridge]

i have no idea what is or isn't possible with mypy plugins. it would be very useful if you could provide a jsonschema and have mypy infer a TypedDict from it

[rustic gull]

ee

[cunning plover]

i started to try using stubgen to generate mypy compatible types for library
I discovered that dataclass init typed aren't generated :/
and NewType values are translated to Incomplete

I worked around that by using regular Class with defined types in __init__
And using type alias (SomeType = str), instead of NewType

Do u have perhaps some alternative paths to solution?
Something that is more automatically generating generating types for mypy
(End goal for distributed library to be recognized as typed, instead of having Any stuff)

[cunning plover]

2.4k issues... i guess mypy is having all the different surprises as it is

[trim tangle]

I think they're just not as trigger-happy as pyright to close stuff as "as designed" or "wontfix" 🙂

[reef yacht]

Hah. I had exactly this problem with my hack. I solve dit by generating __ini__. https://github.com/boxed/ivrit

[cunning plover]

Terrific 😅 Okay, writing hacks as a solution then.

[reef yacht]

Feel free to copy paste from my thing.

[cunning plover]

i think i found plausable very minimalistic solution to my problem.
I am mostly not satisfied only that NewType in stubgen is replaced to InComplete. (I can disregard inability to use dataclasses)
i can literally replace generated stubs, MyType = Incomplete, with original code back where MyType = NewType("MyType", str)
And it works fine as stubs

very small script to make, to hack NewTypes back to working. looks like really plausable solution with minimal hackiness

that will help me keep my lib code type safe, since i continue using NewType... 🤔 ....
Although i did not test how much this issue propagates in other code parts.
needs to be verified behaviour with classes, function arguments and etc

Okay verified

NewType alias nicely propagates in stubgen generated code

i literally need to replace in generated code only NewType Declaration/Creation moment and that's it. Very simple to do

[trim tangle]

Could someone help me cirlce in on a smaller reproducible example? I have this code:

from typing import Generic, TypeVar, Protocol

class Foo(Protocol):
    def bar(self) -> int:
        ...

class MyFoo:
    def bar(self) -> int:
        return 42

T = TypeVar("T")


class AppKey(Generic[T]):
    def __init__(self, name: str, tp: type[T]) -> None:
        pass

FOO_KEY = AppKey("foo", Foo)

class Container:
    def __getitem__(self, key: AppKey[T]) -> T:
        ...

container = Container()

foo = container[FOO_KEY]
foo.bar()

And pyright is giving me an inexplicable error on the last line: ```
Method "bar" cannot be called because it is abstract and unimplemented

or would this be "small enough"?

[viscid spire]

What is the point of MyFoo in your code?

[trim tangle]

err yeah it's not needed

already posted this https://github.com/microsoft/pyright/issues/7151

[viscid spire]

What happens if you pass MyFoo instead of Foo?

[trim tangle]

?

[viscid spire]

In Appkey there

[trim tangle]

Then of course it does work, because MyFoo is not a protocol

[viscid spire]

I c

What if you make MyFoo inherit from Foo?

[brazen jolt]

from typing import Protocol, TypeVar


class Foo(Protocol):
    def bar(self) -> int: ...


T = TypeVar("T")


def test(x: type[T]) -> T: ...


test(Foo).bar()

slightly smaller example

with the same issue

it is a weird one though

this one works fine btw:

from typing import Protocol, TypeVar, cast


class Foo(Protocol):
    def bar(self) -> int: ...


class MyFoo:
    def bar(self) -> int:
        return 2


T = TypeVar("T")


def test(x: T) -> T: ...


x = test(cast(Foo, MyFoo()))
x.bar()

so it's something to do with that type[T]

[trim tangle]

||should we place bets if this is an as-designed||

Oh wait, Eric replied

this is actually a duplicate 👀 https://github.com/microsoft/pyright/issues/7105

i should've looked

[rough sluice]

:incoming_envelope: :ok_hand: applied timeout to @dusk zinc until <t:1706503885:f> (10 minutes) (reason: burst spam - sent 8 messages).

The <@&831776746206265384> have been alerted for review.

[cunning plover]

Wait. So if i wish types of libs recognized directly... all i need supplying py.typed? 😅

Why i can't use thie method always ^_^ And how to force libraries not having stubs, to be py.typed turned on for mypy?

i wonder if i need py.typed at every module level, or package root is enough 🤔

is it possible turning default mypy behavior, having fallback to py.typed if stubs aren't found? 😅

we could check, but forcing the file to library folders 🤔

[tranquil turtle]

root is enough

[cunning plover]

good

The installed packages marked as safe for type checking (see PEP 561 support)
aha. Some people code not safe for import code i guess. that's why not recommended as default. Hmm is there any easy way to force installed libraries being recognized as safe 🤔

[cunning plover]

That makes very easy writing libs with typing then 😊 stubgen is pain in the ass barely working. thanks god i don't have to use it

[hollow prism]

"Found Item with id 1" got printed, because in checks for equality for every element, but pylance is giving me a reportUnnecessaryContains warning. is there something i did wrong?

class Item:
    def __init__(self, id: int, value: str) -> None:
        self._id: int = id
        self._value: str = value

    def __eq__(self, other: object) -> bool:
        if isinstance(other, Item):
            return self._id == other._id
        elif isinstance(other, (int, float, Decimal)):
            return self._id == other
        return False


item1: Item = Item(1, 'foo')
item2: Item = Item(2, 'bar')
item3: Item = Item(3, 'baz')

if 1 in [item1, item2, item3]:
    print('Found Item with id 1')

[dull lance]

Is there a way to type hint assert-like functions (such as hypothesis.assume) so that the type checker can perform type narrowing based on the provided condition for subsequent code? I've tried

from typing import Literal, NoReturn, overload
from hypothesis import assume

@overload
def assume_cond(cond: Literal[True]) -> Literal[True]: ...
@overload
def assume_cond(cond: Literal[False]) -> NoReturn: ...

def assume_cond(cond: bool):
    return assume(cond)

def test_func(x: int | None):
    assume_cond(x is not None)
    return x + 1  # The type checker should understand that x must be an integer

but it doesn't seem to work (I'm using pyright).

I can work around this by adding an assert statement with the exact same condition after calling assume, like:

def test_func(x: int | None):
    assume(x is not None)
    assert x is not None

    return x + 1  # The type checker should understand that x must be an integer

Still, it would be nice if I could avoid doing so.

[trim tangle]

There's TypeGuard, but it only works in an if

TypeScript has the feature you want btw: https://github.com/microsoft/TypeScript/pull/32695

[stray summit]

anyone aware of there are any predefined protocols for Data/NonData descriptors that can be defined as instance variable?

context is that i need to have a descriptor as instance variable and defer to it - the descriptor may be a property or non data descriptors

[tranquil turtle]

no, descriptor protocol is only executed if there is no instance-level thing and class-level thing is a descriptor

btw: https://github.com/python/mypy/issues/13608

[stray summit]

im aware of that - i have a descriptor that i want to defer to another descriptor thats passed in the ctor
i want to type annotate the instance var holding it

[tranquil ledge]

iirc, there are general protocols for descriptors in typeshed (or maybe _typeshed), but you could get away with making your own simple protocols if you're willing to import typing.

Not quite sure how *those would interact with your instance variable shenanigans though.

[stray summit]

my general observation is that mypy pretends descriptors are applied even when they are instance vars - which is consistent with the bug above

[tranquil turtle]

iirc, pyright does it correctly

[chrome hinge]

Any, otherwise it won't be valid to do val: int = func(param).

Because an object is not an int, but Any is special and can be narrowed to any type.

[soft matrix]

pylint isnt a type checker, you should disable explict any on your type checker i think?

[chrome hinge]

!docs typing.Any

[rough sluice]

typing.Any

typing.Any```
Special type indicating an unconstrained type.

• Every type is compatible with [`Any`](https://docs.python.org/3/library/typing.html#typing.Any).

• [`Any`](https://docs.python.org/3/library/typing.html#typing.Any) is compatible with every type.

Changed in version 3.11: [`Any`](https://docs.python.org/3/library/typing.html#typing.Any) can now be used as a base class. This can be useful for avoiding type checker errors with classes that can duck type anywhere or are highly dynamic.

[chrome hinge]

Any is compatible with every type.
^ because of this detail. that's what makes it different from object.

[soft matrix]

oh it doesnt have an option to do so

not sure what you should do then

[chrome hinge]

i don't know, what's read_value?

Pylance is correct, then - Any is compatible with any type, so it can be, say, compared with an integer.

(a typehint of Any is kind of like opting out of all type checking)

what does QueryValueEx return? it seems to me the type should be something like str | None or maybe str | int | None

In that case you could do a typing.cast to force the type.

Like, basicdisplay_start = typing.cast(int, ...)

[soft matrix]

theres currently no way to do this in the type checker like this unfortunately because the Unknown type isnt exported

[chrome hinge]

I'd probably use the "true" typehint for the return type (something like None | str | int | list[int] | list[str], might need some research) and use typing.cast (or assert isinstance for that matter) when I know what the type would be and need to ensure it.

[soft matrix]

pyupgrade can do it for you but im not sure pyright has deprecated union yet

[chrome hinge]

i don't think it'll ever be deprecated since it's useful for old version support

pyupgrade has such a rule as gobot said, and looks like ruff has inherited it: https://docs.astral.sh/ruff/rules/non-pep604-annotation/

[soft matrix]

i forgot ruff was so goated

[grave fjord]

pytest is switching to ruff

[soft matrix]

well pyright currently warns about Dict and Tuple etc

[tranquil ledge]

True, though if the spec gets adjusted so that Union[*Ts] is valid at type-checking time, that particular construct might end up sticking around for some things.

[soft matrix]

Well union probably should anyway but you can still only recommend it in that situation

[tranquil ledge]

Agreed. Nothing else comes to mind atm that can’t be handled by the pipe operator now.

[cunning plover]

What is this function doing, describe us (provide pseudocode or its code), perhaps we can rewrite it into type safe way

what 🤔

reading values from windows registry?

could u show how it does it under the hood, i think we can stilll fix it

ah yes, i see

here you go

from typing import Any, Optional

class Config:

    @classmethod
    def read_int(cls, path: str, value_name: str) -> Optional[int]:
        return cls._read_value(path, value_name)

    @classmethod
    def read_str(cls, path: str, value_name: str) -> Optional[str]:
        return cls._read_value(path, value_name)

    @staticmethod
    def _read_value(path: str, value_name: str) -> Optional[Any]:
        """Read keys from Windows registry"""

    try:
        with winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, path, 0, winreg.KEY_READ | winreg.KEY_WOW64_64KEY) as key:
            return winreg.QueryValueEx(key, value_name)[0]
    except FileNotFoundError:
        return None
    
config = Config()
your_value = config.read_int("SYSTEM\\CurrentControlSet\\Services\\BasicDisplay", "Start")

with little focus making Any returning function as private and exposing only more typed public functions, we turned this code from isolated not typed one into typed one

it will help code readability, since now u are able to guess correctly that your_value is integer without any extra help

during code usage with other values, your code will be able to validate that it is returned integer and how it interacts with other values (without u needing to cast it to another type, and hoping u did not forget to do it)

in this way u achieve less chances to shoot into your own legs by accident (without and with mypy/pyright, although with mypy/pyright u will achieve better safety of this code, but even without it, u will achieve extra bonuses to its autodocumenting capabilities more protected from mistakes for a future you)

[azure mural]

It's okay, I don't have any legs left, so it'll just hit empty space

[cunning plover]

Same as: Value | None. (Value or None)
Habit of typing before 3.10 python

This version is compatible for higher range of python versions
Kind of preferable if u develop library
Since more python versions become supported

[viscid spire]

Or use Union[T, None] 💀

[tranquil turtle]

i would argue that Any | T is not equivalent to Any
Any is weird in python because it is at the same time the bottom type and the top type

object | T is equivalent to object because T is included in object
Never | T is equivalent to T because Never has no values

if variable is declared as x: Any | T we should not simplify it neither to Any nor to T because none of these conversions are correct:

Any | T -> Any is wrong because:

• by definition, foo(T | G) is valid iff foo(T) is valid and foo(G) is valid
• so, Any | T is valid iff foo(Any) is valid and foo(T) is valid
• we agreed that foo(Any) is always valid (we treat Any as bottom type there)
• so, Any | T is valid iff foo(T) is valid
• so, in the context of foo(Any | T), Any | T is equivalent to T

Any | T -> T is also wrong because:

• by definition, T = G is valid iff G if a subtype of T
• and T is a subtype of A | B iff T is a subtype of A **or ** a subtype of B
• we agreed that T = Any is always valid (we treat Any as top type in this case)
• my brain melts right now, everything after that is nonsense
• we agreed that Any = T is always valid (we treat Any as bottom type in this case)
• so, by definition, Any is a subtype of T => Any is a top type
• Any | T = G is valid iff G is a subtype of Any | T, this is true iff G is a subtype of Any (which is true) or a subtype of T, so it is always valid
• i wish there was a symbol for "is a subtype of" :)
• but if we simplify Any | T to just T, Any | T = G might become incorrect

well, from the fact that Any is a top type and a bottom type at the same time, it follows that the set of all possible python objects is empty

[undone saffron]

That's not correct, you may want to look at the stuff in the intersections discussion, as the behavior of Union was well covered in it for exploring parallels

[tranquil turtle]

what specifically is incorrect?

[undone saffron]

Quite a bit. The problem here is that Any | T means different things in different contexts, and you haven't really gotten to that there.

As an annotation, and looking to assignment, everything is compatible, but as a known type of a value and using the value, it has a known safe interface of T

it's more related to variance and assignment vs use

[tranquil turtle]

Any itself means different things in different contexts, and it causes confusion

# i expect that x might be int
x: Any | int = foo()
x.method() # this is incorrect on ints, but typechecker thinks this is fine
``` when i explicitly write `Any | T` i do not expect it to become useless `Any`, i expect `T`-related checks to happen

```py
def foo() -> Any | int: ... # this is also not equivalent to ->Any
x: Any = foo() # ok
y: int = foo() # not ideal, but fine
z: str = foo() # err, int is not assignable to str

there is literally no case where Any|T can be replaced with Any without making some typesafe code not typesafe

[undone saffron]

there is literally no case where Any|T can be replaced with Any without making some typesafe code not typesafe

That's correct.

Unions have a wider set of things that can be assigned to them, and a narrower set of things that can be assumed about a value that has been assigned, if that helps

I'll grab some links in an hour or so when I have a moment if nobody else does before then, but you're right that as an annotation it can't be simplified, so you might understand it fine and just not have expressed the critical detail there.

[tranquil turtle]

yeah, probably, i have hard time describing nontrivial thoughts in english
my point is that typecheckers should not simplify that, but they do, and it is not good

[undone saffron]

they can safely in some contexts

so, when evaluating what attributes can be safely accessed to something that has already been assigned, prior to discrimination of Any | T, attributes of T are safe

[tranquil turtle]

if you dont care about function body, Any|T in parameter type can be replaced with Any
this will not affect outer world in any way, but might introduce problems in the function body
but func body doesn't always exist - sometimes there is only a stub and parameter type annotation is used only to interact with the outer world, not to typecheck function body

this is the only case i can come up with

[undone saffron]

In an annotation, you can't replace it safely. Part of the this isn't helped by the fact that we don't have accepted terminology between the meanings where types are interacted with, if you're only concerned about simplifying the annotations, you can't reduce Any out of a Union.

@tranquil turtle
This comment from Eric (pyright maintainer) goes over it not being reducible

https://github.com/CarliJoy/intersection_examples/issues/22 Has some pure math that could be used to improve the definition of Union.

and some tangential discussion of where the type system isn't fully specified, but quoting the most relevent part:

I think the confusion can happen because static types aren’t actually properties of values, but of terms in a program. It’s true that types constrain what values the runtime should produce, but the two concepts are not 1:1 and generic values usually have ambiguous static types.

It isn't unique to Any that types have different menaings in different contexts.

[bleak imp]

Curse you for linking extremely interesting issues, I always get sucked into reading GitHub issues.

[abstract plaza]

Hi folks! So I grow increasingly fond with type checking in Python. Today I stumbled upon an interesting problem and I'm curious whether you have any ideas/suggestions to solve it. I have some function def foo() -> Tuple[str | OtherType]: .... Later I want to print this str. Annoyingly I found that instead of printing the first member of the result, I always printed the whole tuple. Is there some way to resolve this with type checking? An imaginary solution would be to have some return type (or protocol?) that signals that it cannot be converted to string and the type checker then recognizing that it therefore cannot be printed. To my dismay I did not find anything that matches this description ...

[tranquil turtle]

no, everything is convertible to string, because object.__str__ exists
you can try adding this to your type: def __str__(self) -> Never: ... but it is not guaranteed to work and it is impossible to attach such method to tuple[str | OtherType]

i would suggest changing function signature to -> str | OtherType if possible and if that makes sense

[abstract plaza]

In this case not really, but thank you! I was afraid that there will be no easy way 🙂

[stone harness]

Does anyone know how pydantic manages to accomplish these type hints?

example:

from pydantic import BaseModel


class User(BaseModel):
    id: int
    name: str

it takes the class notes and turns them into kwargs in __init__

So far I haven't found anything in the documentation that talks about this.

[oblique urchin]

There is a couple of ways to do this sort of thing at runtime. Haven't looked into exactly how pydantic does it, but the most likely way is with an __init_subclass__ method that looks at __annotations__. Here's a very basic version I just wrote ```python
class BaseModel:
def init_subclass(cls) -> None:
args = list(cls.annotations)
lines = [f"def init(self, {', '.join(args)}):"]
for arg in args:
lines.append(f" self.{arg} = {arg}")
ns = {}
exec("\n".join(lines), ns)
cls.init = ns["init"]
cls.init.annotations = {**cls.annotations, "return": None}

class Point(BaseModel):
x: int
y: int

print(Point.init.annotations)
p = Point(1, 2)
assert p.x == 1```

[rare scarab]

It also uses @dataclass_transform() to get your type checker to like it

[stone harness]

thanks!!!

[scenic citrus]

There is some code in the click args library that I have not seen before. in the click/core.py you have.

    @t.overload
    def command(self, __func: t.Callable[..., t.Any]) -> Command:
        ... 

can anyone explain what the ellipsis in the type hinting and in the function definition are doing here?

[viscid spire]

it is special for Callable in its meaning here, and means "any amount of args, with any type"

in the function definition, it's just a placeholder

they are written in typestubs (.pyi files often) instead of an implementation

however, here you don't have a typestub, but instead an overload

which is another reason why a function wouldn't have an implementation (because it gets one on the last definition)

[scenic citrus]

is overload used in the same way its done in c? depending on the arg list will define witch version of the function is called?

[soft matrix]

overloads dont exist at runtime

its purely for typing

[scenic citrus]

i see very perplexing code that i will have to dig into more to understand. Thank you for the explanation. I did ask for an example from chat gbt and it gave this for overload pattern which is very interesting

from functools import singledispatchmethod

class MyClass:
    @singledispatchmethod
    def func(self, arg):
        print("Default implementation: ", arg)

    @func.register
    def _(self, arg: int):
        print("Implementation for int: ", arg)

    @func.register
    def _(self, arg: str):
        print("Implementation for str: ", arg)

obj = MyClass()
obj.func(10)    # Implementation for int: 10
obj.func('abc') # Implementation for str: abc
obj.func(1.23)  # Default implementation: 1.23

But i agree @soft matrix that this is not true overloading as the function is a wrapper function.

[tranquil turtle]

what is the point of @no_type_check if it doesn't disable type-checker completely?
if annotations are not type hints, it means that there is something weird going on, and it doesnt really make sense to complain about anything

inconsistent af

[oblique urchin]

The point of @no_type_check is questionable. See the recent discussions about it on discuss.python.org

In particular, no type checker has implemented support for it on classes

[tranquil turtle]

imo, it should mean @shut_up_completely_in_the_following_block_of_code

and probably @dont_even_look_inside

is there a # type: ignore, but for multiple lines?

[oblique urchin]

https://github.com/python/typing/pull/1615/files is what we're about to change the spec to, for what it's worth. Your interpretation is reasonable but nobody has cared enough to implement this in mypy for like a decade

[tranquil turtle]

TIL: you can place # type: ignore at the top of the file to ignore all errors in it

[tranquil turtle]

thank you pyright

[real ore]

Hello. I have made two base classes: one represents an item, the other one can manage a list of those items. Then I tried to make special subclasses from them with additional methods. In the following code Pyright says Cannot access member "render" for type "BaseItem". I see problem, special_list could also contain BaseItem which does not have render method defined, but I do not know how to make and type structures like that.

from __future__ import annotations

class BaseItem:
    def __init__(self, value: str) -> None:
        self._value = value

class BaseList:
    def __init__(self, items: list[BaseItem]) -> None:
        self._items = items
        self._index = 0

    def __iter__(self) -> BaseList:
        return self

    def __next__(self) -> BaseItem:
        while self._index < len(self._items):
            self._index += 1
            return self._items[self._index - 1]
        raise StopIteration

class SpecialItem(BaseItem):
    def render(self) -> None:
        print(self._value)

class SpecialList(BaseList):
    pass

special_item_1 = SpecialItem("I am special")
special_item_2 = SpecialItem("I am also special")
special_list = SpecialList([special_item_1, special_item_2])

for item in special_list:
    """
    Pyright says:
        Cannot access member "render" for type "BaseItem"
        Member "render" is unknown
    """
    item.render()

[undone saffron]

what you've demonstrated wanting to do there isn't type-safe, and it isn't clear what it is trying to accomplish where specifying the actual type, not some base class is meant to accomplish

[tranquil ledge]

Could make BaseList generic over the type of object it'll contain if you want to do this from scratch, e.g.

from typing import Generic, TypeVar

# Note: Default in TypeVar is only supported by pyright currently, I think.
T = TypeVar("T", bound=BaseItem, default=BaseItem)  

...

class BaseList(Generic[T]):
    def __init__(self, items: list[T]):
        ...

class SpecialList(BaseList[SpecialItem]):
    def render(self):
        ...

...

for item in special_list:
    item.render() # Should be fine now.

[real ore]

Thanks, I will try it out, as I see default argument requires Python 3.13.

[real ore]

It is working great with generics and TypeVar, thanks for your help.

[tranquil ledge]

Yeah, that bit isn’t really necessary for this to work in theory. Threw it in just in case, lol.

[real ore]

Hi. What is the right way to set typing for a custom list-like class inherited from a collections.UserList, using type parameters? In the following example Pyright says that the second and the third parameters have type mismatch. What am I doing wrong?

from collections import UserList
from typing import SupportsIndex

class BaseList[ListItem](UserList):
    # Pyright says the second and the third parameters have type mismatch
    def __setitem__(self, i: SupportsIndex, item: ListItem) -> None:
        return super().__setitem__(i, item)

[viscid spire]

why not just subclass list directly?

[viscid spire]

well I can say that you are missing the type arg on UserList

[real ore]

I was think thinking which one is better, the built-in list or UserList, I want try out both, do you suggest list? What do you mean by missing the type arg on UserList?

[viscid spire]

UserList[ListItem] in the class header

but maybe you don't need it, idk. I just do what pyright tells me to do, and it told me to do that

[scenic citrus]

i dont understand this syntax class BaseList[ListItem](UserList): Can you explain how that works ?

Oh this seems to be from the python 3.12

[untold anvil]

What would be a proper way to represent something is a TypeDict? Currently ```py
LD = TypeVar('LD', bound=TypedDict)

class MetricsCounter(Counter, Generic[LD]):

def get_labels(self, labels_dict: LD):
    return self.labels(**labels_dict)

``` this works but mypy complains that TypedDict isn't a valid type. Following their help link had me try Type[TypedDict] and TypeAlias, the ladder worked but its not truly representing a TypedDict anymore.

[untold anvil]

I guess it would be just a Mapping? Seems to work.

[real ore]

It is a new type parameter syntax introduced in Python 3.12. It provides a shorter way to write generic functions and classes. https://docs.python.org/3/reference/compound_stmts.html#type-parameter-lists

[real ore]

Thanks for tip. I have added ListItem type arg to UserList , but the error still exists. Do you have any advise what could be the problem? Typing works great in every other methods (__init__, append, pop, etc.) except __setitem__.

[viscid spire]

I mean, just read the message from pyright, that's all I would do

[tranquil ledge]

Based on that error, seems like you need to define overloads via typing.overload to account for inputs to setitem that are slices and iterables of ListItem, unless you’re actually intending to disallow slicing on your list.

[fierce ridge]

import logging
from collections.abc import Callable
from typing import Any, ParamSpec, TypeVar


_P = ParamSpec("_P")
_T = TypeVar("_T")


def trace_call(val: _T, func: Callable[_P, object], *args: _P.args, **kwargs: _P.kwargs) -> _T:
    func(*args, **kwargs)
    return val


def trace_print(val: _T, msg: Any) -> _T:
    return trace_call(val, print, msg)


def trace_log(val: _T, logger: str | logging.Logger, level: int, msg: str, *args: Any) -> _T:
    if isinstance(logger, str):
        logger = logging.getLogger(logger)
    return trace_call(val, logger.log, level, msg, *args)


trace = trace_print

does mypyc not support ParamSpecArgs? i get this error when compiling the above with mypyc:

AssertionError: unexpected type <class 'mypy.types.ParamSpecType'>

or am i using ParamSpecArgs incorrectly here?

mypy --strict reports no errors

[undone saffron]

mypyc doesnt support everything mypy does, i wouldnt be surprised if thats the case here, and the use appears correct

[fierce ridge]

good to know. hopefully that's on the roadmap. i'd rather have it work slowly than not at all

[delicate siren]

hwo can i import dict_items for typehints

i see that in builtins it does:

from _collections_abc import dict_items, dict_keys, dict_values

[oblique urchin]

you shouldn't, the class is private

I would recommend using a more general class instead, e.g. Iterable or Collection

[delicate siren]

._.?

[oblique urchin]

or collections.abc.ValuesView

[delicate siren]

returning items and typehinting iterable?

[oblique urchin]

iterating is usually what you do with .values()

oh sorry items not values

[delicate siren]

sure but it has its own type

shouldn't i use it

[oblique urchin]

but the same thing applies, just use ItemsView

[delicate siren]

oh

itemsview is the same as dict_items?

[oblique urchin]

it's the more general version for all mappings

[oblique urchin]

Not necessarily. Giving a more general type means you're less bound to implementation details and more easily able to evolve your API in the future

[delicate siren]

i type hint it normally? [KT, VT]?

ItemsView[KT, VT]

[oblique urchin]

yes, it has two type parameters

[delicate siren]

what abbout values and keys?

like dict_key and dict_values?

[oblique urchin]

KeysView and ValuesView

[delicate siren]

typehinting the same?

[oblique urchin]

they take only a single type parameter

[delicate siren]

i see, so KT and VT respectively

thanks

[trim tangle]

^yep

also Iterable[Thing] is easier to understand as a user

[delicate siren]

i'd say it depends on the context

in my context itemsview is clearer in my opinion

[trim tangle]

Can you share some code maybe?

[trim tangle]

I think there might be a bigger issue if you're returning an items view directly. It will get invalidated if the underlying dict is mutated

[delicate siren]

if i wouldn't the user might think i might return keys or values of the dict

[delicate siren]

well it'd just complicate stuff

[viscid spire]

the name of the function should make it apparent what it does

[delicate siren]

🤷‍♂️

[viscid spire]

share it anyways

[delicate siren]

nah no need to

[viscid spire]

I figured out what it is anyways

just inherit from dict and overwrite the mutating methods for your frozendict

[delicate siren]

it is related to it but sharing all the code would just complicate stuff

🤔

i didn't test anything yet btw

didn't even finish the prototype yet

[real ore]

Hi. I have never heard of @overload, I looked after that and that was to solution. Thanks for your help!

[tranquil ledge]

No problem.

[pulsar pilot]

Is it possible to create generic type identifier with generic bound, for example, something like T: Bound[U]?

I've had some trials before, but they didn't work.

T = TypeVar("T")
It = TypeVar("It", bound=Iterator[T])
                         ~~~~~~~~~~~
           TypeVar bound type cannot be generic (Pylance)

T = TypeVar("T")
def do_something[It: Iterator[T]](it: It) -> T:
                     ~~~~~~~~~~~
    TypeVar constraint type cannot be generic (Pylance)
    ...

def do_something[T, It: Iterator[T]](it: It) -> T:
                                ~~~
                "T" is not defined (Pylance)
    ...

[plush halo]

you just want an alias, I believe.

T = TypeVar("T")
It: TypeAlias = Iterator[T]
``` or ```py
def do_something[T](it: Iterator[T]) -> T:

actually.. the first one might not be valid.

[viscid spire]

do_something(
    iter([1, 2, 3]),
    iter({1, 2, 3}),
)

should be an error with what they are aiming to achieve (I'm gathering)

[pulsar pilot]

But I want to pass the type information inside my code, here's a larger example.

import typing as t
import abc

T = t.TypeVar("T")
B = t.TypeVar("B")


class MyType(t.Generic[T], metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def base(self) -> T:
        ...

    @staticmethod
    def obj(data: T) -> "MyObj[T]":
        return MyObj(data)

    def wrapper(self) -> "MyObjWrapper[T]":
        return MyObjWrapper(self)


Obj: t.TypeAlias = MyType[T]


class MyObj(MyType[T], t.Generic[T]):
    data: T

    def __init__(self, data) -> None:
        self.data = data

    def base(self) -> T:
        return self.data

    def str_base(self) -> str:
        return f"{self.data}"


class MyObjWrapper(MyType[T], t.Generic[T]):
    _obj: Obj

    def __init__(self, _obj: Obj):
        self._obj = _obj

    def base(self) -> T:
        return self._obj.base()

    def do_sth(self, func: t.Callable[[T], B]) -> B:
        base = self._obj.base()
        return func(base)

In MyObjWrapper, both PyCharm and Pylance cannot calculate the type of base, no matter what Obj is (alias or typevar). MyPy does calculate its type, but cannot offer further suggestions.

Actually I'm trying to rewrite some functional libs in python, and the actual typing system is much more complex 😦

[olive parrot]

Does anyone have an example of decorator accepting args with type hints ? I can't find anything meaningful on my side.

atm i have the following but i'm still getting an error for templated return

R = TypeVar("R")
P = ParamSpec("P")


def templated(template: str | None = None) -> Callable[..., Callable[P, R | str]]:
    """Automatically wrap a returned dict with the specified template
        If no template is provided, it default to the endpoint url
    Args:
        template (None | str, optional): _description_. Defaults to None.

    Returns:
        Any: _description_
    """

    def decorator(f: Callable[P, R]) -> Callable[P, R | str]:
        # We wrap the function for it to inherit its caller doc / args
        @wraps(f)
        def decorated_function(*args: P.args, **kwargs: P.kwargs) -> R | str:
            template_name = template
            if template_name is None:
                if request.endpoint is None:
                    raise UnknownEndpointError
                template_name = f"{request.endpoint.replace(".", "/")}.html"

            # Execute the wrapper view and get the return value,
            # default to a dict if nothing is returned
            ctx: R | dict[Any, Any] = f(*args, **kwargs) or {}

            # We return unchanged anything that is not a dict
            if not isinstance(ctx, dict):
                return ctx

            return render_template(template_name, **ctx)

        return decorated_function

    return decorator

[trim tangle]

can you show the error?

You probably want -> Callable[Callable[P, R], Callable[P, R | str]]:

[olive parrot]

(constant) R: type[R@templated]

Expression of type "(f: (**P@decorator) -> R@decorator) -> ((**P@decorator) -> (R@decorator | str))" cannot be assigned to return type "(...) -> ((**P@templated) -> (R@templated | str))"
  Type "(f: (**P@decorator) -> R@decorator) -> ((**P@decorator) -> (R@decorator | str))" cannot be assigned to type "(...) -> ((**P@templated) -> (R@templated | str))"
    Function return type "(**P@decorator) -> (R@decorator | str)" is incompatible with type "(**P@templated) -> (R@templated | str)"
      Type "(**P@decorator) -> (R@decorator | str)" cannot be assigned to type "(**P@templated) -> (R@templated | str)"
        Function return type "R@decorator | str" is incompatible with type "R@templated | str"PylancereportGeneralTypeIssues
(function) def decorator(f: (**P@decorator) -> R@decorator) -> ((**P@decorator) -> (R@decorator | str))

[olive parrot]

at which step ? if I replace -> Callable[..., Callable[P, R | str]]: by what you said i get another error : Expected parameter type list or "..."Pylance

[trim tangle]

Oh sorry, I meant -> Callable[[Callable[P, R]], Callable[P, R | str]]: ( since you have one argument)

in the top level function

[olive parrot]

can you explain why add the [] around the second Callable? why does it fix my issue ?

[trim tangle]

Callable needs the arguments to be in braces: Callable[[int, str], bool] is a function accepting an int and a str, returning a bool

[olive parrot]

too much braces on the same line, my brain is melting

ok, i think i get it, thx !

Another question on the same piece of code, why do i need this line template_name = template. When using template directly, I get an Unbound error.

[trim tangle]

!e
That's because when you assign to a variable without global or nonlocal, it is automatically treated as local.

some_global = 0

def error():
   some_global += 1
   print(some_global)

error()

[rough sluice]

@trim tangle :x: Your 3.12 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 7, in <module>
003 |     error()
004 |   File "/home/main.py", line 4, in error
005 |     some_global += 1
006 |     ^^^^^^^^^^^
007 | UnboundLocalError: cannot access local variable 'some_global' where it is not associated with a value

[olive parrot]

uhm, okay, so if I understand it right, since the value template is coming from outside the func decorator it's considered as global ?

[trim tangle]

Python cannot first read the some_global from global scope and then assign a some_global in the local scope. In the same scope, a single variable will either always be local, always be nonlocal, or always be global

[olive parrot]

ok i think i understand

[silver cave]

Hey folks, hoping this is the right channel for this - I've got a couple of Python question around converting to int to float that I'd love folks thoughts on.

Imagine you have an API that takes a float, and you are building a library for that API where you want strong type validation in code. Should you be able to pass an int and have it convert that to a float behind the scenes? I'd say yes as 3 is the same as 3.0.

Now the converse - if you have an API that takes an int, what should the library do if you pass a float? Should it error? Should it allow round number floats (like 3.0) only that it can convert to an int? What would be typical behavior?

Would love the thoughts from the community.

[soft matrix]

In typing, int is a subtype of float

[oblique urchin]

The typical behavior in the standard library is to accept int for float arguments, but not the reverse

e.g. ```In [12]: [1, 2][0.0:]

TypeError Traceback (most recent call last)
Cell In[12], line 1
----> 1 [1, 2][0.0:]

TypeError: slice indices must be integers or None or have an index method

[silver cave]

That was my thought - so you can pass an int to a float but not the otehr way round

[silver cave]

That's typical for the standard library? Then that feels like the perfect thing to do - consistency is key!

Thanks @soft matrix and @oblique urchin

[signal brook]

best python 3.11 compatible way to represent bytes from a file after using open()?

should i just use from io import BytesIO

[soft matrix]

no you should be using BinaryIO iirc

[oblique urchin]

prefer IO[bytes]

[signal brook]

so the issue is pylance?

[oblique urchin]

I think the issue is that you think open() returns bytes when it doesn't

[signal brook]

this is my code

async def read_file(self, file_path: str) -> BinaryIO:
    async with aiofiles.open(file_path, "rb") as file:
        content = await file.read()
        return content

[soft matrix]

reading the file will return bytes

not an IO class

[signal brook]

yeah but bytes is not a type itself right

[oblique urchin]

it is

[signal brook]

oh i dont think ive ever seen it used as a hint

now i got this

which is what brought me to BinaryIO

[oblique urchin]

well that sounds like a real bug in your code

or possibly your annotations

[signal brook]

its from a library not my code

https://github.com/interactions-py/interactions.py/blob/stable/interactions/models/discord/file.py#L55

[rough sluice]

interactions/models/discord/file.py line 55

UPLOADABLE_TYPE = Union[File, IOBase, BinaryIO, Path, str]```

[oblique urchin]

well that function clearly doesn't accept bytes

so if you're passing bytes to it, your code has a bug

[signal brook]

oh i wasnt passing bytes to it, my issue is im trying everything and its still getting incompatibilities

also i thought BinaryIO is meant to assist in type checking and also represent bytes

[oblique urchin]

BinaryIO represents a file object that returns bytes when read() is called. The function you showed earlier is correctly annotated as returning bytes. Why that gives you issues elsewhere in your program I couldn't say

[signal brook]

async def read_file(self, file_path: str) -> BytesIO:
    async with aiofiles.open(file_path, "rb") as file:
        content = await file.read()
        return BytesIO(content)

i think this is my solution

which is what ive been doing for 10 years tbh lol. i just wanted to know newer stuff

[fierce ridge]

maybe this isn't related to the discussion above, but i'm curious why you'd do this instead of just returning the original bytes

[signal brook]

pylance doesnt like it

with the lib im using or something

[fierce ridge]

can you show how this is used? i suspect that whatever is consuming the output of this read_file function is expecting a file-like object, not a bytes

oh, it's this? UPLOADABLE_TYPE = Union[File, IOBase, BinaryIO, Path, str]

yeah, it doesn't accept raw binary data. it wants to read it from somewhere

[crisp steppe]

Hi all! I have a type Optional[int] and I would like to use this to instantiate an int. But I'm not sure how to extract the underlying type for the _UnionGenericAlias here.

I found this but it seems a bit back-doorish.

>>> Optional[int].__args__[0]
<class 'int'>

[chrome hinge]

At runtime? Well, it's an alias for Union, and- yeah, that's what I was going to suggest.

[vast olive]

Note that if you use int | None or None | int the order in the __args__ also changes

[tranquil turtle]

there is no reliable way to instantiate a type if you have a reference to it

[crisp steppe]

then how can pydantic or dacite work

(im trying to reimplement dacite in rust with pyo3)

[tranquil turtle]

'int | None'
T | None where T is int

[vast olive]

It's an example 🗿

[crisp steppe]

I think if it's a union, I walk the args and try to instantiate based on the provided data until it works. so if someone does A | B | None then I try to instantiate A then B then None.

[vast olive]

Sounds alright

[crisp steppe]

woo, got option and Self working 😅

[cunning plover]

    async def __call__(self):
        await source.feed(target)

how would look typing to this? 😅

Async typing is very new to me and very confuses for now