#type-hinting

1 messages · Page 34 of 1

cinder bone
#

Otherwise it's a pain for both users and you to type

#

(Assuming you're okay with breaking compatibility)

tacit sparrow
#

oh, that's a good idea actually

crimson inlet
# 
class ConfigInterface(abc.ABC):
    @property
    @abc.abstractmethod
    def age(self) -> ReactiveField[ConfigInterface, int]:
        ...

    @property
    @abc.abstractmethod
    def no(self) -> int:
        ...

@reactive_attrs
class Config(ConfigInterface):
    age: ReactiveField[ConfigInterface, int] = field(1)
    no: int = 5

def set_value(config: t.Optional[ConfigInterface]=None):
    if config is None:
        config = Config()

    callback = lambda a: print("Lol")

    config.__class__.age.add_observer(config, callback)``` Hi I currently have this set up, Im  getting the  following error from pyright: `Cannot access attribute "add_observer" for class "property"` because `age` is of type `ReactiveField[ConfigInterface, int] | property`, how can I make the call legal?
crimson inlet
#

Should I use a Protocol instead like this?

class ConfigInterface(t.Protocol):
    age: ReactiveField[ConfigInterface, int]
    no: int``` seems to solve the issue
stiff acorn
#

I've never worked with stub files but want to write a type-hint package for a library I use

#

I read the mypy documentation but it barely scratched the surface

#

running stubgen just gave me 1 init.py file

# 
from ._errors import *
from ._magnet import Magnet as Magnet
from ._stream import TorrentFileStream as TorrentFileStream
from ._torrent import Torrent as Torrent
from ._utils import File as File, Filepath as Filepath

__version__: str

this was all it spat out

#

would appreciate any pointers I can get on how to tackle this

#

I'm guessing i'll have to copy the file structure but with .pyi instead?

tranquil ledge
#

e.g.

  1. Original
# torf/_errors.py
class TorfError(Exception):
    """Base exception for all exceptions raised by torf"""
    def __init__(self, msg, *posargs, **kwargs):
        super().__init__(msg)
        self.posargs = posargs
        self.kwargs = kwargs
  1. .pyi with same content
# torf/_errors.pyi
class TorfError(Exception):
    """Base exception for all exceptions raised by torf"""
    def __init__(self, msg, *posargs, **kwargs):
        super().__init__(msg)
        self.posargs = posargs
        self.kwargs = kwargs
  1. Add annotations to the .pyi
# torf/_errors.pyi
class TorfError(Exception):
    """Base exception for all exceptions raised by torf"""
    def __init__(self, msg: str, *posargs: object, **kwargs: object) -> None:
        super().__init__(msg)
        self.posargs = posargs
        self.kwargs = kwargs
  1. Remove whatever's now unnecessary
# torf/_errors.pyi
class TorfError(Exception):
    def __init__(self, msg: str, *posargs: object, **kwargs: object) -> None: ...
  1. Repeat 1-4 for every other symbol definition as needed.
#

Might necessitate a bit of jumping around as one figures out what the annotations should be based on context, usage, and documentation, but this approach usually gets me to the end.

stiff acorn
#

Thank you for the easy breakdown!

#

That makes sense and should get me started

tranquil ledge
#

Glad it seems helpful. I'm no expert, so don't take this as prescriptive; it's more of an example to potentially base your own workflow off of :D

#

Side note: it's less helpful the more complicated the package is. Then more forethought is required about file structuring, what needs to be "lied" about (i.e. what parts of your stubs won't match the runtime), etc., to avoid stalling partway through the process.

#

There's also a bit of documentation about maintaining type stubs in the typing docs, e.g.
here.

stiff acorn
stiff acorn
tranquil ledge
# stiff acorn Thank you for this

No problem. That particular page just recently underwent (EDIT: and is still undergoing, actually) a massive rewrite to be more helpful for stub writers, so all credit to those involved with that.

dire glen
#

Hey, I have this function but how should I tpe hint it since the return depends on the input ? ```py
def split(list_a: Iterable[Any], chunk_size: int) -> Any:
for i in range(0, len(list_a), chunk_size):
yield list_a[i:i + chunk_size]

oblique urchin
#

or before Python 3.12, define T = TypeVar("T")

dire glen
oblique urchin
#

a generic function

#

!pep 695

rough sluiceBOT
dire glen
#

Nice, this looks great 🙏

dire glen
#

on list[T]

oblique urchin
#

oh sorry, return Generator[list[T], None, None]

#

I didn't read your function right and thought it returned a single list

dire glen
#

Oh okay, this is my bad i said it returns Any instead of Generator too

viscid spire
#

I do suggest to name your parameter something other than list_a if you don't expect only lists

#

oh wait...

#

ok I suggest to use list[T] because you can't slice iterables like that, unless you use itertools.islice in the implementation

dire glen
atomic totem
viscid spire
#

and proxies it

dire glen
#

oh okay

viscid spire
#

to add generic subscription

#

but now they changed it so you can subscript the original

dire glen
#

Iterable = _alias(collections.abc.Iterable, 1) oh right

viscid spire
#

they kept the proxies in typing for backwards compatibility ig

viscid spire
#

added in 3.12

dire glen
#

i never used itertools that much, but it looks like a very powerfull lib

viscid spire
#

yeah it's amazing

#

everyone who has not seen it and learns about it in this server seems to love it

#

there is also more_itertools that you can pip install

dire glen
viscid spire
#

I have not seen it with my own eyes but I've only heard good things

rare scarab
#

should more-itertools be written in c like itertools is?

dire glen
viscid spire
rare scarab
#

no

viscid spire
#

it certainly would be faster that way

rare scarab
viscid spire
#

makefile for what smh

rare scarab
#

makefile can be replaced with taskipy

dire glen
viscid spire
#

spelin 100

viscid spire
dire glen
#

although this is a list and not a generator

viscid spire
#

I'm sure partly because people liked it in more_itertools

viscid spire
rare scarab
#

yes.

dire glen
rare scarab
#

groupby also yields iterables

viscid spire
#

ok well that's incorrect

#

batched yields tuples as I have just read in the docs

dire glen
#

this is from docs.python.org: ```py
def batched(iterable, n):
# batched('ABCDEFG', 3) → ABC DEF G
if n < 1:
raise ValueError('n must be at least one')
iterator = iter(iterable)
while batch := tuple(islice(iterator, n)):
yield batch

viscid spire
#

yeah that's yielding tuples?

rare scarab
#

because the batch := tuple(...)

dire glen
#

isnt a tuple an iterables?

viscid spire
#

I said iterator

#

and I would not generalize to "iterable" in the case of tuple because we know it is only and always tuple here

#

the difference between batched and chunked is that chunked yields lists, while batched yields tuples

#

I would guess the choice is due to which language the functions are implemented in

terse tree
#

Have a question on pylance/pyright

class A(Generic[T]):
    def __init__(self, callable: Callable[[T], T]):
        self.callable = callable
    def method(self, y: T) -> T:
        return y
W = TypeVar("W", bound=int)

def f(x: W) -> W:
    return x

a = A(f)
a.method(6)

a.method(6) errors with Argument of type "Literal[6]" cannot be assigned to parameter "y" of type "W@f" in function "method"

#

Any ideas on why this is?

rare scarab
#

A[T] could be bool

#

do A[int](f)

terse tree
#

@rare scarab interesting. Now I'm really trying to use the class as a decorator and have subclasses like

class A(Generic[T]):
    def __init__(self, callable: Callable[[T], T]):
        self.callable = callable
    def method(self, y: T) -> T:
        return y

W = TypeVar("W", bound=int)

@A[int]
def f(x: W) -> W:
    return x

a = f.method(6)

class MyInt(int):
    pass

b = f.method(MyInt(6))

Here the type of b is inferred as int but it should be MyInt

rare scarab
#

Does A need to be generic, or just the function?

terse tree
#

A needs to be generic because I can use it on different functions

rare scarab
#

That doesn't work with generic functions

#

Try using a TypeAlias. ```py
T = TypeVar("T")
UnaryOperator: TypeAlias = Callable[[T], T]

class A:
def init(self, callable: UnaryOperator):
self.callable = callable
def method(self, y: T) ->: T:
return y

#

You could also call the function type Identity

terse tree
#

When I'm decorating a function as above, how do I indicate in the type argument that I need an int or any subclass of int?

rare scarab
#

bound=int

terse tree
#

Not exactly what I'm looking for but thanks

stiff acorn
#

should overloads use ... for default values or should the use the actual default value?

#

i.e,

# 
    @overload
    def get(
        cls,
        key: str | int | None = ...,
        default: Literal["stored", "deflated", "bzip2", "lzma"] = ...,
    ) -> CompressionType: ...
#

or

# 
    @overload
    def get(
        cls,
        key: str | int | None = None,
        default: Literal["stored", "deflated", "bzip2", "lzma"] = "stored",
    ) -> CompressionType: ...
#

on a similar note, if get is a classmethod, do i need to add the classmethod decorator to the overload too?

# 
    @overload
    @classmethod
    def get(
        cls,
        key: str | int | None = None,
        default: Literal["stored", "deflated", "bzip2", "lzma"] = "stored",
    ) -> CompressionType: ...
viscid spire
stiff acorn
#

thanks

pastel egret
#

Yes you should have the same decorators on all the overloads for clarity. Currently, the default's value has no meaning so you can leave it out, but you might want to leave it in for documentation purposes.

lunar dune
stiff acorn
#

Yep! I noticed that soon after in VS code when I hovered so I did go with adding real defaults in the overloads

stiff acorn
#

What's wrong here?

# 
import tarfile

with tarfile.open("filename.tar", "w", compresslevel=None) as tar:
    tar.add("readme.txt")

This code errors in runtime

#

but passes type-checking

# 
❯ mypy . --strict
Success: no issues found in 1 source file
#

I'm not sure what I'm doing wrong

# 
Traceback (most recent call last):
  File "c:\Users\raven\Documents\GitHub\tarfiletest\test.py", line 3, in <module>
    with tarfile.open("filename.tar", "w", compresslevel=None) as tar:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\raven\AppData\Local\Programs\Python\Python312\Lib\tarfile.py", line 1844, in open
    return cls.taropen(name, mode, fileobj, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\raven\AppData\Local\Programs\Python\Python312\Lib\tarfile.py", line 1854, in taropen
    return cls(name, mode, fileobj, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: TarFile.__init__() got an unexpected keyword argument 'compresslevel'
rare scarab
#

tarfile doesn't support compression

stiff acorn
#

but it's typed as if it does?

rare scarab
#

use tarfile.open("filename.tar.gz", "w:gz", compresslevel=None)

#

But at that point, why bother with gzip?

#

the default is to not compress

oblique urchin
stiff acorn
#

another mistype i think

# 
import tarfile

with tarfile.open("filename.tar.gz", "w:gz", compresslevel=None) as tar:
    tar.add("readme.txt")
#

passes mypy strict

#

but fails at runtime

# 
Traceback (most recent call last):
  File "c:\Users\raven\Documents\GitHub\tarfiletest\test.py", line 3, in <module>
    with tarfile.open("filename.tar.gz", "w:gz", compresslevel=None) as tar:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\raven\AppData\Local\Programs\Python\Python312\Lib\tarfile.py", line 1822, in open
    return func(name, filemode, fileobj, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\raven\AppData\Local\Programs\Python\Python312\Lib\tarfile.py", line 1870, in gzopen
    fileobj = GzipFile(name, mode + "b", compresslevel, fileobj)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\raven\AppData\Local\Programs\Python\Python312\Lib\gzip.py", line 220, in __init__
    self.compress = zlib.compressobj(compresslevel,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object cannot be interpreted as an integer
oblique urchin
#

we probably need compresslevel: int on some overloads and no compresslevel at all on others

rare scarab
#

I looked at the source. tarfile.open will redirect to one of TarFile.taropen, TarFile.gzopen, TarFile.bz2open, or TarFile.xzopen depending on the mode.

#

taropen (the default) will create TarFile, gzopen will wrap gzip.GzipFile

stiff acorn
#

lemme know if I need to add any more details

buoyant ridge
#

hey y'all, I'm struggling to rly wrap my head around the problem I'm having here. I've had a look at co/contra-variance of type variables, but id appreciate if anyone knows a good resource they can redirect me to.

class MyList[X](list):
    def __add__(self, v: "MyList[X] | typing.Sequence[X]"):
        return self + list(v)
# ... Type parameter "_T_co@Sequence" is covariant, but "_S@__add__" is not a subtype of "X@MyList" ...

so the parameter of a sequence is covariant, but the parameter of the add dunder (in the original definition?) isn't compatible?
replacing X with a new type parameter for the add dunder method [S] works - but that doesn't make sense to me. surely they should be of the same type (X)

brazen jolt
#

Sequence is covariant in its generic type, however lists are invariant, also I think you'll need to change that class def to:

class MyList[X](list[X]):
    ...
buoyant ridge
#

oh, yeah, i had forgotten to subscript the list. so in this case do i want to inherit from abc’s Sequence? how do you even like, instantiate a Sequence?

#

the article is pretty good ty! i think i understand variance now, im just a bit confused about why i’m getting this error / the solution for this implementation

viscid spire
buoyant ridge
#

yeah, i meant the subclass of a sequence, like the class i posted above if it inherited from sequence instead of list

frigid jolt
#

NewType(s) are invariant compared to the type from which they derive, is that right?

trim tangle
#

If you do UserId = NewType("UserId", int), then UserId is a subtype of int

from typing import NewType

UserId = NewType("UserId", int)

x = UserId(69)
print(x ** 2)  # ok

def square(y: int) -> int:
    return y**2
square(x)
#

(but int is not a subtype of UserId, of course)

#

variance (co-/contra-/invariance) is a property of a generic type in relation to one of its type variables

leaden ember
#

Whats the best practice for using if TYPE_CHECKING:? Is it only to avoid circular imports, or should I be puting anything in there that I'm only using for type annotations?

#

I guess theres also performance considerations from the imports?

crisp steppe
#

chasing to only pull in types for type checking when you dont use the type explicitly anywhere but type signatures will spend a lot of effort keeping on top of it. on a multi person project you will have a bad time (unless ruff or autoflake or isort can solve this for you).

further, importing can have side effects as code can be run at the top level so your behaviour could, in theory, change if you mess with imports in this way.

#

words words words -> I would say only to avoid circular imports - until tooling can help tidy things up further.

leaden ember
#

I see. We're using Ruff + isort, and our codebase has no types at the moment. So I think I'll just put the imports I'm doing in the TYPE_CHECKING statement, and if in future people want to move it out they can 🙂

#

From what you're saying, "If you have the import already, moving it into the TYPE_CHECKING statement isn't worth it, but if you have the tooling then go for it" would that be an accurate takeaway?

crisp steppe
#

one thing you will be surprised and possibly depressed about is that your types dont line up and getting from no types to some types AND having mypy reject bad PRs it a long and winding road.

#

but if you have the tooling then go for it" would that be an accurate takeaway?

yes, but to be clear, I'm saying I think there is no tooling yet that will shuffle imports such that it can detect you are only using a type as a type description as opposed to an actual value and hence move it into an if TYPE_CHECKING block.

leaden ember
#

Ah gotcha. At this point I'm pretty sure just because I'm the one importing it as part of adding type hints

Yeah, I get what you mean 😂. This codebase isn't structured to have type hints. Lots of funky stuff happening like setattr(self, name, value) on objects which is screwing with the static typechecking. At the moment I'm running mypy in CI and just ignoring the status (instead piping it to CodeCov for reporting)

crisp steppe
#

so the hard fought knowledge i got from this is that you should start at the base types and work down. we began with tests and worked our way inward. this is nice because most tooling starts in tests and once it's good in tests, it can work its way into the main code base.

however the type checkers will type check against the list of files you provide. so if you say test_a and test_b are passing then they will check that file. but they wont check types imported from foo.py. once you think foo.py will pass, suddenly you might find errors in test_a and test_b. This makes the whole process n^2.

starting from base types to get them clean means you have a shot of having a single pass through the system.

anyway, that's my 2c. Maybe others can share their experiences.

leaden ember
#

I see. So you run mypy tests instead of mypy src? (at least to start with?)

#

I didn't think to run mypy tests but I think that makes a lot of sense

crisp steppe
#

we keep a [tool.mypy] section in our pyproject.toml and only put files in when they are fixed up.

leaden ember
#

Gotcha. Incrementally doing it on a file by file basis (by modifying pyproject.toml), starting from the test suite, and focusing on the core types used in the project (as well as type arguments in the public API) and then incrementally working in from there

#

And if you have good coverage, you only really need to worry about mypy with the tests folder

crisp steppe
#

starting from the test suite is what i/we did but i dont think it's a good idea and i wouldn't do that approach again

#

and i would only really focus on src

pastel egret
crisp steppe
brazen jolt
# buoyant ridge yeah, i meant the subclass of a sequence, like the class i posted above if it in...

sequence is just an abstract base class, it declares some abstract methods (__getitem__ and __len__) and gives you some methods for free that you inherit in, like __iter__, __contains__ and some others. How you implement those abstract methods is up to you, a common way is to use an internally stored list, like self._list:

from typing import override
from collections.abc import Sequence, Iterable


class MySequence[T](Sequence[T]):
    def __init__(self, it: Iterable[T], /) -> None:
        self._list = list(it)

    @override
    def __getitem__(self, index: int) -> T:
        return self._list[index]

    @override
    def __len__(self) -> int:
        return len(self._list)
trim tangle
#

🥴 discord syntax highlighting

viscid spire
#

so yes you can do the things that usually require those, but it's not because you are inheriting the methods.

#

!e

class MySequence:
    def __getitem__(self, index: int) -> int:
        if index > 3:
            raise IndexError
        return index


xs = MySequence()

for x in xs:
    print(x)

if 2 in xs:
    print("2 in xs")

if 4 in xs:
    print("4 in xs")
rough sluiceBOT
brazen jolt
viscid spire
brazen jolt
rare scarab
#

Probably a bug in the Sequence mixin

trim tangle
#

I mean, it may be sad that it's slower, but it doesn't promise any particular performance properties 🙂

#

in fact it may be impossible to make it as fast (because Sequence always has an __iter__ method by design, you can't "un-have" it)

oblique urchin
# trim tangle why?

Yeah I'd expect this is because __iter__ as a Python method is slower than whatever it's doing in C with just __getitem__ and __len__

trim tangle
#

yep

buoyant ridge
sinful carbon
#

Is there a way to hint to mypy what the concrete type of an opaque value is? Atleast without isinstance(), type(), or issubclass()?

#

For example I might have stored a value in a larger opaque type I have no control over:

def main(ctx: Context):
  ctx.counter = 42

and this might come back as

def foo(ctx: Context):
  ctx.counter += 1 # mypy says this is dangerous since counter could be Any
#

counter here will only ever be int, but it's hard to suggest this to mypy

rare scarab
#

How is Context defined?

fervent girder
#

cast go brr

fervent girder
brazen jolt
#

also, if you have control over Context class, but it's just defined in a way that doesn't hint for that specific value, you can add an annotation for it explicitly in the class, like this

image.png
#

alternatively, you can subclass Context

#

since with casts, you'll need to do that every time you use that var

brazen jolt
# brazen jolt alternatively, you can subclass Context

or, if you can't subclass, because of some dependency injection logic that will simply always give you Context, you can tell the type checker to assume a custom type, while on runtime, it will just be Context, though this may be overkill at that point, and you might be better off with casts, unless you need this often and for multiple variables

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from my_project.typing import Context
else:
    from external_lib import Context


def foo(c: Context):
    ...

with my_project.typing having: ```python
from external_lib import Context as ExternalContext

class Context(ExternalContext):
x: int
y: str
z: tuple[int, ...]

sinful carbon
#

Subclassing is a neat idea, but not entirely sure if it's the right choice

brazen jolt
#

what lib are you using?

sinful carbon
#

FastAPI

brazen jolt
#

hmm, I'm not aware of any Context class in fastapi, or was that just a simple example?

sinful carbon
#

The actual type is State

brazen jolt
#

hmm, that is a bit annoying, there doesn't seem to be any official typing support there, you could go with the custom class route, or you could make a function that gives you some other properly typed class from your state

#

so say: ```python
async def get_state(request: Request) -> MyState:
return cast(MyState, request.state._state)

#

I'm pretty sure you can even make this into a contextmanager and request it from the endpoint with Depends

sinful carbon
#

Even though is a small shim, doing app.state.foo = ... will add it to it's ._state

brazen jolt
#

oh

#

well, I meant it more like a random variable that you set on initialization

#

didn't realize it clashed

#

you can call it anything

sinful carbon
#

I'm not exactly sure you can use Depends for this? I need to get .app.state from Request, and I can't do that if it's in a separate function body

#

Unless you have some information to connect these two

brazen jolt
#

fastapi has a custom Depends annotation that you can pass a function into, and it will call it for you, I'm pretty sure it passes request onto that function

# 
from contextlib import asynccontextmanager
from collections.abc import Iterator
from dataclasses import dataclass
from fastapi import Request, Response

@dataclass
class MyState:
    x: int
    y: iny

@asynccontextmanager
async def get_state(request: Request) -> Iterator[MyState]:
    yield request.state.my_state


# during initialization, set `state.my_state`

# in individual routes, you can then request this with Depends[get_state] annotation:

@app.get("/")
def my_route(request: Request, state: Depends[get_state]) -> Response:
    ...
#

note that this is very much pseudo-code, I'm writing this from memory of fastapi, so it might not work as-is

#

I'm also not sure how Depends plays with type-checkers

#

I think you may need to do something like: StateDep = Annotated[State, Depends(get_state)]

and then ```python
def my_route(req: Request, state: StateDep):
...

#

to get the proper types

#

but cross-check this with fastapi docs

sinful carbon
#

mypy complains about conversions?

async def get_dep(req: Request) -> Context
   app: FastAPI = cast(FastAPI, req.app) # Type of "app" is Any
#

For whatever reason they thought it was a good idea to complain about this while I'm casting it?

brazen jolt
#

mypy complains about any occurence of the Any type

#

including in a cast statement

sinful carbon
#

Well that's silly

brazen jolt
#

yeah

#

I mean

#

you can ignore it

sinful carbon
#

Especially if the user knows the concrete type...

brazen jolt
#

you're probably using strict settings

sinful carbon
#

To be fair, the Any part isn't my fault, it's just how the library was typed.

brazen jolt
# 
# ruff: noqa: D101,D103

from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from typing import Annotated, cast, reveal_type
from fastapi import Depends, FastAPI, Request, Response
from dataclasses import dataclass


@dataclass
class MyState:
    x: int
    y: int


async def get_state(request: Request) -> MyState:
    return cast(MyState, request.app.state.my_state)


StateDep = Annotated[MyState, Depends(get_state)]

###

app = FastAPI()

app.state.my_state = MyState(1, 2)


@app.get("/")
async def root(request: Request, state: StateDep) -> Response:
    reveal_type(state.x)
    return Response(content=f"{state.x}")
#

this code passes mypy --strict

brazen jolt
sinful carbon
brazen jolt
#

I'm not sure what mypy settings you're using

#

the Any check if definitely not default

#

it's not even in the --strict I don't think

sinful carbon
#

How would I get it to dump my settings?

brazen jolt
#

they're probably in pyproject.toml

sinful carbon
#

That's not there in my project

brazen jolt
#

could also be some settings in your editor

sinful carbon
# brazen jolt

According to this, then it might be these?:

[mypy]
plugins = numpy.typing.mypy_plugin
show_absolute_path = True
implicit_reexport = False
pretty = True
brazen jolt
#

looks like it, but that doesn't seem that strict

#

it's weird that you're even seeing that error

brazen jolt
sinful carbon
brazen jolt
#

run mypy in terminal

brazen jolt
# sinful carbon

what's that editor? the way those errors renders looks very neovim like but it seems too graphical

sinful carbon
sinful carbon
#

Might be because it's inside Neovide?

brazen jolt
#

oh, possibly

#

okay, so I was able to reproduce this with --disallow-any-expr

#

but that's definitely not a rule that's enabled by default

#

also, your editor didn't complain on that app.get decorator

#

which is weird

sinful carbon
#

If I declare my own decorator it does, which I've ran into

brazen jolt
#

that's super weird, do you have some global mypy config?

sinful carbon
#

A lot of this might be because I don't use the standard flavor of mypy?

#

Just now remembering that I'm using basedmypy

brazen jolt
#

oh is it basedmypy?

#

ah

#

yeah that makes sense then

#

in there, the rule probably is just enabled by default

#

that's why I was confused

brazen jolt
#

or add an inline ignore

sinful carbon
#

I'll probably just disable it temporarily

#

Maybe I can fill out an issue to get some workaround, either in FastAPI or mypy

brazen jolt
#

something like # type: ignore[no-any-expr] should be enough on that line

sinful carbon
#

I don't feel like it should be that hard to suggest to mypy that I know what a type is

brazen jolt
#

it's just that mypy doesn't distinguish between unknown & any

#

in basedpyright, you have these rules too, but there's one for reporting uses of any, and another for reporting uses of unknown

#

I don't usually enable the unknown one

brazen jolt
sinful carbon
brazen jolt
#

the point is to get you to use typed libs, or have mypy generate stub files that you can edit and add types for

sinful carbon
brazen jolt
#

yeah

#

I mean you could also just use your own global state

sinful carbon
#

So it's more like a TypedDict, but there's no way for me to tell mypy I know the fields concrete types

brazen jolt
#

just having contants.py with a pre-initialized custom class that stores state would also work

#

it's because python doesn't actually re-execute files that were already imported, so the instances would remain the same even if they're imported multiple times

brazen jolt
crimson inlet
#

Hi is there a way to override stdlib stubs?

#

specifically I want to override the tkinter.Event type hint, as it is typed as a generic class, however the implementation doesnt support subscription which always results in a type error

soft matrix
#

sounds like you should make a pr to cpython to fix it, however you should use future.annotations and or stringise the annotation for now

crimson inlet
#

alright thanks!

oblique urchin
#

But better to make PR (to typeshed not cpython) to fix it

#

actually scratch the typeshed PR part, this doesn't sound like something we'd change in typeshed

lunar dune
crimson inlet
#

I wasn't sure whether the fix should be on typeshed part or cpythons part either

oblique urchin
lunar dune
#

Jelle and I both got the notification, we saw 😄

rustic gull
#

I keep getting confused about this

# 
from collections.abc import Callable
class Base: pass
class Child1(Base): pass
class Child2(Base): pass

def outer(n: Callable[[Child1 | Child2], None]): pass
def inner(b:Child1): pass

outer(inner)

"Child1 | Child2" is incompatible with type "Child1"

#

why??

trim tangle
#

(which would not work, because inner only works with Child1)

rustic gull
#

how do I type them

#

that both work

trim tangle
#

What do you mean?

#

If inner works with either Child1 or Child2, you need to do def inner(b: Child1 | Child2)

#

If inner works with any Base, you can do def inner(b: Base)

rustic gull
#

why doesn't this work

from collections.abc import Callable
class Base: pass
class Child1(Base): pass
class Child2(Base): pass

def outer(n: Callable[[Base], None]): pass
def inner(n: Child1): pass

outer(inner)
#

Base" is incompatible with "Child1

#

it literally isn't

trim tangle
trim tangle
rustic gull
#

yea

trim tangle
#

This is the same as both of your examples

#

Outer needs a function that accepts any Base. But inner only works with some Bases (for example it's not guaranteed to work with Child2)

rare scarab
#

What about using a bound generic?

# 
def do_things[T: (int, str)](f: Callable[[T], int]) -> int:```
trim tangle
rare scarab
#

I'm not sure how outer is implemented now

trim tangle
# rustic gull it literally isn't

The message "Base" is incompatible with "Child" might be confusing, but it has to do with variance. Functions are contravariant in their parameter types. You can read more about that here or here or here

jolly cipher
cinder bone
restive rapids
rustic gull
strong plover
#

I'm using a decorator to wrap some ORM code that returns a bool. I wanna type-hint that it returns a bool like this:

def make_user_mod_in_room(
    user_modding: User, user_being_modded: User, room: ChatRoom
) -> bool:```

Then I call the function with await from async code:
```status = await make_user_mod_in_room(self.user, user_being_modded, self.room)```

When I have the bool type hint, PyCharm underlines the await keyword and says: "Class 'bool' does not define '__await__', so the 'await' operator cannot be used on its instances". Removing the bool type hint removes the warning message.

The code works just fine either way. But do I need to change the type hint in some way to remove the warning message?
restive rapids
#

you'd want to make database_sync_to_sync return a callable which returns an Awaitable[bool], i think?

strong plover
buoyant ridge
#

anyone know if it’s possible to access the type parameter of a generic? like

class Foo[T: int]: …

class Bar[F: Foo]:
    footype: Foo.T
#

or is this like infeasible

trim tangle
#

ah

#

Do you have an example of how you'd use it?

#

Maybe you want something like ```py
class Bar[T: int]:
foo: Foo[T]

buoyant ridge
#

yeah i’ll go for this, ty :)

cinder bone
#

I wonder if it is possible to use typing.NewType with Self
(In my case it doesn't need to exist at runtime necessarily)

soft matrix
#

how/why?

tranquil turtle
#

anonymous NewType's don't make any sense, I think

cinder bone
soft matrix
#

i still dont really see it

#

Self isnt valid outside of a class body

tranquil turtle
# 
type Proxy[T] = T
cinder bone
cinder bone
# soft matrix Self isnt valid outside of a class body

well yeah but it would be nice for like

class Object:
    AnimationBuilder = NewType("AnimationBuilder", Self)
    @property
    def animate(self) -> AnimationBuilder:
        return cast(self.__class__.AnimationBuilder, ObjectProxy(self))  # proxies everything to Object and then builds it later
tranquil ledge
#

You could lie:

if TYPE_CHECKING:
    class ObjectProxy[T]:
        def __new__(cls, proxied: T) -> T: ...
else:
    # Actual implementation
    ...

class Example():
    ...

reveal_type(ObjectProxy(Example()))   # Type is “Example”

Version with sample implementation

#

This works with pyright, at least.

tranquil ledge
leaden ember
#

TLDR; experiencing problems getting NameErrors for things I put in the TYPE_CHECKING block

cinder bone
tranquil ledge
cinder bone
tranquil ledge
#

I'm at a loss, sorry. It feels like you're looking for another generic proxy with slightly different characteristics that don't compose well at runtime or type-checking time.

buoyant ridge
#

Not that I need a solution to this (I think it would just be explicit casting of the Foo) but do people that the following would be a desirable and feasible feature? (The feature being the replacement for Unknown)

class Foo[X]:
    pass

my_list: list[Foo[int]] = []

foo = Foo()
my_list.append(foo)

foo # Foo[Unknown] COULD BE? Foo[int]
my_list # list[Foo[int]] (NOT Foo[Unknown])

Nevermind, looking over this again I can see that this could infer a more specific type than necessary, preventing it from being used in other contexts

viscid spire
#

huh, pyright bug

from typing import TypeVar, reveal_type

T = TypeVar("T", bound=int)

def f(x: T) -> T:
    if isinstance(x, int): # somehow the isinstance makes pyright pass
        return int(x)

reveal_type(f(123))
reveal_type(f(True))
tranquil ledge
viscid spire
tranquil ledge
#

Oh, I think I see.

#

Right. Hmm.

viscid spire
#

if you remove the isinstance, it does not pass

#

but somehow adding it in removes the typechecking error

tranquil ledge
#

I guess the error is in how pyright calculates the type of x after the isinstance check; it looks like it's synthesizing int as the result, but it should be maintaining the type is still T since T has a bound of int. The isinstance check should be a no-op from the type-checker's perspective. Feels like a bad intersection calculation.

rigid sapphire
#

bound doesnt do much here

#

imagine you didn't have the bound

#

would return int(x) do the right thing here, well why would it do it if bound

#

T being bool would be confusing

#

this example becomes clearer if you use int -> A|B and bool -> B

#

where if you pass B, it is bound by A|B, but is if you return A is wrong

#

thats the difference between being bount by a thing and being that thing

rustic gull
#

hi,
is a: ... the same as a: Any?
is typing.Protocol enforced at runtime (does it affect performance)?

lunar dune
lunar dune
trim tangle
#

runtime_checkable is also a very superficial check, it only checks the names of methods, not even the number and names of arguments

lunar dune
undone saffron
#

runtime_checkable is basically a trap.

#

should only be using protocols when you are being permissive to begin with, and so static analysis should be plently, no runtime branching on that protocol neccessary.

lunar dune
#

yeah, I basically agree

iron brook
lunar dune
#

a: Any will be accepted by type checkers but they will probably complain if you do a: ...

steady lily
#

Question about type hinting and dataclasses... is there anyway using dataclasses for the type of an input parameter to be different than the type of the class attribute after initialization? For instance, is it possible to do something like the following but with dataclasses?

class Config:
    def __init__(self, a: int, b: Optional[int]=None):
        self.a = a
        if b is None:
            self.b: int = a + 10
        else:
            self.b: int = b
cinder bone
steady lily
rare scarab
#

Also, if T is bound to int, then the isinstance check will always return True, which means your return type was wrong anyway

trim tangle
#

That's the point, pyright is not flagging it

rare scarab
#

Hm, I see

#

But it does warn without the isinstance

#

I think what pyright is doing should only be technically valid for type(x) == int

arctic merlin
#

hello

tranquil basin
#

Not sure how to ask this, but, I'd like to create a subtype which is based on a string ... I'm not able to articulate this well so example below:

Currently have something like:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from mypy_boto3_dynamodb.service_resource import Table

class DynamoDB():
    def table(self, name: str) -> "Table":
        return self._resource.Table(name)

But, I'd like to be able to have a type for each "table sub-type", for type-checking in my code so I don't accidentally use one Table when I meant to use a different Table.

Ie, something like this, where I have a UserTable type and a SessionTable type. Following code won't run obviously, just trying to show what I want:

UserTable = TableTypeFromName["user"]
SessionTable = TableTypeFromName["session"]

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from mypy_boto3_dynamodb.service_resource import Table

class DynamoDB():
    def table(self, name: str) -> ???:
        return cast(TableTypeFromName[name], self._resource.Table(name))
#

How would I do something like this? ^

trim tangle
tranquil basin
#

is there a way to do it without overloading?

trim tangle
tranquil basin
#

name is put into an interpolated string, so "user" becomes something like "myapp-staging-user". I don't currently have the UserTable or SessionTable types, but I'd want them to subclass Table, which I'm not sure how to do ... is it possible to do this with generics or something?

I used to have the code as something like this, I could go back to doing it this way, but I guess how would this all be typed?

class DynamoDB():
    def user_table(self) -> "Table":  # not sure how this should be typed
        return self._resource.Table("user")
trim tangle
crimson inlet
#

How does TypeVar's and ParamSpec's default keyword translate to the type parameter syntax?

trim tangle
crimson inlet
#

Ah alright, what about ParamSpec bound

#

does it even serve any purpose

tranquil basin
trim tangle
trim tangle
#

maybe I don't understand what happens at runtime

crimson inlet
trim tangle
#

no idea

tranquil basin
# trim tangle Is UserTable generated somehow?

I don't have UserTable atm, I'd like to create a UserTable type now though which is what I'm wondering how to do. From Python's perspective, I'm just doing normal Table operations, which I'd like to continue doing (I don't want to use an ODM/ORM and define my tables in Python code), I just want mypy to check that I'm actually operating on a UserTable when I do certain things, like:

def put_user(user_table: UserTable, some_user_dict: Dict[str, Any]):
  ...

I'm not actually doing it this way ^, but I want to be able to check "Hey, put_user should actually get a UserTable type when this is run, so make sure it's actually a UserTable and not a SessionTable"

#

right now the above looks like:

def put_user(user_table: Table, some_user_dict: Dict[str, Any]):
  ...

so I could accidentally pass a session table or something

trim tangle
#

(and make Table generic)

tranquil basin
#

thank you, will try something like that

earnest thorn
#

i am relatively new to mypy, so i hope this isn't a dumb question.

i have a function that looks like this:

def getfile(path: str|None = None, raw: str|None = None) -> bytes:
    if path is None and raw is None:
        raise ValueError()
    if path is not None and raw is not None:
        raise ValueError()
    if path:
        # open file and read it into value
    else:
        value = raw
    return value

mypy is yelling at me for the value = raw line:

Incompatible types in assignment (expression has type "bytes | None", variable has type "bytes")  [assignment]

what's the best way to get mypy to be happy here?

trim tangle
#

if you already have the data, why call this function?

earnest thorn
#

(the real function is a bit more complex)

trim tangle
#

Does it have this at the top? ```py
if path is None and raw is None:
raise ValueError()
if path is not None and raw is not None:
raise ValueError()

earnest thorn
#

yes

trim tangle
#

Then it's better to make two functions: file_from_path and file_from_raw

earnest thorn
#

there's just a bunch more lines between value = raw and return

#

yeah, maybe refactor is better now that i look at it :-p

trim tangle
#

Type checkers generally understand a fixed set of hard-coded guards, and currently they don't understand that the above lines mean that path is not None => raw is None and path is None => raw is not None

earnest thorn
#

yeah. i think i was in the wrong mindset. it's just bad code that should be re-worked

#

i was so focused on getting mypy happy, i wasn't thinking about that, lol

trim tangle
#

Well, this might be perfectly fine code if you're reading something from an existing config, for example. YAML doesn't have a way to express a "tagged union" natively

earnest thorn
#

yeah, makes sense

#

thanks

#

im going to try another question since that was so helpful 🙂

#

im writing a class to interact with a series of web panels across my company. there's at least three, all built on the same framework, all with terrible apis not meant to be interacted with like this, and each with different features

this has led me to a complex architecture that looks like this:

  • class Admin(ABC) - has basic functions like loading a config, handling login, basic get and post requests, search functionality, etc
  • three classes, say Admin1-3 that all inherit Admin and mixins.
  • a series of mixins that define functionality that may show up on one or more of the Admin classes. if a feature is only on 1 and 3, then i can just only include that mixin on those objects and they get the functionality.

this is complicated, but it has been working.

but now i am trying to get type hinting up to snuff, and im getting a ton of errors like "TagMixin" has no attribute "post_update". basically, attributes and methods from the Admin and Admin1-3 classes are not seen in the mixins.

i found this page: https://mypy.readthedocs.io/en/latest/more_types.html#mixin-classes

is it saying im supposed to create a protocol that defines what things are available? it makes sense for one attribute, but when each mixin needs 2-4, and they are a different 2-4, it seems like it'll get very complex very quickly.

i can try to make a toy example of the structure if that would be helpful

tranquil basin
#

fwiw I was able to do the "table typing" I wanted to do with the following, if anyone thinks this could be improved lmk 🙂

# in types.py
from mypy_boto3_dynamodb.service_resource import Table as TableClass

UserTableName = Literal["user"]
SessionTableName = Literal["session"]

TableName = Union[UserTableName, SessionTableName]

TableNameT = TypeVar("TableNameT", bound=TableName)

class Table(Generic[TableNameT], TableClass): ...

UserTable = Table[UserTableName]
SessionTable = Table[SessionTableName]


# in db.py
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from .types import Table, TableNameT

class DynamoDB():
    def table(self, name: "TableNameT") -> "Table[TableNameT]":
        return cast("Table[TableNameT]", self._resource.Table(name))


# somewhere else
def put_user(user_table: "UserTable", some_user_dict: Dict[str, Any]):
    ...

Which seems to work, though I'm not sure I should be calling cast in table

fervent girder
#

what does "if TYPE_CHECKING" mean?

#

never seen it before

earnest thorn
#

A special constant that is assumed to be True by 3rd party static type checkers. It is False at runtime.

fervent girder
#

like mypy and pyright?

earnest thorn
#

ive used it when i want to import something for type checking, but importing it at runtime would cause an import loop

#

(im a bit of a noob at this, so hopefully someone will correct me if im wrong)

oblique urchin
#

in general it's for having type checkers see different code than what happens at runtime

tame flame
#

Hey folks, mypy is complaining because I'm using modules as types:

from Crypto.Hash import SHA1, SHA256, SHA512  # pyCryptodome

HashType = type[SHA1] | type[SHA256] | type[SHA512]

def hash(msg: bytes, hash_alg: HashType):
    return hash_alg.new(msg).digest()

print(hash(b"Test", SHA1))

stuff.py:3: error: Module "Crypto.Hash.SHA1" is not valid as a type  [valid-type]
stuff.py:3: note: Perhaps you meant to use a protocol matching the module structure?
stuff.py:3: error: Module "Crypto.Hash.SHA256" is not valid as a type  [valid-type]
stuff.py:3: error: Module "Crypto.Hash.SHA512" is not valid as a type  [valid-type]
stuff.py:6: error: Item "type" of "type[SHA1?] | type[SHA256?] | type[SHA512?]" has no attribute "new"  [union-attr]
stuff.py:8: error: Argument 2 to "hash" has incompatible type Module; expected "type[SHA1?] | type[SHA256?] | type[SHA512?]"  [arg-type]
Found 5 errors in 1 file (checked 1 source file)

Any suggestions other than ignoring it? I don't want to write a Protocol

trim tangle
tame flame
#

Thanks, but I have to use PyCryptodome 😄

trim tangle
#

Why?

#

If you have to use pycryptodome, then the best you can do is make a protocol. There is no way in the current type system to use a module as a type

tame flame
#

Because I'm monkey patching the RSA module to generate faulty signatures 😄 This is research so it's no problem if the typehints are missing, I was just wondering whether there's a way

#

I do wonder why there's no way to use a module as a type though 🤔 Is there something about modules that makes it complicated or is it just not intended?

trim tangle
#

You probably care that it has particular attributes

tame flame
#

I mean, I do have the requirement right now because I want the caller to choose exactly one of those three modules 😄
pycryptodome could have made them classes, I could do a string argument instead, there are other options, sure. But since modules are objects, why not treat them as such in regards to type-hinting? Literal[SHA1, SHA256, SHA512] isn't allowed either

stiff acorn
#

Literal["SHA1", "SHA256", "SHA512"] will work

tame flame
#

Those are strings tho

stiff acorn
#

Then you can probably do some thing like getattr(Crypto.Hash, hash_alg).new(msg).digest()

tame flame
#

Sure, but that's rather gnarly

#

I could also have a dict or something but the cleanest thing would be to be able to typehint the argument correctly

#

I mean the code works perfectly, there's no reason to change it other than the typehints not working

trim tangle
stiff acorn
#

Lots of options to work with literal type of you really want to

tame flame
# stiff acorn Lots of options to work with literal type of you really want to

I'm repeating myself, but the code works as it is and passing a module as an argument to a function is a perfectly valid thing to do in Python. If rewrite it to accept a string instead and run that through a match case or something, I'm just adding unnecessary complexity, for the sole reason that the type system doesn't have good support for modules. I'm not going to do that

stiff acorn
#

I never said otherwise, was just answering your question on trying to make typehints play nice with your code via alternative methods

tame flame
tame flame
earnest thorn
#

im getting an error with this code:

        try:
            vpn_server: dict[str, str|int] = next(
                (v for v in vpn_servers if v["Name"].lower() == vpn_server_name.lower())
            )
            return self.get_vpn_key_by_id(vpn_server.get("id"))
        except StopIteration as ex:
            raise ValueError(f"VPN server {vpn_server_name} not found.") from ex

vpn_servers is a list of dicts, each with a "Name" key that has a str, and a "id" key that has an int. im trying to grab the first result with the given name (there will only ever be one), and then get the id.

mypy says:


Argument 1 to "get_vpn_key_by_id" of "VpnMixin" has incompatible type "str | int | None"; expected "int"  [arg-type]
#

maybe this is just another case of i should rewrite the code, but im curious if it's possible to typehint this correctly

tame flame
#

Mypy can't know what the value in your dict is going to be

#

If you're certain id is an int, you can use cast, otherwise you could check with an if before calling the function.

earnest thorn
#

sure... when i write it this way, why doesn't it have the same problem?

        vpn_servers = [
            vpn_server
            for vpn_server in vpn_servers
            if vpn_server["Name"].lower() == vpn_server_name.lower()
        ]
        if not vpn_servers:
            raise ValueError(f"VPN server {vpn_server_name} not found.")
        assert len(vpn_servers) == 1
        return self.get_vpn_key_by_id(vpn_servers[0]["id"])
tame flame
#

One thing, get can return None

#

Also you don't have a typehint for vpn_server here

#

It's probably inferred as Any and ignored, I'd guess

trim tangle
rare scarab
#

Found another pyright bug. ```py
from typing import TypeVar
from dataclasses import is_dataclass

T = TypeVar("T")

def load(t: type[T]) -> None:
if is_dataclass(t):
print("Hello")

It reports `is_dataclass(t)` will never be True.
#

microsoft/pyright#8686

mighty lindenBOT
young tundra
#

how do I typehint a direction? ```py
DIRECTION = Literal[(1, 0), (0, 1), (-1, 0), (0, -1)]

#

for that pylance says: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value

pastel egret
#

A tuple can't be a literal, but it can contain one:

DIRECTION = (
    tuple[Literal[1], Literal[0]] |
    tuple[Literal[0], Literal[1]] |
    tuple[Literal[-1], Literal[0]] |
    tuple[Literal[0], Literal[-1]]
)

I'd suggest using an enum instead though:

class Directions(enum.Enum):
    NORTH = (0, +1)
    SOUTH = (0, -1)
    EAST = (+1, 0)
    WEST = (-1, 0)
young tundra
#

oh what is an enum?

pastel egret
young tundra
#

perhaps you could improce these types too?

SCREEN_NAMES = Literal["display", "sprite", "buffer"]
COORD = tuple[int, int] | tuple[float, float]
pastel egret
#

It's a special class where only certain values can exist. Type checkers special case it heavily to implement all the functionality, it's very useful if you have a small number of fixed options.

#

Well screen names could also be an enum, but literals are fine too. COORD should just be tuple[float, float] - type checkers treat int as a subclass of float for convenience, so the union is actually redundant.

young tundra
#

thanks

#

could I say something like this? ```py
DIR_VECT = tuple[0 <= float <= 1, 0 <= float <= 1]

restive rapids
#

no we don't have propositions in types

pastel egret
#

DIR_VECT = tuple[Literal[-1, 0, 1], Literal[-1, 0, 1]] would work though. To a limited extent - type checkers are very limited in the kind of inferences they'll make with literals, because it would be really easy for them to end up computing absurdly large unions.

fervent girder
#

Is there any alternative to reinterpret_cast<...>(...) in python?

#

I just need
Cell* cell(size_t index) { return reinterpreter_cast<Cell*>(&m_storage[index + cell_size()]);

But in python

restive rapids
#

!d typing.cast

rough sluiceBOT
#
typing.cast 

typing.cast(typ, val)```
Cast a value to a type.

This returns the value unchanged. To the type checker this signals that the return value has the designated type, but at runtime we intentionally don’t check anything (we want this to be as fast as possible).
fervent girder
#

thanks

fervent girder
#

!d typing

rough sluiceBOT
#
typing

Source code: Lib/typing.py

Note

The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.

This module provides runtime support for type hints.

Consider the function below...

Similar names: numpy.label.typing, numba.typing, rich.typing
fervent girder
#

!d typing.assert_type

rough sluiceBOT
#
typing.assert\_type 

typing.assert_type(val, typ, /)```
Ask a static type checker to confirm that *val* has an inferred type of *typ*.

At runtime this does nothing: it returns the first argument unchanged with no checks or side effects, no matter the actual type of the argument.

When a static type checker encounters a call to `assert_type()`, it emits an error if the value is not of the specified type:

```py
def greet(name: str) -> None:
    assert_type(name, str)  # OK, inferred type of `name` is `str`
    assert_type(name, int)  # type checker error
```  This function is useful for ensuring the type checker’s understanding of a script is in line with the developer’s intentions:
young tundra
#

notice that the monster.rect.center is colored meaning hinter knows that its a monster

image.png
#

and with this it doesn't know:

image.png
fervent girder
#

I think

#

assert does affects runtime, assert_type doesn't

young tundra
#

and how do I make it so vscode highliter recognises assert_type?

fervent girder
#

Install mypy or add # type: ignore

young tundra
#

I have mypy installed

fervent girder
#

add # type: ignore

#

after assert_type line

#

like

#

assert_type(monster, Monster) # type: ignore

young tundra
#

I did assert type for vscode to color the stuff but it doesn't

fervent girder
#

Because it affect only type checkers

#

not highlighters

young tundra
#

comment typehint works:

image.png
#

is there a better way to do this?

restive rapids
#

define monsters such that it would be an Iterable[Monster]?

young tundra
#

Thanks

#

that works

young tundra
#

why does mypy care if I set a variable float to an int?

trim tangle
young tundra
rustic gull
#

is there any way to type it so that a function accepts all the same arguments as another function plus some extra ones

tranquil ledge
#

In a limited way, yes, with typing.Concatenate and typing.ParamSpec.

rustic gull
#

is there any way to use it to add an argument?

#

I was able to add a positional-only argument, can I add a normal one?

#

I did this

def paramspec_from(_: Callable[_P, _T]) -> Callable[[Callable[_P, _S]], Method[Concatenate[Any, _P], _S]]:
    def _fnc(fnc: Callable[_P, _S]) -> Method[_P, _S]:
        return fnc
    return _fnc
restive rapids
#

this is incorrect, do you even have a typechecker on?

rustic gull
restive rapids
#

your goal is "i have a function with some long signature, and i want to create a class with a method that has a signature like that function but with additional parameters", right?

rustic gull
restive rapids
#

the first part is fine (method with same signature as function), the additional/removed parameters are simply not expressible. like, you just cant do that, there is no way to make a generic solution to this problem with the current typesystem.
there is, in fact, no need for a generic solution if you have a concrete problem. you can just copy the signature and then modify it.

rustic gull
cinder bone
#

Something you could try is

def a(x: int) -> None: ...
def b(*args: P.args, f: Callable[P, T] = a, **kwargs: P.kwargs) -> T: ...
#

Not at my computer so can't check rn

rustic gull
cinder bone
#

I do agree with lambda though, you probably shouldn't be doing this unless you're making a decorator or something

rustic gull
cinder bone
#

Usually decorators look something like

def decorator(f: Callable[P, T]) -> Callable[P, T]:
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
        ...
        return f(*args, **kwargs)
    return wrapper
restive rapids
#

there has been like 3 cases where you said "it works" when it literally doesnt, how are you checking this?

viscid spire
#

P.args and P.kwargs have to be right next to eachother which is why that one doesn't work

rustic gull
#

and in all other cases it worked

restive rapids
#

.. you put # type: ignore, that will literally make anything "work"

rustic gull
#

I know

#

but I get type hints in b

#

and I can add new parameters to it

#

I only don't know yet how to make it into a decorator

#

tried this and it doesnt work

def wrapper(wrap) -> Callable[P, T]:
    f: Callable[P, T] = wrap
    def inner(*args: P.args,**kwargs: P.kwargs):
        def b(*args: P.args, f: Callable[P, T] = wrap, **kwargs: P.kwargs) -> T: return f(*args, **kwargs) # type:ignore
        return b
    new: Callable[P, T] = inner
    return new
somber edge
#

Any one need a website or an App developer?

vapid sparrow
rough sluiceBOT
#
Rules

9. Do not offer or ask for paid work of any kind.

tacit sparrow
#

Hi! What "Overload 1 for "myfunc" overlaps overload 2 and returns an incompatible type" means in this case and how can I resolve this?

from typing import overload, Union, Literal, assert_type


@overload
# PylancereportOverlappingOverload ERROR:
# Overload 1 for "myfunc" overlaps overload 2 and returns an incompatible type
def myfunc(v: int = 25, arg: Literal[True] = True) -> str: ...
@overload
def myfunc(v: int = 25, arg: Literal[False] = False) -> int: ...
@overload
def myfunc(v: int = 25, arg: bool = True) -> Union[int, str]: ...
def myfunc(v: int = 25, arg: bool = True) -> Union[int, str]:
    if arg:
        return "something"
    else:
        return 0

assert_type(myfunc(), str)
assert_type(myfunc(arg=False), int)
assert_type(myfunc(arg=True), str)


def test_bool_type(bl: bool):
    assert_type(myfunc(arg=bl), Union[int, str])
grave fjord
tacit sparrow
soft matrix
#

if youre wanting to do this it might be best to just have 2 different functinos

rare scarab
# 
@overload
def myfunc(v: int = ..., *, arg: Literal[True] = ...) -> str: ...
@overload
def myfunc(v: int = ..., *, arg: Literal[False]) -> int: ...
def myfunc(v: int = 25, *, arg: bool = True) -> Union[int, str]:
  pass
oblique urchin
#

That should be the overload that gets picked if the user doesn't pass that argument

tacit sparrow
tacit sparrow
# oblique urchin Only one of the overloads should have a default

But arg is following argument with default value, so it should have some default too.
I tried this but it has the same issue

@overload
def myfunc(v: int = 25, arg: Literal[True] = True) -> str: ...
@overload
def myfunc(v: int = 25, arg: Literal[False] = ...) -> int: ...
@overload
def myfunc(v: int = 25, arg: bool = ...) -> Union[int, str]: ...
def myfunc(v: int = 25, arg: bool = True) -> Union[int, str]:
    if arg:
        return "something"
    else:
        return 0
oblique urchin
#

The cause of the overlapping errors is that a call like myfunc() will match both overloads, and they ahve different return types

#

So to fix it, you need to make it so calls like myfunc() match only one overload

#

I also agree with @soft matrix's advice above that it's probably better to make this two functions

#

But if you need to keep it in one function, this is the way to go

tacit sparrow
#

oh, this works!

# 
from typing import overload, Union, Literal, assert_type


@overload
def myfunc(v: int = 25, *, arg: Literal[True] = True) -> str: ...
@overload
def myfunc(v: int = 25, *, arg: Literal[False]) -> int: ...
@overload
def myfunc(v: int = 25, *, arg: bool) -> Union[int, str]: ...
def myfunc(v: int = 25, arg: bool = True) -> Union[int, str]:
    if arg:
        return "something"
    else:
        return 0

assert_type(myfunc(), str)
assert_type(myfunc(arg=False), int)
assert_type(myfunc(arg=True), str)


def test_bool_type(bl: bool):
    assert_type(myfunc(arg=bl), Union[int, str])
#

thank you so much

quasi spindle
#

Is there a valid way according to mypy to define a non-hashable class?
If I add __hash__ = None, mypy complains and is unable to narrow down the typing:

from enum import EnumType
from typing import Any, Hashable, Final


class Default(metaclass=EnumType):
    __hash__ = None  # error: Incompatible types in assignment (expression has type "None", base class "object" defined the type as "Callable[[object], int]")  [assignment]
    token: Final = 0

    def __init__(self, *args: Any, **kwds: Any) -> None:
        pass


_default = Default.token
x: Hashable | Default = _default

if x is _default:
    reveal_type(x) # note: Revealed type is "Union[typing.Hashable, Default]"
rare scarab
#

Maybe if you do ```py
def hash(self):
raise TypeError

#

related? python/typeshed#6243

mighty lindenBOT
rare scarab
#

python/typeshed#2148
python/mypy#4266

mighty lindenBOT
rare scarab
#

all the unhashable types in typeshed have ```py
hash: None # type: ignore

#

python/typeshed#3219

mighty lindenBOT
quasi spindle
quasi spindle
cinder bone
quasi spindle
# cinder bone Maybe like add a `__eq__`?

mypy still wont narrow, same result:

from enum import EnumType
from typing import Any, Hashable, Final


class Default(metaclass=EnumType):
    __hash__: None  # type: ignore
    token: Final = 0

    def __init__(self, *args: Any, **kwds: Any) -> None:
        pass

    def __eq__(self, other: Any) -> bool:
        return isinstance(other, type(self))


_default = Default.token
x: Hashable | Default = _default

if x is _default:
    reveal_type(x)  # note: Revealed type is "Union[typing.Hashable, Default]"
rare scarab
#

Why should this not be hashable?

quasi spindle
#

It's the default value when no hashable is supplied in a function. I would've used None normally, but it is Hashable.

cinder bone
#

That seems like a mypy problem

from typing import Hashable

class Foo:
    def __eq__(self, o):
        return self is o
        
x: Hashable = Foo()
hash(x)

Fails at runtime🤔

oblique urchin
#

hashability is difficult to model in the type system

cinder bone
#

Is it because object is Hashable?

oblique urchin
#

yes, that's a problem

jolly cipher
rough sluiceBOT
#

stdlib/typing.pyi lines 425 to 431

@runtime_checkable
class Hashable(Protocol, metaclass=ABCMeta):
    # TODO: This is special, in that a subclass of a hashable class may not be hashable
    #   (for example, list vs. object). It's not obvious how to represent this. This class
    #   is currently mostly useless for static checking.
    @abstractmethod
    def __hash__(self) -> int: ...```
jolly cipher
#

@oblique urchin I got slightly overwhelmed by a first issue I wanted to fix in mypy, so I can maybe try to fix a simpler one.

tested = (None, 0, 1)

reveal_type(list(filter(None, tested)))  # note: Revealed type is "builtins.list[builtins.int]"
reveal_type(list(filter(bool, tested)))  # note: Revealed type is "builtins.list[Union[builtins.int, None]]"

am I correct that mypy should display "builtins.list[builtins.int]", as in the filter(None, ...) case, to keep the same semantics?
I looked into https://github.com/python/mypy/issues/12682, the answer seems to be yes.

oblique urchin
jolly cipher
#

actually no, a listcomp seems easier

jolly cipher
#

although.

#

oh, i rely on bool

#

but that's essentially part of the problem, isn't it

#

i'd expect the outcomes of operator.truth() (i'll check that one now) and bool to match

oblique urchin
jolly cipher
#

ok, to mypy if operator.truth(x) and if x is not the same.

oblique urchin
#

another approach is to grab the typeshed stub for filter and simplify them as much as possible

jolly cipher
#

alright.

jolly cipher
oblique urchin
jolly cipher
#

i'll experiment with typeshed, will see where this gets me

agile rivet
#

Using Union instead of logical OR -- |, is good practice?
Example:

def fn(items: Union[str, list[str, EconItems], EconItems], ...)

def fn(items: str | list[str, EconItems] | EconItems, ...)
viscid spire
#

did you mean list[str | EconItems]?

viscid spire
# 
def fn(items: (str | list[str | EconItems] | EconItems), ...)
#

what is the return value of the function btw?

agile rivet
#

Return value is instance of class ItemObjects(BaseModel)

viscid spire
#

i c

#

so the return type doesn't depend on the type of items

rare scarab
#

Use Union for python < 3.10

trail kraken
#

You might get away with quoting "A | B" if you don't want to import typing. Also needed for list[C] for < 3.9. Won't work as type alias.

hidden geode
#

so I have a code like this:

type YEAR = int
type WEEK = int

type DATE = datetime | tuple[YEAR, WEEK]

now when I'll make a function or smth that takes an argument and type hint with DATE and then hover over that argument (in pycharm) it will "solve" it (look at screenshot). I can make YEAR and WEEK TypeVars but it seems like overcomplicating. Is there any other way?

image.png
trail kraken
#

Would NewType work?

hidden geode
#

I don't think that's what I'm looking for

#

maybe it's my editor. I want it to show that DATE is datetime | tuple[YEAR, WEEK]

hidden geode
trim tangle
#

e.g. DATE[int, bool] is a valid type, but DATE isn't

#

If you just want the "names" to propagate, then you will have to use NewType. Type alias names don't propagate like that

rare scarab
#

I think typing.NewType would be more appropriate

hidden geode
#

yea now that I've done it like this:

YEAR = NewType("YEAR", int)
WEEK = NewType("WEEK", int)
type DATE = datetime | tuple[YEAR, WEEK]

it works

trim tangle
#

I remember that project where a coworker annotated every function without a return statement with -> NoReturn... They did not use mypy, just whatever hints PyCharm gives you. 😬

hidden geode
#

also I have a question. Why is most stuff in typing library depreceated? Imo it seems to belong there

trim tangle
#

Are you referring to the PEP 585 deprecations?

rare scarab
#

only the collections are deprecated.

#

and Union/Optional depending on how you look at it

trim tangle
#

Well, stuff like List and Dict is deprecated because you can now use list and dict directly. Stuff like Iterable is deprecated because now the stuff in collections.abc is subcriptable (it existed before PEP 484, but it did not support the angle brackets syntax)

#

It is "formally deprecated" but I don't expect this stuff actually being removed in the next few years (besides stuff that's already scheduled for removal like typing.re)

hidden geode
#

so collections.abc is the primary code and typing was made just for the typing features that collections.abc didn't have

trail kraken
#

Ah. Didn't know there is no time frame for removal yet.

rare scarab
#

collections.abc is just mixins

trim tangle
rare scarab
#

typing_extensions even has @deprecated

#

Don't believe indently's recent video about deprecating typing. It's misleading

trail kraken
#

There are videos on type hints?

trim tangle
#

Btw, what's the policy on removing stuff from typing-extensions? It currently has Doc. Doc is just in a draft pep for now. typing-extensions, so... it will stay there forever? Even if it's rejected?

rare scarab
#

tiangolo would be devestated

#

and several versions of fastapi would break

trim tangle
#

Yeah, that's what I meant. People now depend on this stuff. And it's supposed to be experimental.

#

Now it's not experimental, it's just... there

rare scarab
#

mark Doc with @deprecated

trim tangle
#

If people in the wild will use Doc like it is used in FastAPI, I will be somewhat devastated.

#

It has a very low signal to noise ratio

#

but, well, whatever makes my future job more secure 🙂

hidden geode
#

also is this fine?

NONE = Literal[0]
PLANNED = Literal[1]
CONFIRMED = Literal[2]
RECEIVED = Literal[3]
STATUS = NewType(
    "STATUS",
    Literal[NONE, PLANNED, CONFIRMED, RECEIVED]
)
trim tangle
rough sluiceBOT
hidden geode
#

thx

trim tangle
#

Or you can use Status = Literal["none", "planned", "confirmed", "received"] if you don't need the numbers

hidden geode
#

I mean it's more efficient to store ints than strings

rare scarab
#

Then use an enum

trim tangle
hidden geode
#

in database

#

but still even if I were to convert ints from database to strings then it would use much storage on client side

trim tangle
#

Well, there are many tradeoffs here...

#

First, it's probably a good idea to decouple how you're representing the statuses in your program from how they're stored. I'd have something like ```py
def status_from_db(int) -> Status: ...
def status_to_db(Status) -> int: ...

#

Second, if you use plain integers for an "enum" on the database side, you're going to have trouble with e.g. analytical queries. You'll write a query and get something like status = 25 in return. That will force you to consult something that maps these numbers to human-readable statuses. (if you can find that mapping!)

#

It might be more appropriate something like Postgres's enum

hidden geode
#

I mean for now I use placeholder data with no db but Ik that storing Strings in database is costly

trim tangle
#

Enums in postgres take up 4 bytes each

hidden geode
#

hmm yea it seems like a good idea to have Enums in database

#

thx

hidden geode
#

Is there a type hint for something Unique?

#

like you know in databases some values must be unique.

jolly cipher
#

can you provide a specific case where you'd like to have such an annotation?

#

some container types imply unique members

#

like set[...] for instance

hidden geode
# 
@dataclass
class Schedule:
    id: Unique[int]
jolly cipher
#

hm, this doesn't make sense in the type system, but you could store an info that you'd like this unique to some runtime processor that examines the ID against other known IDs

restive rapids
jolly cipher
#

like this ```py
from typing import Annotated

@dataclass
class Schedule:
id: Annotated[int, Unique]

where `Unique` is your own, arbitrary runtime object presence of which in annotated type metadata signifies being "unique" (unique in known set of IDs that are never known at type checking time)
hidden geode
jolly cipher
#

you can't, because type system doesn't know what are known values to compare against

hidden geode
#

I mean it's just a hint for the programmer right?

jolly cipher
#

a hint for the programmer and potentially a runtime backend that could parse annotations of Schedule (typing.get_type_hints(Schedule)) and ensure that Unique applies at runtime

#

the same way you can specify validators via Annotated[] in pydantic models

#

but your Unique object would most likely require the context of what are the rest of the Schedule objects to check

#

building on that idea, you can make it a function

something like

def unique(new_id, ctx):
    assert new_id not in ctx.id_index

type Unique[T] = Annotated[T, unique]  # consider *Ts to allow other metadata

@dataclass
class Schedule:
    id: Unique[int]

and then your logic could examine id type annotation's __value__

#

and knowing the __metadata__ of it, pass proper args to the function found in Annotated type metadata

hidden geode
#

also I have other question tho I don't know if it's for this channel.

Can you add docstrings with parameters for dataclassses (eg:

@dataclass
class Foo:
  """
  :param bar: xyz
  """
  bar: int

cause for me in pycharm it doesn't seem to parse it (or fails at parsing it)

restive rapids
hidden geode
#

also yea I wanted to ask what does the new type keyword do? Does it create a TypeAlias?

restive rapids
#

in type U = ... U is an instance of TypeAliasType yeah

jolly cipher
#

the question was about a type hint

restive rapids
#

you're suggesting to implement runtime usage of type annotations

jolly cipher
#

because the question was about a type annotation

jolly cipher
#

note: typing.Annotated is 3.9+, use typing_extensions.Annotated for 3.8

jolly cipher
#

or, well

#

just a simple around-constructor validation

crimson inlet
# 
from __future__ import annotations

from collections.abc import Callable
import functools
import typing as t
import libcst as cst

_T = t.TypeVar("_T")
_OriginalNodeT = t.TypeVar("_OriginalNodeT", bound=cst.CSTNode)
_UpdatedNodeT = t.TypeVar("_UpdatedNodeT", bound=cst.CSTNode)
_ClassT = t.TypeVar("_ClassT", bound=A)

class A(t.Protocol):
    unsafe: bool

def unsafe(
    func: Callable[[_ClassT, _OriginalNodeT, _UpdatedNodeT], _T]
) -> Callable[[_ClassT, _OriginalNodeT, _UpdatedNodeT], t.Union[_T, _UpdatedNodeT]]:
    @functools.wraps(func)
    def inner(
        self: _ClassT, original_node: _OriginalNodeT, updated_node: _UpdatedNodeT
    ) -> t.Union[_T, _UpdatedNodeT]:
        if self.unsafe is False:
            return updated_node
        
        return func(self, original_node, updated_node)

    return inner

class Lol:
    def do_something(self, original_node: cst.CSTNode, updated_node: cst.CSTNode) -> cst.CSTNode:
        ...

class B(A, Lol):
    def __init__(self):
        self.unsafe = False

    @unsafe
    def do_something(self, original_node: cst.CSTNode, updated_node: cst.CSTNode) -> cst.TypeAlias:
        ...``` Hi I got the following code and pyright is saying: ```
Method "do_something" overrides class "Lol" in an incompatible manner
  Parameter 2 mismatch: base parameter "original_node" is keyword parameter, override parameter is position-only
  Parameter 3 mismatch: base parameter "updated_node" is keyword parameter, override parameter is position-only```
Could anyone explain why?
oblique urchin
crimson inlet
#

so Callable makes it positional only?

brazen jolt
#

yeah, when you use a decorator that returns a Callable with explicitly declared parameters, it considers those parameters as keyword-only, the type-checker isn't smart enough to understand that they will be the same here. You can avoid the error by defining the original Lol.do_something use positional-only, or potentially, with the use of ParamSpec

oblique urchin
brazen jolt
#

ah yeah

crimson inlet
#

Also: Isn't this a pyright bug then? mypy seems to resolve the override types just fine and I wouldn't say this is expected behaviour either, having to work around this "limitation" seems kind of annoying

brazen jolt
#

type-wise, the function would still match the hinted return type of that callable

#

but if someone used keyword args when caling the func: do_something(original_node=x, updated_node=y), it would fail

#

if mypy is fine with this, I'd consider it a bug on their side (though maybe it's just an optional setting that you need to turn on if you want to enforce it)

crimson inlet
brazen jolt
crimson inlet
#

Okay yeah that definitely makes sense

brazen jolt
#

Here's a few examples of what you can do, but again, preserving everything without doing positional-only isn't really viable afaik

from collections.abc import Callable
from typing import Concatenate, ParamSpec, Protocol, TypeVar

class A(Protocol):
    unsafe: bool

class Node: ...

T_A = TypeVar("T_A", bound=A)
T_Node1 = TypeVar("T_Node1", bound=Node)
T_Node2 = TypeVar("T_Node2", bound=Node)
T_Node3 = TypeVar("T_Node3", bound=Node)

P = ParamSpec("P")

def bad_deco(func: Callable[[T_A, T_Node1, T_Node2], T_Node3]) -> Callable[[T_A, T_Node1, T_Node2], T_Node3]:
    def inner(self: T_A, x: T_Node1, y: T_Node2) -> T_Node3: ...
    return inner


def good_deco(func: Callable[P, T_Node1]) -> Callable[P, T_Node1]:
    def inner(*args: P.args, **kwargs: P.kwargs) -> T_Node1: ...
    return inner

def better_deco(func: Callable[Concatenate[T_A, P], T_Node1]) -> Callable[Concatenate[T_A, P], T_Node1]:
    def inner(self: T_A, *args: P.args, **kwargs: P.kwargs) -> T_Node1: ...
    return inner

class Foo(A):
    def __init__(self, unsafe: bool) -> None:
        self.unsafe = unsafe

    def test1(self, orig: Node, updated: Node) -> Node: ...
    def test2(self, orig: Node, updated: Node, /) -> Node: ...
    def test3(self, orig: Node, updated: Node) -> Node: ...
    def test4(self, orig: Node, updated: Node) -> Node: ...

class Bar(Foo):
    @bad_deco
    def test1(self, orig: Node, updated: Node) -> Node: ...  # incompatible override
    @bad_deco
    def test2(self, orig: Node, updated: Node, /) -> Node: ...  # this works
    @good_deco
    def test3(self, orig: Node, updated: Node) -> Node: ...  # this works
    @better_deco
    def test4(self, orig: Node, updated: Node) -> Node: ...  # works
#

you can go with better_deco and do a runtime check to enforce that the arguments are Nodes

#

or just do positional only

crimson inlet
#

Or I just ignore the report i guess? I mean in practice it will never fail right?

brazen jolt
#

it's true, in your case, inner does use the correct parameter names, so you can ignore it

crimson inlet
#

Alright I think I will go for that, thanks for your help and making me understand, appreciate it 👌

brazen jolt
#

hm, although testing it out, ignoring might lead to issues too

#

since pyright now assumes that the test1 is positional-only, you just ignored the fact that the override was incompatible

#

it would work with Foo though

#

@crimson inlet

#

imo unless you really need to preserve the arg names there, just go positional only

#

if you do need them, best you can do is this I think: ```python
def better_deco(func: Callable[Concatenate[T_A, P], T_Node1]) -> Callable[Concatenate[T_A, P], T_Node1 | Node]:
def inner(self: T_A, *args: P.args, **kwargs: P.kwargs) -> T_Node1 | Node:
# If you want, you can also ensure these are nodes on runtime instead of the casts
orig = cast(Node, kwargs.get("orig") or args[0])
updated = cast(Node, kwargs.get("updated") or args[1])

    if self.unsafe:
        return updated

    return func(self, *args, **kwargs)

return inner
crimson inlet
brazen jolt
#

but yeah, it's not ideal, since it won't properly preserve that the updated node type can be the return type, rather, it just goes all the way down to | Node

#

so basically, it's just a hard-coded -> Node return

#

I'd only do this if you really can't do positional-only

crimson inlet
#

I really cant haha because in reality the parent class is a third party implementation

#

and to be fair I dont think the decorated function will be ever called directly from myself (only from the third party lib) so even then ignoring the report should be fine and in practice the return type using your runtime check should be preserved too since _T will always include the type of the updated_node which is why I could just return only _T instead of a Union

vivid ore
#

Is there a way to model that type variable is both contra and covariant?

#

For instance, in this code

from typing import TypeVar, Generic
from collections.abc import Callable


T_contra = TypeVar("T_contra", contravariant=True)
U = TypeVar("U")
V_co = TypeVar("V_co", covariant=True)


class Deposit(Generic[T_contra, U, V_co]):
    pre_transform: Callable[[T_contra], U]
    internal_value: U
    post_transform: Callable[[U], V_co]
    
    def __init__(
        self,
        pre_transform: Callable[[T_contra], U]
        initial_value: T_contra
        post_transform: Callable[[U], V_co]
    ) -> None:
        self.pre_transform = pre_transform
        self.post_transform = post_transform
        self.internal_value = pre_transform(initial_value)
    
    def withdraw(self) -> V_co:
        return self.post_transform(self.internal_value)
    
    def deposit(self, value: T_contra) -> None:
        self.internal_value = self.pre_transform(value)
#

making U invariant is not really correct

trim tangle
vivid ore
#

after all, a Deposit[T, U1, V] is completely indistinguishable from a Deposit[T, U2, V] regardless of U1 and U2’s relationships to each other

vivid ore
vivid ore
trim tangle
#

Since U is not "observed" from the outside, it might be better to do something like ```py
class _Deposit(Generic[T_contra, U, V_co]): ...
def init(
self,
pre_transform: Callable[[T_contra], U]
initial_value: T_contra
post_transform: Callable[[U], V_co]
) -> None:
self._pre_transform = pre_transform
self._post_transform = post_transform
self._internal_value = pre_transform(initial_value)

def withdraw(self) -> V_co:
    return self._post_transform(self._internal_value)

def deposit(self, value: T_contra) -> None:
    self._internal_value = self.pre_transform(value)

class Deposit(Protocol[T_contra, V_co]):
def withdraw(self) -> V_co: ...
def deposit(self, value: T_contra) -> None: ...

def make_deposit(
pre_transform: Callable[[T_contra], U],
initial_value: T_contra,
post_transform: Callable[[U], V_co],
) -> Deposit[T_contra, V_co]:
return _Deposit(pre_transform, initial_value, post_transform)

vivid ore
#

that makes sense

trim tangle
#

This kind of makes it apparent that classes (the concept) conflate the interface of a thing with its implementation a bit 🙂

vivid ore
#

Is there a way to make a type variable that’s constrained to all possible instantiations of a generic type?

#

So for instance

# 
T = TypeVar("T")

class A: pass

class B(Generic[T]): pass

U = TypeVar("U", A, B)
#

is not what I want

#

I want to be able to specifically say that U can take on all possible B[…]s

undone saffron
#

bound=B

vivid ore
#

this is not what this does, as can be seen here

# 
def id(x: U) -> U:
    if isinstance(x, B):
        reveal_type(x)
    return x
#

this shows that here x is of type B[Any]

#

I would like it to be B[object]

#

hm I guess this only works if T is covariant

#

which it isn’t here

fierce ridge
# vivid ore I would like it to be `B[object]`

I tried a couple of different things like U = TypeVar("U", bound=Union[A, B[object]]) but Mypy kept emitting B[Any] -- it might be treating object and Any as synonymous top types here, although clearly they are not synonymous. Maybe this is a feature request

vivid ore
#

ok

fierce ridge
hidden geode
#

Hi. How can I add type hints to a function that returns a pd.DataFrame?

#

with specific column names?

grave fjord
hidden geode
#

I'll look into it thanks

vivid ore
#

Is there a way to be generic over TypedDicts?

#

Neither of

D = TypeVar("D", bound=dict)
class A(Generic[D]):
    def f(self, **kwargs: Unpack[D]) -> None: pass

or

D = TypeVar("D", bound=TypedDict)
class A(Generic[D]):
    def f(self, **kwargs: Unpack[D]) -> None: pass

work

# 
class A[**D]:
    def f(self, **kwargs: Unpack[D]) -> None: pass

also doesn’t work

oblique urchin
vivid ore
oblique urchin
#

class X[MappingT: Mapping[str, object]]:

vivid ore
#

Thanks

#

oh

#

fuck me

#

I’m not smart

#

thanks

vivid ore
#

Is there a way to do something like Union[*Ts]?

#

i.e. if Ts represents T1, T2, T3, ..., to write T1 | T2 | T3 | ...

oblique urchin
vivid ore
oblique urchin
vivid ore
#

ok

jade viper
#

Hey guys, any idea on why my linter is complaining that Type "dict[str | float, str | float]" is not assignable to return type "dict[str, float]" on:

@staticmethod
def flatten_prediction(prediction: dict[str, Union[str, list[str], list[float]]]) -> dict[str, float]:
    return {label: score for label, score in zip(prediction['labels'], prediction['scores'])}
#

Or rather, how should I typehint this correctly? prediction is something like:

{'sequence': 'Carlson\'s Choke Tubes 87003 Replacement Barrel  12 Gauge 28" Vent Rib, Matte Blued Stainless Steel, Fiber Optic Sight, Fits Remington 870 -- FIREARM ACCESSORIES -- EXTRA BARRELS',
 'labels': ['Gun Parts > Rifle Parts',
  'Gun Parts > Pistol Parts',
  'Gun Parts > Other Gun Accessories & Parts',
  'Gun Parts > Shotgun Parts',
  'Scopes, Sights & Optics > Scope Accessories & Scope Parts',
  'Scopes, Sights & Optics > Gun Scopes',
  'Gun Parts > 1911 Parts',
  'Scopes, Sights & Optics > Gun Sights',
  'Scopes, Sights & Optics > Laser Sights',
  'Gun Parts > AK47 Parts',
  'Scopes, Sights & Optics > Red Dot Sights'],
 'scores': [0.2194734811782837,
  0.17365244030952454,
  0.13916060328483582,
  0.12419769912958145,
  0.08215249329805374,
  0.07504958659410477,
  0.0590294748544693,
  0.05863707512617111,
  0.028359295800328255,
  0.023273425176739693,
  0.017014361917972565]}
#

Should I create a TypedDict?

cinder bone
#

Yes

# 
class MyDict(TypedDict):
    sequence: str
    labels: Sequence[str]
    scores: Sequence[float]
supple agate
#

Type hinting

#

This is all pretty good information

#

keep it pushing everyone

fierce ridge
buoyant ridge
#

am I correct in saying that for overloads (considering only overloads with the same parameters but different types) vs generic functions:

  • generic functions preserve a specific relationship across any number of parameters and the return type e.g. (x: T) -> T
  • overloads (among other functions) preserve an arbitrary mapping between the parameters and the return type (x: A) -> B
    but overloads don't make any guarantees for the mapping across parameters:
    (unless i've done something wrong)
@typing.overload
def foo(x: int, y: str) -> str: ...
@typing.overload
def foo(x: str, y: int) -> int: ...
def foo(x: int | str, y: int | str):
  if isinstance(x, int):
    # logically we know y is str, but an overload doesn't give us this type guard
    typeof(y) # int | str

Something like this would require like... a multiple dispatch? but that's not actually type hinted in python's stdlib

trim tangle
#

There's

#

!d functools.singledispatch

rough sluiceBOT
#
functools.singledispatch 

@functools.singledispatch```
Transform a function into a [single\-dispatch](https://docs.python.org/3/glossary.html#term-single-dispatch) [generic function](https://docs.python.org/3/glossary.html#term-generic-function).

To define a generic function, decorate it with the `@singledispatch` decorator. When defining a function using `@singledispatch`, note that the dispatch happens on the type of the first argument:

```py
>>> from functools import singledispatch
>>> @singledispatch
... def fun(arg, verbose=False):
...     if verbose:
...         print("Let me just say,", end=" ")
...     print(arg)
```...
trim tangle
#

But I don't think type checkers understand it at all

buoyant ridge
#

yeah, they don't seem to

buoyant ridge
#

i suppose i should just be using different functions really if i find myself wanting to say 'x is int if y is str'

trim tangle
#

Yes, if you have some variant of py def handle_foo(foo: int | str) -> None: if isinstance(foo, int): ... else: ... and the caller always knows whether they have an int or a str, it's often better to make two functions

buoyant ridge
#

thanks! (actually, i see you providing a lot of useful advice here pretty often; you're a legend dude)

trim tangle
#

no I'm just terminally online

buoyant ridge
#

and terminally helpful! i always think ur answers are rly like friendly and insightful

undone saffron
warm zodiac
#

Hi, could I ask here for advice, please?

 def calculate_result_shape(self) -> None:
        """
        Calculate the shape of the Plugin results.
        """
        _shape = self._config.get("input_shape", None)
        
        if not isinstance(_shape, tuple):
            raise UserConfigError(
                f'Cannot calculate the result shape for the "{self.plugin_name}" '
                'plugin because the input shape is unknown or invalid.'
            )
            
        # currently an upper boundary for the expected shape of the result, affects only second dimension
        self._config["result_shape"] = (3, int(np.ceil(_shape[0] / 2 + 1)))

Do you think this is correct to write None as type hint for the output?
` self._config["result_shape"] is a tuple[int, int], but technically the function does not retunr

trim tangle
#

side effects such as raising exceptions, sleeping or mutating objects are not captured in the return type

#

(unless you always raise an exception or enter an infinite loop, in which case Never is used)

warm zodiac
trim tangle
#

Yes, -> None is correct here

warm zodiac
#

Thank you very much!

jolly cipher
trim tangle
#

IIRC Never was added because the name NoReturn confused people

oblique urchin
trim tangle
#

yeah, it is also strange to have list[NoReturn]

warm zodiac
#

@jolly cipher I understood NoReturn should be used if the function "return" raise Error, but in my example this is not the case

jolly cipher
#

but you can also do

# 
def panic() -> NoReturn:
    sys.exit(1)
jolly cipher
# jolly cipher `Never` or `NoReturn` -- as it's the same, "bottom" type. a myth is that there's...

using Never can be problematic on Python <3.11.
if you don't want to use NoReturn and you're using mypy, you're fine with just relying on typeshed and adding ```py
from future import annotations
from typing import TYPE_CHECKING

if TYPE_CHECKING:
from typing_extensions import Never # doesn't have to exist at runtime (doesn't have to be installed)!

(`typing_extensions` has a stub in typeshed stdlib stubs, so by type checkers it should generally be treated like a standard library (again, except pyright))
but this won't work out well on pyright since they have an error-prone approach to what is and isn't runtime afaik
#

so the simplest approach is to pull typing-extensions and add it as a dependency, also in case you need it at runtime (TYPE_CHECKING is False, things conditional to it won't execute and serve only as a typing-related only code)

stiff acorn
#

I found this odd behavior in mypy's type inference

# 
def test(backup_name: str | None = None) -> None:
    default_backup = return_default_backup()

    reveal_type(backup_name)
    reveal_type(default_backup)

    a = backup_name or default_backup
    reveal_type(a)

    b = default_backup if backup_name is None else backup_name
    reveal_type(b)

    c = backup_name if backup_name else default_backup
    reveal_type(c)

    d = backup_name if backup_name is not None else default_backup
    reveal_type(d)
#

only the a correctly evaluates to a union of str and backup object

#

b, c, and d end up as objects

#

mypy

main.py:15: note: Revealed type is "Union[builtins.str, None]"
main.py:16: note: Revealed type is "__main__.BackupFile"
main.py:19: note: Revealed type is "Union[builtins.str, __main__.BackupFile]"
main.py:22: note: Revealed type is "builtins.object"
main.py:25: note: Revealed type is "builtins.object"
main.py:28: note: Revealed type is "builtins.object"
#

pyright

Type of "backup_name" is "str | None"
Type of "default_backup" is "BackupFile"
Type of "a" is "str | BackupFile"
Type of "b" is "BackupFile | str"
Type of "c" is "str | BackupFile"
Type of "d" is "str | BackupFile"
jolly cipher
oblique urchin
jolly cipher
#

must be hard as hell.
or, in other words, costly.

#

i think it could be a great contrib.

trim tangle
jolly cipher
soft matrix
#

Would it be that hard to just cherry pick from them the commit?

#

Actually that probably has licensing issues

oblique urchin
#

I don't think making the change itself is hard, it's more about making sure it doesn't break existing mypy users

jade viper
#

Is there a tool that checks that a codebase has a certain amount of typehint coverage?

soft matrix
#

pyright has a flag for it

#

--verifytypes

rustic gull
#

Should u type annotate kwargs

viscid spire
#

Yep (depends on how is needed tho)

jade viper
soft matrix
#

no idea probably not

stiff acorn
#

Currently I just leave them as **kwargs: Any

fierce ridge
#

i've been complaining about it for years. it's also important for things like overriding methods where you aren't changing the signature, but need to copy-paste the whole thing anyway including all overloads

stiff acorn
#

Yea, a bunch of my libraries would also benefit from this but currently they just gotta cope with **kwargs: Any

stiff acorn
#

I get this error with mypy

# 
Skipping analyzing "pocketbase.utils": module is installed, but missing library stubs or py.typed markerMypyimport-untyped
#

I've already excluded the parent pocketbase package

[[tool.mypy.overrides]]
module = ["pocketbase"]
ignore_missing_imports = true
#

from pocketbase import xx works fine

#

from pocketbase.yy import xx raises the issue

#

how do i exclude all of pocketbase?

fierce ridge
#

basically wildcard syntax for module name components

stiff acorn
wicked scarab
# 
    def __init__(self, matrix: MatrixBase | list[float | int]):
        if not all(isinstance(row, list) for row in matrix):
            self.matrix: MatrixBase = [matrix]
        else:
            self.matrix: MatrixBase = matrix
``` with pyright, is there a way to do this type narrowing? Pyright still assume that matrix is `list[list[int | float]] | list[float | int]` for both conditions.

`MatrixBase` is a type alias of `list[list[int | float]]`.
trim tangle
wicked scarab
# 
        if matrix and not all(isinstance(row, list) for row in matrix):
            self.matrix: MatrixBase = [matrix]
        else:
            self.matrix: MatrixBase = matrix
``` I've changed the code a little bit
tranquil turtle
trim tangle
wicked scarab
wicked scarab
trim tangle
#

If this is a new interface, I would just do ```py
def init(self, matrix: MatrixBase) -> None:
self.matrix = matrix

If someone has a single row, they can wrap it in a list themselves
wicked scarab
#

okay, that seems cleaner

wicked scarab
trim tangle
#

This has to do with variance, you can read about it here or here or here

wicked scarab
#

I've tried using Sequence and I don't think it can work, since I need to use list's methods

#

I've also tried using TypeVar, like

T = TypeVar("T", bound=int | float)
MatrixBase = list[list[T]]
``` but T will be `Unknown` everywhere
trim tangle
#

You can annotate it manually: a: list[list[float]] = [[1.0]]

#

(btw, float | int is the same as float, but that's another can of worms...)

wicked scarab
#

is there a way to solve it from the class side?

trim tangle
#

int is considered to be assignable to float, so it is legal to do e.g. foo: float = 42. So you can replace float | int with float

wicked scarab
#

Argument of type "list[list[int]]" cannot be assigned to parameter "matrix" of type "MatrixBase" in function "__init__"   "list[list[int]]" is incompatible with "list[list[float]]" I've changed MatrixBase to be list[list[float]]

trim tangle
#

yeah, this still has the same variance issue

#

Because again, the class is allowed to do matrix[0].append(3.14), which means the externally supplied list[list[int]] not contains a float.

wicked scarab
#

hmm I see

trim tangle
#

If you want the caller to be able to provide a list[list[int]], you can use collections.abc.Iterable and accept Iterable[Iterable[float]]

#

Then construct the list yourself. (essentially making a copy)

wicked scarab
#

ohh, I got it

#

that's what sympy also did, making a copy

buoyant ridge
tiny ingot
#

What is the point of creating classes? Isnt a dictionary the same thing?

rare scarab
#

classes are the template for creating objects.

old dagger
slender iris
#

Coming from a background of Coding in JS/TS this would make sense. When I write TS it’s interfaces type JSON objects aka dictionaries

tiny ingot
slender iris
#

Technically your correct @tiny ingot

#

Raymond hettinger said something along the lines of, Python class are just dictionaries of attributes and functions, modules are just dictionaries of classes and variables

#

So if your asking class vs dict, what to use? I would answer with the question, Which is easier to read and reason about? Because the typing module offers TypeDict too. It would come down to preference.

viscid spire
slender iris
# viscid spire JS has classes tho

For sure. I didnt mean they DONT have classes, Im just saying that personally, I dont use them like Python. Most cases a JSON object with typing ie. TS interface does what I needed it to do

viscid spire
#

what do you mean a JSON object

#

JSON means Javascript Object Notation

#

it's a string representing a JS object

slender iris
#

Context. In many cases, talking in the context of JS dev, you can say a JSON object and it would be undestood that its a dictionary or mapping. But yeah, in Python JSON is a string. For example in JS, theres JSON.stringify and JSON.parse.

viscid spire
#

a "JSON object" would be an object loaded from JSON

slender iris
#

Taken from MDN docs. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse
| The JSON.parse() static method parses a JSON string, constructing the JavaScript value or object

MDN Web Docs
JSON.parse() - JavaScript | MDN

The JSON.parse() static method parses a JSON string, constructing the JavaScript value or object described by the string. An optional reviver function can be provided to perform a transformation on the resulting object before it is returned.

viscid spire
#

yeah it makes the Javascript object...

slender iris
viscid spire
slender iris
# viscid spire I honestly have no clue what you are trying to say here

Do you write React or any JS. Im not trying to be rude, just asking? Many times when writing React (ie JS based context) or any front-end situation that uses Javascript, you can say I want to use a JSON object for "X" and its understood that the expect object is mapping of some sort. Thats the context I am referring to. This is differeunt from Python and saying I want to return JSON

viscid spire
#

I don't understand what you mean when you say

JSON object for "X" and its understood that the expect object is mapping of some sort.

viscid spire
slender iris
viscid spire
#

I don't undestand what the situation you are referring to is

#

if you can write some code then maybe I can understand

slender iris
# 
const json = '{"result":true, "count":42}';
const obj = JSON.parse(json);

Just saying obj is an object or JSON object thats all

viscid spire
#

right so you have a JSON string, then you parse it to a Javascript object

slender iris
viscid spire
#

JSON object would refer to the string, or to the JSON object itself in JS. But not what it parses

slender iris
viscid spire
#

of course it's semantic??

#

relating to meaning in language or logic.

slender iris
#

Cool at least we can agree, semantically, I mean an object in JS

viscid spire
#

thus the discussion is semantic

#

I don't think you actually agree with what I'm saying here, so idk why you thumbs-up'd

slender iris
#

JSON is a representation of a javascript object

#

Your right

viscid spire
#

mhm

slender iris
#

Internal JS, like using the object in JS code is JS object, when you put the object in a file {"some":"value"} its JSON, JSON requires quotes around keys, JS object doesnt. So my use of JSON object even in an JS context was incorrect

rare scarab
#

just remember back in the day, javascript didn't have a standard json parser. You had to use eval("{}")

#

some apis took advantage of this by wrapping the response in a callback function. callback({})

#

^ that was part of the full response body

jolly cipher
#

this entire conversation about classes isn't on topic for this channel

stiff acorn
#

I have something along the lines of this:

class AsyncClient:
    def __init__(self, base_url: str = "https://example.com/", cache: bool = True, **kwargs: Any) -> None:
        self._base_url = base_url
        self._cache = cache
        self._kwargs = kwargs
        self._extensions = {"force_cache": self._cache, "cache_disabled": not self._cache}
        self._storage = AsyncFileStorage(base_path=get_user_cache_path())

    async def get(self, url: str) -> Page:
        async with AsyncCacheClient(storage=self._storage, **self._kwargs) as client:
            page = await client.get(url, extensions=self._extensions)
            return Page(**parse(page))

    async def search(self, query: str) -> AsyncGenerator[Page]:
        async with AsyncCacheClient(storage=self._storage, **self._kwargs) as client:
            results = ...
            for link in results:
                yield await self.get(link)

and similar for sync:

class Client:
    def __init__(self, base_url: str = "https://example.com/", cache: bool = True, **kwargs: Any) -> None:
        self._base_url = base_url
        self._cache = cache
        self._kwargs = kwargs
        self._extensions = {"force_cache": self._cache, "cache_disabled": not self._cache}
        self._storage = FileStorage(base_path=get_user_cache_path())

    def get(self, url: str) -> Page:
       with CacheClient(storage=self._storage, **self._kwargs) as client:
            page = await client.get(url, extensions=self._extensions)
            return Page(**parse(page))

    def search(self, query: str) -> Generator[Page]:
        with CacheClient(storage=self._storage, **self._kwargs) as client:
            results = ...
            for link in results:
                yield await self.get(link)

@autumn glen already explained the answer for sync (i.e it should be Iterable[Page]), so now I'm extending this question to the async part too. Can I simply do AsyncIterable or is there more to it?

soft matrix
#

I'd say it returns a Generator and AsyncGenerator but yes doing Iterable works

viscid spire
#

I would prefer Iterator and AsyncIterator

stiff acorn
#

I've gotten mixed answers on this, so how do you decide which one to pick?

viscid spire
#

almost never use Generator

#

use Iterator when you can do next(obj) directly

#

otherwise use Iterable

#

for a generator function you are generally gonna want Iterator

#

also Generator takes 3 type args, of which two type something you likely aren't using

grave fjord
viscid spire
#

oh AsyncGenerators are closeable?

#

like context managers?

viscid spire
#

looks like AsyncGenerator[YieldType, SendType] is a bit different. I will have to look into that more

trim tangle
#

but... now that I think about it, that seems wrong, and for normal generators as well.

grave fjord
#

You get a warning on trio if you try

trim tangle
# grave fjord No it doesn't really work

Suppose that you do py def extract_things(extractor, tag, my_id): extractor.send_audit_event("start_extracting", {"service_id": my_id}) try: while True: thing = extractor.fetch(tag, {"service_id": my_id)) if thing is None: break yield thing finally: extractor.send_audit_event("done_extracting", {"service_id": my_id}) and then py def print_5_things(hub, tag, my_id): with hub.connect_extractor(service_id=my_id) as extractor: for _, thing in zip(range(5), extract_things(extractor, tag, my_id)): print("Got thing:", thing) # 'here' At 'here', the extractor is gone, but the generator might still be alive. So when the generator is garbage collected, it will try sending an audit event to an invalid extractor...

#

so I don't understand why it's not a problem with sync generators

#

Does it just happen to work in many cases because of refcounting?

#

!e

def uhh():
    try:
        yield 1; yield 2; yield 3; yield 4; yield 5
    finally:
        print("It is time...")

def main():
    print("starting")
    for x in uhh():
        if x == 3:
            print("3 is too much for me")
            break
    print("for loop finished")

main()
rough sluiceBOT
trim tangle
#

On PyPy, It is time... is not printed

grave fjord
stiff acorn
viscid spire
#

allows next()

grave fjord
#

I'd still be tempted to use Generator anyways

viscid spire
#

Maybe for consistency

#

(If you have both sync and async)

#

You will have to supply all 3 type args tho

cinder bone
#

I mean if it ends up being Generator[T, None, None] you might as well use Iterable[T]

stiff acorn
stiff acorn
viscid spire
#

That is typing extensions, which is not builtin

stiff acorn
#

yea, I'm using it from there to avoid writing None None

viscid spire
#

the one from typing and collections.abc do not have defaults, because type defaults don't exist until Python 3.13 (which is coming soon)

stiff acorn
#

excited for this in stdlib

viscid spire
#

But I don't think a change is coming to those types even with the defaults added to typevars and type args

stiff acorn
#

hopefully it does

#

Being able to just do Generator[T] is pretty neat

viscid spire
#

I guess it doesn't break backwards compatibility anyways because length of type args is not checked at runtime

#

I will continue to type my generator functions as Iterators either way, because I don't need the extra methods and such of a Generator

stiff acorn
#

do you write

        Yields
        -------
        AsyncGenerator[Object]

or

        Yields
        -------
        Object

in the docstrings

viscid spire
#

Object

#

the function returns the generator

#

and the generator yields the Object

stiff acorn
#

that makes sense

viscid spire
#

Although I don't write docstrings usually for types

#

that's what typehints are for

stiff acorn
#

anyway, thanks for the help. Appreciate it 😁

undone saffron
#

@oblique urchin while I won't be around to discuss further re: #internals-and-peps message in depth tonight, I don't want to leave it entirely unanswered with a suggestion of so many inaccuracies in the typeshed either.

The problem is that inaccuracies in one type often (but not always) lead to the type of something else being inaccurate, and there are a few issues with even the typing of object , MutableMapping, and Sequence (I hope you understand why I don't really want to rehash these when there are easier things to find potential agreement on for why not on types in docs)

I'm doing a ground-up version of the typeshed as more than just an exercise in frustration. Finding where we can't accurately express things right now can help in figuring out what additions to the type system would lead to better user outcomes that don't involve as many "tricks"