#type-hinting

1 messages · Page 22 of 1

oblique urchin
#

mypy doesn't support this syntax at all yet. To parse it, mypy needs to also run on Python 3.12, which the playground likely doesn't support

#

but even if it could parse it, it doesn't know what it means

simple field
simple field
oblique urchin
simple field
#

ohhh ok i see

dreamy cove
#

can i inform the type checker that a function parameter should not be mutated?

#

i'm not finding much from my searches

hallow flint
#

the python type system doesn’t really keep track of mutability. you can write your own protocol that only describe the immutable aspects of your object. for instance, the way typing.Sequence describes only the immutable parts of list

dreamy cove
#

unfortunate

trim tangle
steel edge
# 
T = TypeVar("T", covariant=True)

class A(Generic[T]):
    def foo(self, arg: list[T]) -> None:
        ...

class B(A[int]):
    def foo(self, arg: list[int]) -> None:
        ...

a: list[A[object]] = [B()]
a[0].foo([""])

Isn't this problematic ? both mypy and pyright don't report any errors

oblique urchin
trim tangle
#

yep

#

Actually the problem here is the foo method

#

If T is covariant, then you can't have a method like that

oblique urchin
soft matrix
#

@trim tangle rejoice

image.png
steel edge
#

confusion

class A:
    a: int
    
class B(A):
    a: bool

def foo(a: A) -> None:
    a.a = 1

B().a = 1 # not ok

foo(B()) # ok
soft matrix
tranquil turtle
# steel edge confusion ```python class A: a: int class B(A): a: bool def foo(a... 
class A:
    a: int
    
class B(A):
    a: bool
``` this ![this](https://cdn.discordapp.com/emojis/470903994118832130.webp?size=128 "this") is incorrect
`a: int` means that you get `int` when you read this and you can write any other `int` to this attr
`a: bool` still returns `int` on read (`bool` actually), but it disallows assignment of arbitrary ints to this attr
steel edge
#

I'm confused does that mean I shouldn't do a: bool or the typing system/type checkers are flawed

oblique urchin
#

Type checkers generally allow this for pragmatic reasons

hallow flint
#

except in protocols, where got an opportunity to be stricter 🙂

trim tangle
soft matrix
#

Don't think so

undone saffron
lyric ingot
#

Hello, does this list[str, str, ...] mean a list of at least 2 strings? or is it garbage?

soft matrix
#

It's garbage

#

Can't track list length in the type system

lyric ingot
#

Oh, so there's no way to say a list of 2 strings? I should use tuple instead?

soft matrix
#

Tuples yes

lyric ingot
#

and ... is equivalent to typing.Any?

soft matrix
#

No

#

They aren't the same thing

lyric ingot
#

Ok thanks!

jade viper
#

Hey guys, how do I type hint an exact return dict? Example:

{
    "url": "my_aws_url",
    "fields": {
        "key": "MANUAL_FILE/test.csv",
        "AWSAccessKeyId": "my_key",
        "policy": "my_policy",
        "signature": "my_signature"
    }
}
trim tangle
rough sluiceBOT
#
typing.TypedDict 

class typing.TypedDict(dict)```
Special construct to add type hints to a dictionary. At runtime it is a plain [`dict`](https://docs.python.org/3/library/stdtypes.html#dict).

`TypedDict` declares a dictionary type that expects all of its instances to have a certain set of keys, where each key is associated with a value of a consistent type. This expectation is not checked at runtime but is only enforced by type checkers. Usage...
jade viper
#

Great, thank you!:)

delicate rapids
#

Whats the difference between var: Tuple and var: tuple with a lowercase t?

tranquil turtle
#

Tuple is deprecated

rare scarab
#

Before 3.9, tuple wasn't generic

trim tangle
rare scarab
trim tangle
#

Not the first line, but I am happy to take some tuples in me

pulsar pilot
#

Is it possible to add a more strict type restriction on a single method of a class? For example:

class Foo(Generic[T, U]):
    def foo(self) -> T:
        ...

    def bar(self) -> U: # restrict this method
        ...

The Foo.bar method only run when Foo's T is another specific type or generic type. That is, for example, when Foo is Foo[int, U] then we can use bar, and when Foo's T is not int, then bar is not available.

#

I dont need runtime check, but static check for maybe IDE's to use.

hallow flint
#

maybe try def bar(self: Foo[int, U]) -> U: ...

fierce ridge
#

i'm working with a library that has a bunch of cdef classes with names like _Foo. how should i write type stubs for this? should i use the exact names from the pyx file, or should i do something like class Foo(Protocol)? or should i just write class Foo:?

#

(specifically the library is lupa if anyone is interested or knows of existing type stubs)

rare scarab
#

Class foo

#

Type the usage, not the impl

fierce ridge
rare scarab
#

Unless you want to use it to represent several unrelated types

fierce ridge
#

makes sense

fossil nest
#

I get a pylance error when defining the following paramter:
format: Type[Record | dict] = dict
Expected type arguments for generic class "dict"

Is there any way to resolve these issues without disabling type checking?

#

the format parameter expects a literal type such as dict which shouldn't require type arguments right?

simple field
#

Type[Record] | Type[dict]?

fossil nest
#

It still expects type arguments for generic class "dict"

#

Perhaps this is an issue with pylance's comprehension because Type[dict] should not result in an error

frigid jolt
fossil nest
#

damn, thanks

trim tangle
#

though you can ignore this error through the config

fossil nest
trim tangle
#

in general, if you have a generic type, you must fill in the parameters

#

Why do you need to accept a class as a parameter though?

#

99% of the time what you really want to accept is a function

fossil nest
#

I have a method that returns a list of objects, the type of object depends on the format parameter. It isn't very pythonic but heres what it looks like:

    def select(self, columns: tuple[str, ...] = (), /, *, format: Type[dict | Record] = dict, where: Callable[[Record], bool] = lambda record: True) -> None:
        # other stuff here ("rows" is currently a list of dictionaries)
        return rows if format == dict else [format(row) for row in rows]

But the format parameter is used to specify the return type of the objects

#

I would call it like

select(format=dict, ...)
select(format=Record, ...)
trim tangle
#

format: Callable[<YourParams>, dict] = lambda x: x

#

(or whatever default is appropriate)

#

or don't include this argument at all, keep the function interface more simple. The caller can call the function on their own

#

If it's some kind of database library, the factory is usually specified on the connection/connection pool

#

or just make Record good enough so that people don't have to fiddle with record factories like in sqlite3

steel edge
#

This shouldn't be allowed right ? Type checkers don't complain tho.

class A:
  def __lt__(self, other: Self) -> bool:
    ...

class B(A):
  # the new __lt__ is incompatible with the old one
  def __lt__(self, other: Self) -> bool:
    ...

I wonder what is the best way to do this.

trim tangle
#

ah hmmmmmm maybe

steel edge
# 
def foo(a: A) -> bool:
  return a < A() # ok

foo(B())

B() < A() # error
trim tangle
#

Yep

steel edge
#

Using Self in parameter types is useless anyways. You can't guarantee that self and the parameter are of the same type. So might as well do (self, other: A)

undone saffron
#

thats just not true. you can't guarantee it, but it's how you indicate to a consumer and their tooling (static analysis) the intended use.

undone saffron
#

(edited, wrong link on hand, may not find the right related issue right now)

#

Actually, there's an interpretation where it's fine as well, but it requires:

def __lt__(self: Self, other: Self) -> bool:
    ...
#

(pyright handles missing self annotation for self impliticly, dont remember if mypy does as well, if it does, your original form is fine (unless you're using an even less used option that also does not))

rare scarab
#

Self didn't always exist.

oblique urchin
#

thanks to typing_extensions, it now always exists

rare scarab
#

typing_extensions, time travelling since 2021

steel edge
steel edge
#

The main motivation to why Self was added in the first place is to use it in the return type. The same problem exists with TypeVar I think the method should behave like the function and allow all subclasses of A otherwise it doesn't follow the Liskov Substitution Principle.

T=TypeVar("T", bound="A")

class A:
    def foo(self: T, _: T): ...

def foo(a: T, _: T): ...
paper salmon
# steel edge The main motivation to why `Self` was added in the first place is to use it in t...

from PEP 673:

class Shape:
    def set_scale(self, scale: float) -> Shape:
        self.scale = scale
        return self

Shape().set_scale(0.5)  # => Shape``` However, when we call `set_scale` on a subclass of `Shape`, the type checker still infers the return type to be `Shape`.
... 
We propose a more intuitive and succinct way of expressing the above intention. We introduce a special form `Self` that stands for a type variable bound to the encapsulating class.

perhaps Self isnt the correct thing to be using there in the first place? If you typehint it as def __lt__(self, other: "A") -> bool: ..., the contravariance of other means subclasses have to support A if they want to comply with LSP

steel edge
paper salmon
#

ah right, im not sure how you used generics but as for the "best way", typehinting it with the least specialized class makes the most sense

stone harness
#

How could I make typing in this style work?

from typing import Any, Callable, ParamSpec, TypeVar


T = TypeVar('T')
P = ParamSpec('P')

def test_function(f: Callable[P, T]) -> Callable[P, T]:
    def test(*args: P.args, **kwargs: P.kwargs) -> T:
        return f(*args, **kwargs)
    return test

class Sla:
    a: int
    b: int


response = test_function(Sla)
#response(
#   a=1,
#   b=2
# )
#

I don't want to put the parameters in init

#

is there any way?

brisk hedge
#

use a dataclass or namedtuple

stone harness
#

I wanted to do something dynamic

brisk hedge
#

something dynamic but with static annotations?

stone harness
brisk hedge
#

you could look into @dataclass_transform if you're trying to implement an alternative dataclass

#

and want it to be typed

stone harness
#

but so far I haven't found anything that takes the class annotations and turns them into a type...

broken pollen
#

I've only recently started using mypy --strict and came across am [arg-type] warning when using list unpacking on a function call that I'm not sure how to fix (python 3.10).

def validate(cmin: int, cmax: int, char: str, word: str) -> bool:
    charcount = word.count(char)
    if cmin <= charcount <= cmax:
        return True
    return False

testlist = [1, 3, "a", "wasd"]
validate(*testlist)

Mypy error: Argument 1 to "validate" has incompatible type "*list[object]"; expected "str" [arg-type]

And if I add a type hint to testlist such that testlist: list[int | str] I still get Argument 1 to "validate" has incompatible type "*list[int | str]"; expected "str" [arg-type]

I was looking at Unpacking from typing_extensions but that seems to be for *args and **kwards at function definition.

#

what's weirder is that arg1 should be int, not str

dry fossil
#

@broken pollen a list has no notion of a positional type

#

use a tuple

broken pollen
#

🤦‍♂️

#

that makes sense

eager vessel
#

You have to add __init__ or use a namedtuple/dataclass

grave fjord
#

I'm looking to make a function with multiple @overload definitions also accept its main definition

#

I think

dry fossil
#

I would guess you need to move the union outside the callable in l68

grave fjord
dry fossil
grave fjord
cedar sundial
#

I also seem to be having issues with overloads, but maybe that's my lack of understanding. https://mypy-play.net/?mypy=latest&python=3.10&gist=dae368817733e41934d94913dc7f3004
main.py:7: error: Overloaded function signatures 1 and 2 overlap with incompatible return types [misc]
I can fix it by removing the default on the Literal[False] but then everywhere I've called the function, I have to set folder=False because it otherwise defaults to True, even though the actual function has a default of False. Any ideas where I'm going wrong?

mypy Playground

The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.

cedar sundial
viscid spire
#

I don't know if it's a mobile bug, but when I open your link it goes to some default code

cedar sundial
# viscid spire I don't know if it's a mobile bug, but when I open your link it goes to some def... 
from typing import Literal, overload

class DataFrame:
    ...
    
@overload
def extract_s3(bucket: str, key: str, folder: Literal[False] = False) -> DataFrame | None:
    ...


@overload
def extract_s3(bucket: str, key: str, folder: Literal[True] = True) -> tuple[list[DataFrame], list[str]]:
    ...


def extract_s3(
    bucket: str, key: str, folder: bool = False
) -> DataFrame | None | tuple[list[DataFrame], list[str]]: ...

Works on pc. Here is the code

oblique urchin
#

though I think type checkers will interpret it correctly anyway

cedar sundial
#

Yeah, that worked all fine. I will assume that is how overloads are meant to work then

round bolt
#

How do I typehint a List with 2 different arguments, like so:

from typing import List, Dict

List[str, Dict[str, str]] # Isnt valid
soft matrix
#

lists dont have a concept of having length in the type system

undone saffron
#

if you're expecting specific types at specific indices, there's not a way, but Union of types works™️

soft matrix
#

so it depends on what youre trying to do either its a union or you should be using a tuple

trim tangle
#

Or use a tuple

undone saffron
#

which of course doesn't mesh well if the type is actually a list

trim tangle
#

Or use a namedtuple/dataclass/TypedDict so that it's not a mystery to the reader what the str and the dict[str, str] are

#

If the dict[{"foo": int, "bar": str}] syntax gets accepted it's going to be great

undone saffron
#

(reminder to people that sometimes we have to deal with types we can't change and aren't ours, telling people "just use a tuple" needs investigation first to see if that's viable)

trim tangle
#

that is a good point

trim tangle
round bolt
trim tangle
#

You can index into a tuple

#

Can you show more code maybe? And the way in which mypy yells

undone saffron
#

there's an open proposal for a more flexible, but still statically typed dictionary, I wonder if there should be a mirror to that for this soemwhat common case with list, rather than tuple just being special cased as non-homogenous sometimes.

brittle socket
#

Say I have a Point class with x and y attributes.
I have function f elsewhere which receives a Sequence of point objects.
I want f to be agnostic of the Point type, but still hint that sequence elements are expected to have x and y attributes.
Can this be done with typehinting?

round bolt
trim tangle
#

That's because it can be None, it's a different issue

round bolt
#

Ah

#

alright, thanks

trim tangle
brittle socket
#

Ohh right, protocols. That's f-ing beautiful, thank you.

trim tangle
#

if something has a z the function will probably treat it in unintended ways

brittle socket
#

Oh but f only operates on thing.x and thing.y, it ignores any other attributes

trim tangle
#

I know

#

I meant generally about structural typing

#

You might accept something you didn't anticipate, because it accidentally matches the interface

#

like my favourite example which integrates smoothly into yours: ```py
class Vec2D(Protocol):
@property
def x(self) -> float:
...

@property
def y(self) -> float:
    ...

def length(vec: Vec2D) -> float:
return (vec.x2 + vec.y2) ** 0.5

@dataclass(frozen=True)
class Vec3D:
x: float
y: float
z: float

l = length(Vec3D(3, 4, 12)) # hmm...

trim tangle
brittle socket
#

Oh, I see what you mean. In your example, I'd encode the restriction into the name length_vec2d or something

#

Nah, I have just the one type

#

This is it in my actual code:

class TextAndMetadata(Protocol):
    @property
    def text(self) -> str: ...
    @property
    def metadata(self) -> dict: ...

and then

def main(sourced_texts: Sequence[TextAndMetadata]) -> None:
  ...
  docs = tuple(
      Document(page_content=src_txt.text, metadata=src_txt.metadata)
      for src_txt in sourced_texts
  )
trim tangle
#

oh, that was hypothetical naming

brittle socket
#

Yeah, sorry, I wanted to simplify to focus on my issue 😅

trim tangle
#

That's perfectly fine

brittle socket
#

Updated my example; so that's the only use I make of that sequence of objects

#

I think it should be fine

#

Maybe it wouldn't if the caller sends an object with text, metadata and other attributes he expects Documents to be created with, but hey, that's what I support

trim tangle
#

An alternative design might be: accept a sequence of a concrete struct like ```py
@dataclass(frozen=True)
class Source:
text: str
metadata: dict[str, Any]

brittle socket
#

If it's concrete, the caller to main has to be aware of Source so they can instantiate it and pass it

#

A tuple...wouldn't enforce attributes. A namedtuple would, but yeah more code on the caller site

trim tangle
#

the caller to main has to be aware of Source so they can instantiate it and pass it
Well, the caller has to be aware of the interface main supports anyway, otherwise it doesn't know how to call it

brittle socket
#

True

#

Oh I'm an idiot. TextAndMetadata could just be a TypedDict and I can play with total=True and Required

#

I don't really care for them being attributes, I just want a mapping with exact names

#

Simple as this

class TextAndMetadata(TypedDict, total=True):
    text: str
    metadata: dict

def main(sourced_texts: Sequence[TextAndMetadata]) -> None:
  ...
  docs = tuple(
      Document(page_content=src_txt["text"], metadata=src_txt["metadata"])
      for src_txt in sourced_texts
  )

main([{"text": ..., "metadata": ...} for x in y])
#

whereas before I was requiring the call to have their own class or namedtuple with text and metadata attributes

#

Anyhow, thanks!

fossil sorrel
#

How can I type hint "if return_count is given as False, function will return a Tuple" with overloads?

# 
                return_count: Optional[bool] = False,
    ) -> Union[List[Mapping[str, Any]], None, Tuple[List[Mapping[str, Any]], int]]:
``` this parameter in original
# 
        return_count: Optional[bool] = True
    ) -> Tuple[List[Mapping[str, Any]], int]:
``` and this parameter in overload?
frigid jolt
fossil sorrel
#

and default value?

#

or no default value?

frigid jolt
#

should'nt matter but you can pass the default value

fossil sorrel
#

thanks a lot

trim tangle
#

This is a very common occurrence, not sure why

#

I would recommend splitting such function into two functions

#

Or replacing it with some other construct

#

The caller will have to provide a concrete flag anyway, so might as well call a different function. Makes the interface a lot simpler

oblique urchin
#

Agree. Any time you write a new interface using @overload you should ask yourself whether you should be using multiple different functions instead

tranquil turtle
#

What do you all think about inventing something like typing.Logical? It would be an alias to object.

Why: some arguments are passed only to be converted to boolean value (implicitly or explicitly). Annotating this args would be too restrictive, because passing 0/1/[]/"abc" would not be allowed. So the only option (while allowing arbitrary objects) is to annotate arg as object, which is also bad, because it is not clear from type annotation that the only important thing is a boolean value of this arg.

So i propose typing.Logical (i dont like this particular name), which is an alias to object, but says to the reader that it is only used in logical context.

(one problem i see: object is not inferred to Truthy/Falsy inside if checks, so it might require typechecker changes, if it is necessary)

trim tangle
#

I would just use bool

#

if you have something non-boolean, well, convert it to bool

tranquil turtle
#

To mny chrs

trim tangle
#

k

vivid ore
#

Why does mypy reject this code? I have strict=True

from abc import ABC, abstractmethod
from dataclasses import dataclass
from enum import Enum

class X(Enum):
    X1 = 1
    X2 = 2

class A(ABC):
    @abstractmethod
    def a(self) -> X:
        pass

@dataclass
class B:
    a: A

    def b(self) -> int:
        match self.a.a():
            case X.X1:
                return -1
            case X.X2:
                return 4
# 
test.py:18: error: Missing return statement  [return]
#

(at def b)

#

oh nvm

#

all that complexity isn’t necessary

#

this is enough

from enum import Enum

class X(Enum):
    X1 = 1
    X2 = 2

def a() -> X:
    raise

def b() -> int:
    match a():
        case X.X1:
            return -1
        case X.X2:
            return 4
brazen jolt
#

interesting, this one works:

def b() -> int:
    r = a()
    if r is X.X1:
        return -1
    elif r is X.X2:
        return 4
tranquil turtle
#

thats because a() -> X doesnt necessarily return exactly X, it can as well return subclass of X, which can have some other keys, so your cases are not exhaustive

#

(actually, at runtime you cant subclass enum and add items, so typecheker is wrong there)

soft matrix
#

enums are implicitly final

brazen jolt
soft matrix
#

yeah

vivid ore
#

can you subclass bool?

soft matrix
#

nop

vivid ore
#

ok it rejects this too

def mk_bool() -> bool:
    raise

def do_something() -> int:
    match mk_bool():
        case True:
            return -1
        case False:
            return 4
tranquil turtle
# brazen jolt why does my example work then?

then i think it has problems with working with type inside of match parameter
x=...; match x: works fine, while match ...: doesnt work
mypy had problems related to this in the past, im pretty sure

vivid ore
#

ok yeah and it accepts this

def mk_bool() -> bool:
    raise

def do_something() -> int:
    the_bool = mk_bool()
    match the_bool:
        case True:
            return -1
        case False:
            return 4
brazen jolt
vivid ore
brazen jolt
#

no, it works

#

oh bool works too

vivid ore
#

oh

brazen jolt
#

didn't notice that, yeah ok that's the same example

tranquil turtle
#

so it doesnt work, iff you put nontrivial expression in match *:

vivid ore
brazen jolt
#

yep, mypy bug

trim tangle
#

welcome to mypy

#

and... type checking in general

brazen jolt
# 
def mk_bool() -> bool:
    raise

def do_something() -> int:
    x = mk_bool()
    match x:
        case True:
            return -1
        case False:
            return 4
``` this doesn't produce error, while `match mk_bool()` does, like denball said
vivid ore
#

what do you call the value in the match <this-value>:

#

does it have a name

tranquil turtle
#

expression

#

or match target

#

or not

#

it is called "subject" (in ast, syntax and pep)
thing in case ...: is called "pattern" (and is called once "target" in tutorial pep)

vivid ore
#

I have reported it

#

at least extracting to a variable is a good workaround

brazen jolt
vivid ore
brazen jolt
#

oh nvm

#

it's my ublock settings

#

lol, it blocked gist github domain

vivid ore
#

What I’d guess happens here is that mypy treats the match as a series of ifs and narrows the type to a literal union until it reaches never, and if you did this with ifs it would be wrong to narrow the return type to a literal union because the function could return different values/types each time

hallow flint
bleak imp
wooden panther
#

m

#

why not use a statically typed language then

#

one of the main ideas of python is that its dynamic

#

ducktyping and all that

bleak imp
#

It's also nice to not have to do this. If I need some of the safety of actual type checking, I can get it. If I don't need or want it, I don't have to.

wooden panther
#

the same can be said about using different languages

#

need type checking -> strongly typed language
needn't type checking -> weakly typed language

bleak imp
#

Then the real answer would be that I just like writing code in python, despite how silly the type hinting can be.

brazen jolt
# wooden panther why not use a statically typed language then

to be honest, you probably should, but generally, the idea is that typing often makes it much easier to intuitively see a lot of things, without it, a function like add(x, y) can do a bunch of different things, it can be a function adding 2 custom types that you have, and if you pass it something else, it might even accidentally work, but do something completely unexpected. If you do add(x: int, y: int) -> int it's suddenly very clear what this fucntion is for.

Knowing the type can then bring a lot of advantages in terms of quickly detecting bugs, and having nice auto-completion in your editor. That's because if you have a function argument x the type of whcih you know to be str, your editor will auto-complete say x.removesuffix, since it's a method of the string type, if you don't know the type it can't do that for you. Additionally, if you did x.deletesuffix your editor can immediatley let you know that this fucntion doesn't exist for strings, since it knows x must be a string.

wooden panther
brazen jolt
#

however yes, python is dynamic, and these types are purely hints, they're not anything more, and they don't affect performance (at least they generally shouldn't), like they would in compiled langs, where these types are necessary for the compiler to be able to make a bunch of optimizations, since it will know that x will be of certain type

wooden panther
brazen jolt
#

a more helpful one could for example be: get_data(client: httpx.Client), from which you'll now know you're expected to pass in a httpx.Client instance, rather than say a requests.Client instance, or aiohttp.Client instance

#

if you passed anything else but httpx.Client, your editor would be able to warn you with the annotation there, but without it, you may run into a runtime error instead

#

a runtime error however only gets triggered if that part of the code does actually run, that means you'd need it to ideally be covered by unit-tests, and can generally just be much more annoying in comparison to seeing the issue right there from the editor

bleak imp
#

This is also the thing with specifying multiple types. By doing process_data(form: str | os.PathLike) I can let the person know that this function needs some sort of path, but if I just did os.PathLike then just a file string wouldn't be a valid input, and the editor would complain.

wooden panther
#

why not just read documentation and/or function source

bleak imp
#

Because that's extra work, when most editors will show you this info with a simple hover. For a lot of things, you don't need to know more than what types it wants, especially if it is well named.

brazen jolt
#

editors can often show you both the signature and the docstring

#

but who has time to read the docstrings

#

consider a func like this

image.png
#

it's really neat to just immediately see what are all of the arguments this fucntion can take, whether they have default values, and what are the types I can pass there

#

without that, I would have to go to the docs which would really just tell me the same thing

#

but with extra work

wooden panther
brazen jolt
#

perhaps you don't, perhaps you do, it's safer when you do

bleak imp
#

Sometimes you just forget

brazen jolt
#

because editors will catch your mistakes, which I know I made a bunch of times

wooden panther
brazen jolt
#

also, if you really don't care about the type, you can type-hint that, you can do x: object

#

what's obvious to you, might not be obvious to everyone, and even to you after a long enough break from that code base

bleak imp
#

I wonder if there is any difference between x: object and x: Any

wooden panther
brazen jolt
wooden panther
#

because sometimes we forget what functions do

#

yes?

brazen jolt
bleak imp
#

That's just a part of good naming

wooden panther
#

the same can be said of what types it accepts

#

and comments, and documentation

bleak imp
#

But then that gets back to having the editor catch this for you

wooden panther
#

unit tests

bleak imp
#

Unit tests can take forever to write, and I don't need a unit test if I am just prototyping.

brazen jolt
#

as is the signature

#

it's simply a much better experience to intuitively know what a fucntion will probably do, and what kinds of values you're expected to pass and what you get back immediately, just by looking at it, than having to read the docs constantly, because the fucntion isn't named intuitively enough, and the arguments are really unclear

wooden panther
bleak imp
#

But type hints are basically 0 cost for extra information, so why not

brazen jolt
# brazen jolt consider a func like this

yeah, type hints then extend that name, as a part of the function's "signature", which is basically what you saw in the picture I sent above though, along with argument names

#

without them, you'd get say this:

image.png
#

(don't really have any better exaples lying around here now)

#

so the point is, yeah, you can have that information in the docstirng, which this function does have

bleak imp
#

The other nice thing about type hints is that they are right there, and thus easier to keep up to date. Even if the docstring is in the function, if it is a beefy docstring that could put the type info quite far from where you defined the paramaters.

brazen jolt
#

but isn't it much nicer to see it immediately?

#
  • you get that immediate sanity check by the editor, that you passed the right things
#

this is incredibly underrated for beginners, but it really can be very helpful and catch a ton of mistakes

wooden panther
# bleak imp But type hints are basically 0 cost for extra information, so why not

it adds a lot more noise to function signatures, and takes more time to write + space to store

and, although it's 100% negligible (im just talking about it because i can and no one can stop me, muahahahaha), there is a runtime slowdown -- it takes longer to parse bigger files, and the typehints have to get put into __annotations__ at runtime

brazen jolt
#

yes, but to talk about runtime slowdowns in python is a little bit pointless here, python is already really quite slow, annotations usually aren't gonna slow it down that much, where it would actually be significant for you

wooden panther
#

information density ig is how one would phrase it

bleak imp
#

That also goes back to the why type hint if python is dynamically typed. Why care about a negligible slowdown, if python is already slow?

wooden panther
#

like i said, it's 100% negligible

#

it can be ignored

brazen jolt
#

however especially with libraries, I find it super important that they do define those types there, if not everywhere, at least for the functions that are expected to be used by users (are part of the public API of that lib)

#

that's because you get to know the return types, and hence you get that autocompletion

bleak imp
#

This also applies to function arguments, if you don't type hint them your editor won't know what type, and thus what methods/properties are available for that argument, making things a lot harder.

wooden panther
# brazen jolt consider a func like this

if we look at this for example, it's fairly obvious for most of these that they take strings
things like allow_abbrev, exit_on_error, add_help are boolflags, etc.
good naming conveys a lot of information already

#

the typehinting communicates a lot less information than the names do, while still taking up roughly the same amount of characters

brazen jolt
wooden panther
#

bug checking
like unit testing?

#

regression testing?

#

etc.?

brazen jolt
#

better, it's checking for bugs statically:

from dataclasses import dataclass


@dataclass
class Monster:
    hp: int
    name: str


def damage_monster(monster: Monster, dmg: int) -> None:
    monster.health -= dmg  # type checker reports an error here, that "health" attribute doesn't exist for type Monster
#

so you can see how this can really nicely prevent typos

bleak imp
#

Also using enums and state machines to make invalid states unrepresentable.

brazen jolt
#

and you see these errors immediately

#

without having to wait for unit-tests to run, and inspecting the traceback you find

wooden panther
brazen jolt
#

of course, you'll still need unit-tests, but those do a very different kind of testing

bleak imp
wooden panther
brazen jolt
#

without this kind of static checking, you would need your unit-tests to catch even these NameErrors, which means you'll need really big coverage, and you might even write bad mocks that make the code actually pass during unit-tests, because the whole time you thought the variable name was different, and therefore everything will seem to work, but really it won't because health attribute doesn't actually exists, suddenly your code fails in production, as opposed to it failing in your editor

wooden panther
#

and if you want static analysis why not prove the correctness of functions

#

or use a statically typed language

brazen jolt
wooden panther
#

or both, if you really truly need correctness

wooden panther
#

you only need to read the parts that you're writing right then and there

#

once that part is free of typos, it can be included in the codebase

#

repeat ad infinitum

tranquil ledge
wooden panther
#

that would lead to wasted time/effort as you'd have to implement stuff twice

viscid spire
#

🤔

bleak imp
wooden panther
#

then write code that doesnt need typechecking

tranquil ledge
#

Seems like a bit of a reductive argument, imo, though I don’t completely disagree. There are myriad of reasons coders start things in Python; doesn’t mean the tools shouldn’t exist to statically check their code later.

wooden panther
bleak imp
brazen jolt
#

TypeScript doesn't actually even do any optimizations based on these types, it just transpiles to javascript

wooden panther
brazen jolt
#

the sole reason why typescript exists is to give developers types, because developers just really like them

tranquil ledge
wooden panther
#

i would say that it would be those people who actually need to verify correctness of a function

brazen jolt
#

and the reason they like them is that they convey mroe information about functions, especially for code-bases that you're not used to/haven't worked with in a long time, and for libraries, where it prevents you having to study the docs as much, and doing that much familiarizing with the lib

wooden panther
#

which is, not many

tranquil ledge
brazen jolt
#

and because of the massive benefit of statically finidng a lot of bugs, and giving you that auto-completion

wooden panther
#

you get the downsides, but not the upsides

#

i.e., not a beneficial trade-off

brazen jolt
tranquil ledge
bleak imp
#

Then install MyPy to get most of the upsides. Plus you already do get the upside of autocomplete.

brazen jolt
#

and to be honest, I had the exact same opinion about types a few years back

wooden panther
#

and iirc (its been a while since i last looked at mypy) it doesnt perform any optimisations

bleak imp
#

Then stop using notepad xD

wooden panther
brazen jolt
#

I would honestly prefer using types even if I didn't have autocompletion, i.e. on an editor like notepad, purely for the amazing bug uncovering feature of type-checkers, as you can always just run the type-checker on your code manually, with a command

#

it's the first thing you can do before unit-tests

#

because it's simply faster

wooden panther
#

unit-tests have the same property: you can always run unit-tests with a command to amazingly uncover bugs

tranquil ledge
# wooden panther you get the downsides, but not the upsides

I think an argument can be made here about ease of use and learning vs. performance, but I don’t think I know enough to really make it. Imo, if you don’t need type annotations in your Python to keep track of everything, good for you — I certainly don’t have the brain to reliably do that.

brazen jolt
#

you can, but writing unit-tests is much more time taking than writing out the types

#

and often running unit-tests can take a while too in comparison to running a type checker

#

(especially when it's integrated into your editor)

brazen jolt
wooden panther
brazen jolt
#

I'm not saying they replace unit-tests, to be clear though

wooden panther
bleak imp
#

The documentation will usually include the types anyways, so why not make it explicitly part of the function signature?

wooden panther
brazen jolt
tranquil ledge
wooden panther
brazen jolt
#

why do ```py
def foo(x, y):
"""Do X

:param int x: This argument is for this thing
:param int y: This argument is for that thing

when you can dopy
def foo(this_thing: int, that_thing: int):
...

#

yes, just doing foo(this_thing, that_thing) is an improvement, but it's not always one that's intuitive enough

brazen jolt
tranquil ledge
wooden panther
#

sorry if the question skewed or misrepresented your point -- i was just trying to get clarification about what you're saying so i can understand better

brazen jolt
#

consider this: ```py
class WeekDay(Enum):
MONDAY = 1
TUESDAY = 2
...

def handle_day(day: WeekDay) -> str:
if day is WeekDay.MONDAY:
return "monday"
elif day is WeekDay.TUESDAY:
return "tuesday"


with a type-checker, if you didn't add ifs for all of the possible cases (variants of the `WeekDay` enum), you'd get an error, that a fucntion can return a `None` value here (because all of the ifs were just skipped, and we just got to the end of the function), this is therefore a way to know that you matched this `day` exhaustively, and accounted for everything
#

a unit test might catch this, by having subtests/parametrization, but it also might not, what if WeekDay was an enum comming from an external library, and they just made a change where they added a new variant

#

with a type-checker, you'll immedately be warned everywhere because your fucntions might now be returning something else than expected

#

with unit-tests, you still only have parametrizations for the variants that the original lib contained before, and you'd need to know that the lib changed this enum and update them

#

a type-checker can give you a lot of safety in your code-base, for a pretty small cost of just writing the types in the function definitions

wooden panther
bleak imp
#

Side note, there is also the from typing import assert_never which you can use in matches/elif chains to make sure you handled all the cases.

brazen jolt
wooden panther
wooden panther
#

because that's what they're for

brazen jolt
wooden panther
#

if you know what something did previously (which you do since you wrote code using it), then by reading the changelog you can determine what it does currently

brazen jolt
#

whereas with a type checker, I just update the dependencies, run the type-checker, and if it doesn't show any problems, I know that it will very likely work

#

if it does, I immediately see what changed and where, and perhaps that means I should read the changelog, or perhaps it's a super obvious change, and I can just figoure it out from the type checker warning I got

wooden panther
brazen jolt
#

what if I maintain 15 libraries?

#

and I want to do dependency updates every week on all of those

bleak imp
#

It's basically an extension to how fun function signatures are. If the backend of a function changes but the signature stays the same, you can be pretty confident the code will still work. If a library had no type hinting and removed support for a type, how would the user know?

brazen jolt
#

all of these libs have a bunch of dependencies, say 8

#

the point here is that it makes your life much easier, I'm not saying it's not possible to do without them

#

but wouldn't you agree that having a tool to just immediately check something like this can be really nice and convenient

wooden panther
#

or documentation in general
or source code

#

that is the order of precedence that i would generally use, at least

bleak imp
#

See the above, I don't want to do that if I have 12 dependencies, each of which have their own dependencies, which can update often, and I have a several thousand line long program. The types let me know before I get a runtime error.

brazen jolt
#

I do read source of libs I'm interested in, I don't read git diffs on every change for libs that update, sometimesnot even changelogs, because I maintain a lot of things, and it would take me a LOT more time if I had to go through it all, when in vast majority of cases, those updates are just minor things that don't really affect me

bleak imp
#

And please do not say unit tests.

brazen jolt
brazen jolt
wooden panther
wooden panther
brazen jolt
bleak imp
brazen jolt
#

it's not really relevant though, the point here wasn't necessarily to just demonstrate that it fits my very specific workflow here

#

the point was that it's just easier to use in general, and how it can make your life easier in may ways

#

and faster

#

All I would suggest you to do is to give it a shot, just spin up a mid sized project and try to use types there, see how it feels

bleak imp
#

If you aren't already, also try using an editor with proper auto-complete, since they help each other.

brazen jolt
#

Python is a dynamic language, and sure, types might not be for everyone, but a lot of people who do try usually end up sticking with them, and liking them quite a lot. Me included. I originally really liked the idea of python's dynamicness, and I think it's what made learning to code easier for me here, and why I liked it so much. But after working with it for years, and refusing to use types, all it took was to try them out for a while, and I realized just how great they can actually be, and how much time they can save you.

Even though it might seem like extra work without that much benefit, that's likely because you haven't enabled the type checking behavior in your editor, so you aren't actually even getting those benefits. Without it enabled, all you're getting is the auto-complete. No bug finding behavior and type enforcement.

#

and really, to just talk about those benefits just won't do it justice here, try it out, see it yourself

wooden panther
#

i dislike autocomplete/autocorrect features

#

they tend to misinterpret what i want to type, especially when im not using english

bleak imp
#

I can understand for autocorrect, but that doesn't make much sense for autocomplete. It should only pull from what's in your namespace/valid params/args

wooden panther
#

though it has been a few years since ive last messed about with that stuff

brazen jolt
#

nobody's forcing you to use those, I'm much more excited about the bug finding behavior of type-checkers then I am about the autocompletion, although in my opinion, while it's important to be able to code without completion, and to know the language well, autocomletions can greatly speed things up once you already know those things, there's not really a great reason to keep typing everything out manually, especially when it can bring in bugs because of typos

wooden panther
#

so it might have improved since then

bleak imp
#

One of the best parts about autocomplete is I can name my functions super literally, like make_surface_from_2d_array and autocomplete will make it actually managable.

oblique urchin
#

I think for me the main benefit is understanding unfamiliar code. If you go into an untyped codebase and see lots of functions with no parameter or return types, it's hard to follow what they're doing. Types help quite a lot there

bleak imp
wooden panther
wooden panther
wooden panther
#

thank you all for talking about this with me and explaining/answering my questions :}

bleak imp
#

👋 It definitely has, especially in the from typing import x department. Good luck

brazen jolt
#

oh yeah, there were a ton of changes in that time, new PEPs are constantly appearing about typing, and I do believe you might just end up liking it!

stone harness
#

How could you type this?

def check(b):
    if b is None:
        raise Exception('None')
    return b
oblique urchin
#

I think I got one variant working with mypy and another with pyright

stone harness
#

thanks

jade viper
#

Hey guys!

#

How do I type hint a not-so-specific dict?

#

Example:

{
    'engine': positions_engine,
    'cfg': {'margin': 1, 'round_decimals': 0}
}
oblique urchin
jade viper
#

The engine key must always be present, the cfg key may or may not be present, but it's contents can be anything as long as it's also a dict

jade viper
oblique urchin
#

!pep 655

rough sluiceBOT
oblique urchin
#

(and you can use it before 3.11 too, with typing_extensions.NotRequired)

rare scarab
#

I swear I've used NotRequired since 3.8

jade viper
#

That seems perfect

#

Thanks as always Jelle!:)

vivid ore
#

What is the mechanism behind the fact that when you inherit from list[int], you only get list in your MRO?

tranquil turtle
#

__mro_entries__ attr, iirc

#

If you create class, python firstly replaces all bases with their corresponding mro_entries values, and only then does all the magic

#

Original bases are available in orig_bases attr

#

Im on mobile rn, so i omitted some dunders...

#

!d object.mro_entries

rough sluiceBOT
#
object.\_\_mro\_entries\_\_ 

object.__mro_entries__(self, bases)```
If a base that appears in a class definition is not an instance of [`type`](https://docs.python.org/3/library/functions.html#type), then an `__mro_entries__()` method is searched on the base. If an `__mro_entries__()` method is found, the base is substituted with the result of a call to `__mro_entries__()` when creating the class. The method is called with the original bases tuple passed to the *bases* parameter, and must return a tuple of classes that will be used instead of the base. The returned tuple may be empty: in these cases, the original base is ignored.
vivid ore
#

thanks

rare scarab
#

Like in most languages, generics don't actually exist.

#

Reification excluded

trim tangle
#

well, define exist 🙂

rare scarab
#

objects don't know that they are a list[str]. just list

#

typevars exist. Generics don't.

#

unless you explicitly create a class that extends list[str]

oblique urchin
#

and even then they only exist in __orig_bases__

rare scarab
#

mhm

#

even in java, you can break shit by casting to ((List)list).append(new Object())

#

because generics don't exist

soft matrix
rare scarab
#

I said reification excluded

#

which basically means: "genrics don't exist, except when they do"

bleak imp
#

Doesn't rust do monomorphization to implement generics? I'm not too clear, but it sounds like generics existing.

rare scarab
#

then rust "reifies" the generics

#

which is a fancy way of saying it adds a hidden argument.

soft matrix
#

yeah rust does do monomorphisation but the generics disappear dont they?

rare scarab
#

that argument can be inferred depending on the language

trim tangle
#

define exist and disappear 🙂

soft matrix
#

well i was more under the impression ```rs
fn id<T>(x: T) -> T {
x
}

id(1) // makes fn id(x: i32) -> i32 { x } appear and the original disappear

bleak imp
#

I mean when you get to the bottom, assembly doesn't have types so at some point in the processes, if a language is ever compiled to/past asm it can't have generics.

lunar dune
frigid jolt
minor hedge
#

I'm getting "x could not be determined because it refers to itself" from pylance and not sure what it means

async def ahhhh(bot: commands.Bot):

    async def looper():
        while True:
            if <some condition>:
                wait.cancel()
                return

    t = asyncio.create_task(looper())  # Type of "looper" could not be determined because it refers to itself
    wait = asyncio.create_task(bot.wait_for('message'))

    await t
    if wait.done():
        ...

How does this code trigger it?
Why does it go away if I comment out either wait.done() or wait.cancel() in looper, or swap wait_for to something else like asyncio.sleep?

dusky field
#

how do I get pycharm to hint to me that this goes against my type hint?

# 
from typing import Literal
def something(mylit: Literal['taco', 'burrito']):
    print(mylit)
something('whatever') # this works, I see yellow highlight
class Something:
    mylit: Literal['taco','burrito'] = None
    def __init__(self):
        pass 
Something('hi there') # this works, I see yellow highlight
x = Something()
x.mylit = 'no highlight' # this is where I wanted to see the highlight but there's not one
stray summit
#

is there a way to typehint mixing classmethods in such a way that they self-type to the class inheriting from them

i have a dump/load mixin that sets up the serializers/deserializers but defers details to the concrete classes,

if i call the mixin methods for dumping/loading on the concrete classes, i end up with a mixing lcass instances (minimal example will follow)

stray summit
viscid spire
viscid spire
# 
from typing import Literal

MexicanFood = Literal["taco", "burrito"]

def f(mylit: MexicanFood):
    print(mylit)

f("taco")  # no type issues
f("whatever")  # type issue

class C:
    x: MexicanFood

    def __init__(self, x: MexicanFood):
        self.x = x

C("hi there")  # type issue
x = C("burrito")  # no type issues

x.x = "no highlight"  # type issue
dusky field
#

@viscid spire "Do you ever want the value to be None?" no I do not

#

What am I looking for in the code you made

viscid spire
#

your previous code didn't even work, because you didn't have any parameters in your init (besides self of course)

dusky field
#

as mentioned, it works fine

#

not sure why you would say that

viscid spire
dusky field
#

let me add more of the code then

#

the point remains

#

maybe of the rest will be the issue

#

here is the full

#

same issue

# 
from dataclasses import dataclass
from typing import Literal
def something(mylit: Literal['taco', 'burrito']):
    print(mylit)
something('whatever') # this works, I see yellow highlight
@dataclass
class Something:
    mylit: Literal['taco','burrito'] = None
Something('hi there') # this works, I see yellow highlight
x = Something()
x.mylit = 'no highlight' # this is where I wanted to see the highlight but there's not one
#

sorry about not giving you the full before, hate when people do that then I do it 🙂

viscid spire
#

using a dataclass is an important detail you shouldn't have omitted yeah 😄

dusky field
#

I'm wondering if I can make pycharm complain about that last assignment

viscid spire
#

well, let's start with the = None

#

as that is incompatible

dusky field
#

I don't want a default value

viscid spire
#

then simply remove that assignment, then you don't need to worry about typehinting the None-ness

dusky field
#

I mean, I don't want them to have to input, and I don't want a default value

viscid spire
dusky field
#

if I remove it then I get other type warnings about how Something doesn't have mylit

#

The original issue is this

viscid spire
#

after creating an object, it should ideally have all the attributes needed defined

dusky field
#

I have class like Package

#

sometimes a package will have an rpm_path for example

#

maybe that's a bad example

#

ok here's one based on what I'm actually doing

viscid spire
# dusky field

pyright recognizes the last line as an issue 🤷‍♂️

dusky field
#

I am making a hierarchical representation of my it environment in python classes mostly for fun

#

I have let's say Company > Product > Component > Package and for some components I have Company > Product > Component > Service

#

if my component is like.. apache httpd, then it has a service httpd

#

but if my component is notepad or something, it doesn't even have a service

#

not sure if that's relevant but

#

anyway so instead of addressing like this

#

comp.packages['my_package']

#

since my names are unique, I want to be prettier

#

comp.package.whatever()

viscid spire
dusky field
#

I'm sort of rambling but am I making sense

#

I'm on a lot of painkillers sorry

viscid spire
#

so is Package a subclass of Component?

#

or is it a member?

dusky field
#

It's just another class with a cross link

#

I don't really understand any of the stuff you're saying so it's definitely not any of that

#

I'll show you hang on

viscid spire
#

you don't know... subclass?

#

!e

class Parent: ...
class Child(Parent): ... # subclass of Parent
print(issubclass(Child, Parent))
rough sluiceBOT
#

@viscid spire :white_check_mark: Your 3.11 eval job has completed with return code 0.

True
dusky field
#

I don't know subclass

#

here's a part of it with a few things stripped to be more concise

viscid spire
#

ok well you still have the issue with = None with an invalid typehint

dusky field
#

well pycharm and python aren't caring.. is that contributing?

#

and here's how I'm talking to it

viscid spire
#

then your pycharm is broken

dusky field
# 
my_company = Company()
dev = Environment('development')
dev.company = my_company
stag = Environment('staging')
stag.company = my_company
prod = Environment('production')
prod.company = my_company
for env in [dev, stag, prod]:
    env.app = Component('app', environment=env)
    env.components.append(env.app)
    env.app.service = Service('appservice123', component=env.app)
    env.app.package = Package('app', component=env.app)
    env.app.server = Server(component=env.app)
#

oh I have this in there probably related: from future import annotations

#

when I talk to it this way, I can very cutely talk to my stuff

#

for example I love it: dev.app.server.reboot()

viscid spire
#

so idk what's wrong with your pycharm, but you can't really try to fix your typing until you fix your pycharm

#

and unfortunately, idk about pycharm at all

dusky field
#

well thank you anyway. I really can't stand writing 15 words just to allow None

#

python really needs to sort this part out, it's very obnoxious

viscid spire
dusky field
#

I don't want to allow None as a value

viscid spire
#

I don't really understand what you mean by "15 words"

dusky field
#

field(default_factory=list)

#

T|none and allowing none

#

all this crap to work around

rare scarab
#

Write a Maybe class.

viscid spire
#

how about just pass in the values you need

dusky field
#

Don't know the value when it's created

viscid spire
#

well, I don't know why you have all these circular references in your design

dusky field
#

so when I'm at one thing I can find details about it's pretend parent

#

like an rpm doesn't have a version, it has the version of the product

rare scarab
# 
from typing import Generic, TypeVar, overload

T = TypeVar("T")
R = TypeVar("R")

class Maybe(Generic[T]):
  def __init__(self, value: T | None):
    self._value = value

  def map(self, func: Callable[[T], R]) -> Maybe[R]:
    if self._value is None:
      return self
    return Maybe(func(self._value))

  @overload
  def get(self) -> T | None: ...
  @overload
  def get(self, default: T) -> T: ...

  def get(self, default: T | None = None) -> T | None:
    if self._value is None:
      return default
    return self._value
#

no more Nones

viscid spire
rare scarab
#

They're still in typing_extensions

viscid spire
#

oh already? nice... how to use?

rare scarab
#

of course i could just make an overload

dusky field
#

I have no idea what anything I'm looking at is or does so

#

I guess I'll just not get a warning about setting it to the wrong type

rustic gull
#

is this properly typed? ```py
T = t.TypeVar("T")

@dataclass(kw_only=True)
class Variable(t.Generic[T]):
name: str
default: T = MISSING
cast: type[T] = MISSING

@cached_property
def value(self) -> T:
    _value = os.getenv(self.name, self.default)
    if _value is MISSING:
        raise MissingEnvironmentVariable(self.name)

    if _value != self.default and self.cast is not MISSING:
        try:
            _value = self.cast(_value)
        except Exception as e:
            raise ConversionError(self.name, self.cast, e) from e
    
    return t.cast(T, _value)

def __get__(self, instance: t.Any, owner: t.Any) -> T:
    return self.value

class Environment:
TOKEN = Variable(name="TOKEN")
LOG_CHANNEL_ID = Variable(name="LOG_CHANNEL_ID", cast=int)

young hawk
# 
class _HelpersBase(Base, Generic[_T]):
    _klass: type[_T] | None = None

    @classmethod
    def helpers(cls) -> _T:
        if cls._klass is None:
            msg = 'no'
            raise ValueError(msg)

        return cls._klass(cls)

Anyone have an idea why mypy thinks that return is Any and not an instance of _T?

lunar dune
young hawk
#

awesome thank you! I'll just add a cast and a todo

vivid ore
#

I think I found yet another mypy probably typeshed bug

#

functools.cache breaks function arguments checking

# 
from functools import cache

@cache
def foo(a: int, b: int) -> tuple[int, int]:
    return a, b

foo()

is accepted

soft matrix
#

Yeah it's a typeshed bug

lunar dune
vivid ore
#

ok good to know it’s known

#

thanks

young hawk
# 
_ModelActions = TypeVar('_ModelActions', bound=SQLModelActions[ModelWithActions])

Got what feels like another oddity with mypy, It doesn't like me making that where ModelWithActions is another TypeVar with an upper bound

trim tangle
young hawk
trim tangle
#

Why did you want to specify a typevar?

#

Any also works, yeah

young hawk
#

I have a few different classes that are interacting in generic land:

disclaimer: names are a wip

ModelWithActions = TypeVar('ModelWithActions', bound='SQLModelWithActions[Any]')
class SQLModelActions(Generic[ModelWithActions])

_ModelActions = TypeVar('_ModelActions', bound=SQLModelActions[Any])
class SQLModelWithActions(SQLModel, Generic[_ModelActions]):

And I have to do that because SQLModelActions is a base class and needs to be aware of the model it's going to work with, and SQLModelWithActions has a classmethod that returns an instance of the subclass of SQLModelActions.

Oh and SQLModelActions has a helper that returns an instance of a third generic object _Objects[ModelWithActions]

#

I either an doing something wrong, or just hitting the limits of the typing system in 3.11, hard to know for sure

#

Another pain point, is that even though mypy is resolving the types correctly at this point, pycharm is failing to do so, so I get no autocompletion for Model.actions() which returns the instance of the subclass of SQLModelActions

#

not sure if all that makes sense @trim tangle 🙂

trim tangle
#

Why can't you be generic over just the model type? Can you show more code maybe?

terse swallow
#

Any ideas? This is what I get from mypy

RESPONSES: Dict[str, JSON] = {"/test1": func.res_notation("en")}
# other file
JSON = Union[Dict[str, Any], List[Any], int, str, float, bool, Type[None]]
Dict entry 0 has incompatible type "str": "dict[str, Any] | list[Any] | int | str | float | type[None] | None"; expected "str": "dict[str, Any] | list[Any] | int | str | float | type[None]"  [dict-item]
young hawk
# trim tangle Why can't you be generic over just the model type? Can you show more code maybe?

here's some poc-level usage of the api

class AuditLogEntryActions(SQLModelActions['AuditLogEntry']):
    """Actions for audit log entries."""

    def get_count(self) -> int:
        return self.objects.count()

    def get_first(self) -> Optional['AuditLogEntry']:
        return next((obj for obj in self.objects.all()), None)


class AuditLogEntry(SQLModelWithActions[AuditLogEntryActions], DatedTableMixin):
    """Log of an action completed while impersonating another user."""

    # database fields here


with session_maker() as session:
    AuditLogEntry.actions(session).get_count()
#

it actually works, and mypy can infer the types (even though I'm not 100% happy with how I got it to do so), biggest annoyance is that pycharm doesn't realize that the return value from actions(session) is an instance of AuditLogEntryActions

young hawk
young hawk
terse swallow
viscid spire
soft matrix
#

You should be using sequence and mapping here not list and dict

viscid spire
soft matrix
#

Variance

#

Makes the recursive nature much more useable

viscid spire
#

🤔

#

not sure what you mean

bleak imp
# viscid spire not sure what you mean

I'm not 100% certain, but at least for Sequence I think it's because of the guarantees it offers. A Sequence object will always have those few methods, so when traversing through the structure, even though you lose some specific helper methods per container you know that the code you write will work with a lot more containers.

viscid spire
#

but... that's useless when you know for sure that is will only be a list, or only be a dict

#

because it's JSON

#

it's not gonna be some other sequence or mapping

bleak imp
#

It could also be meant just for the instantiation, where most Sequence objects can be cast to a list, so there is less explicit casting the user has to do. Beyond that, I also don't know.

soft matrix
# 
type JSON = dict[str, JSON] | list[JSON] | str | int | float | bool | None

def do_stuff(thing: JSON): ...
def call_api() -> list[int]: ...

do_stuff(call_api())  # will not type check
#

change it to mapping and sequence it will work cause list[int] can be assigned to Sequence[JSON]

viscid spire
#

hmm

noble pilot
#

Hey all, I want to type hint an object that can either be accessed through dot notation or can be indexed

#

I was able to solve indexing by making a protocol with __getitem__

#

but how would I type hint that the attributes could be accessed through dot?

viscid spire
noble pilot
#

does that also work for indexable or I need two different protocols?

#

I fixed the error from showing by putting typing.Self but that seems inappropriate

heady flicker
#

Type hints are nearly useless when you go that dynamic

#

Just implement getattr and getitem, and you'll be all set.

rustic gull
#

is there a way to assign default value to typevar?

image.png
fervent sierra
rustic gull
#

ah okay

viscid spire
cosmic agate
#

Ok, I've found a weird issue with mypy that I can't quite seem to resolve... Given this code

Example = type("Example", (), {})

match Example():
    case Example():
        print("It's an example")

    case _:
        print("It's not an example")

On the first case mypy gives this error

error: Expected type in class pattern; found "builtins.type"

Is this a bug??? What am I missing?

cunning plover
#

Typing is an addiction.
After u start working with typing
u can hardly see garbage untyped code 🤔 Tbh, all untyped code looks garbage.
If it is unit tested at least, is somewhat okeyish though

#

Recently audited code of a project
it was not typed
it was not unit tested

it was garbage of long mutations of data through dictionaries/hashmaps
with randomly accessed dynamic keys and values through it

#

i can hardly imagine now... how can people enjoy working with not typed code, it is... tech debt gathering so much quickly 🤔
that code in particular was nightmare though, it was multiple times except Exception as e, obfusacted a hundred times in addition
So many runtime surprises, i could bet, debugging it was pain like nowhere else

lunar dune
cunning plover
# cunning plover i can hardly imagine now... how can people enjoy working with not typed code, it...

good typing can simplify code many many many times pretty much.
its potential impact i think should not be understimated

it does more than just types, it structures the code in the way u will not have surprises in terms of your data
and communicating this data across services included
It makes a difference between keeping code hardly possible to run without breaking smth, and keeping it predictably working across big amount of code calls

lunar dune
# lunar dune In general mypy doesn't have (and probably never will have) a particularly good ...

To be clear, the reason why it's emitting an error here is that mypy's internal representation of a class is a mypy.nodes.TypeInfo instance. But it doesn't construct a TypeInfo internally to represent classes that are dynamically created using type(); it just sees them as "instances of type" without understanding that they're classes. So it's not emitting an error on your code for a particularly principled reason at the moment; it's emitting an error because it just doesn't understand what's going on in your code at all.

cosmic agate
#

Thanks

soft matrix
trim tangle
#

Is this intended behaviour in pylance??

def f(x: object):
    match x:
        case {"foo": "bar"}:
        #     ^^^^^^^^^^^^
        # Pattern will never be matched for subject type "object" (reportUnnecessaryComparison)
            ...

this sounds really bad

#

and is factually incorrect

viscid spire
#

that appears to be correct behaviour

viscid spire
#

object definitely isn't a Mapping, so it is clear it won't match that case

trim tangle
viscid spire
trim tangle
#

By this logic, you could say this py def f(x: int): match x: case 42: ... is also invalid, you should've used x: Literal[42]

viscid spire
#

no, because 42 is an instance of int

trim tangle
#

!e

print(isinstance(42, object))
print(isinstance({"a": 1}, object))
rough sluiceBOT
#

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

001 | True
002 | True
viscid spire
#

but you could not use int methods on it then

trim tangle
#

The error is saying that the pattern will never be matched. This is easily disproven with ```py
f({"foo": "bar"})

viscid spire
#

it won't be matched by the typehint

trim tangle
#

hm?

#

This call passes type checking, I can pass a dictionary where an object is expected. I can pass any object

viscid spire
#

hrm

trim tangle
#

In my case I'm parsing some arbitrary JSON. I guess I could annotate with an actual JSON type, but it's a pain and doesn't add any value

soft matrix
viscid spire
#

My worry is because of this

def f(x: object):
    x.bit_count()
#   ~~~~~~~~~~~~~
# Cannot access member "bit_count" for type "object"
# Member "bit_count" is unknown Pylance(reportGeneralTypeIssues)
soft matrix
#

That's different

trim tangle
#

Yeah, I'm not performing any operation that some objects don't support

soft matrix
#

There's nothing wrong with narrowing with an isinstance check before accessing

trim tangle
#

yeah

viscid spire
trim tangle
#

Well, first it checks if it's a dict

#

!e

def f(x):
    match x:
        case {"foo": "bar"}:
            return 42

print(f(69))
rough sluiceBOT
#

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

None
trim tangle
#

Also this is fine with Pylance: ```py
def f(x: int):
match x:
case True:
...

viscid spire
#

{} is not exclusively for dicts in pattern matching, it is for Mappings (iiuc)

#

same like [] is for Sequence, as well as ()

trim tangle
#

Yeah maybe

soft matrix
#

Mapping is still a subclass of object

trim tangle
#

but still, it will not crash on an integer

bleak imp
soft matrix
#

How's this relevant

bleak imp
#

not 100%, just curious about where these might go from reportGeneralTypeIssues

cunning plover
#

is there a way importing type for usage from library like types-psycopg2 ?

#

trying to grab from types-psycopg2 type for _Cursor (that is returned by cursor() function)

soft matrix
#

!d typing.TYPE_CHECKING

rough sluiceBOT
#
typing.TYPE\_CHECKING 

typing.TYPE_CHECKING```
A special constant that is assumed to be `True` by 3rd party static type checkers. It is `False` at runtime.

Usage:

```py
if TYPE_CHECKING:
    import expensive_mod

def fun(arg: 'expensive_mod.SomeType') -> None:
    local_var: expensive_mod.AnotherType = other_fun()
```  The first type annotation must be enclosed in quotes, making it a “forward reference”, to hide the `expensive_mod` reference from the interpreter runtime. Type annotations for local variables are not evaluated, so the second annotation does not need to be enclosed in quotes.
soft matrix
#

unless youre using it at runtime

cunning plover
#

yeah, fixed code. type was hidden in extra routes from normal usage

if TYPE_CHECKING:
    from psycopg2._psycopg import _Cursor

extracted 🙂

spice locust
#

i've been confused about why pyright is not liking my return value, and i seems it comes down to the fact that its bugging out on this instance variable. the type of it is supposed to be type[ItemT] but even if i explicitly give it that annotation it is still Unknown

image.png
viscid spire
spice locust
# 

class Base(Generic[ItemT]):
    EXPIRES_ATTRIBUTE = "__expires"
    PUT_LIMIT = 25

    def __init__(
        self: Base,
        name: str,
        item_type: type[ItemT] = BaseItem,
        project_key: str | None = None,
        project_id: str | None = None,
        host: str | None = None,
    ) -> None:
        if not name:
            msg = f"invalid name '{name}'"
            raise ValueError(msg)

        self.name = name
        self.item_type: type[ItemT] = item_type
        self.project_key = project_key or get_project_key()
        self.project_id = project_id or get_project_id(self.project_key)
        self.host = host or os.getenv("DETA_BASE_HOST") or "database.deta.sh"
        self.util = Updates()
        self._client = Client(
            api_key=self.project_key,
            base_url=f"https://{self.host}/v1/{self.project_id}/{self.name}",
            timeout=300,
        )

    @overload
    def get(self: Base, key: str) -> ItemT:
        ...

    @overload
    def get(self: Base, key: str, default: DefaultItemT) -> DefaultItemT:
        ...

    def get(self: Base, key: str, default: DefaultItemT = ...) -> ItemT | DefaultItemT:
        if not key:
            msg = f"invalid key '{key}'"
            raise ValueError(msg)

        key = quote(key, safe="")
        response = self._client.get(f"/items/{key}")

        if response.status_code == codes.NOT_FOUND and default is not ...:
            return default
        self.item_type: type[ItemT]
        if issubclass(self.item_type, BaseModel):
            x = self.item_type.model_validate_json(response.raise_for_status().json())
            return x
        return response.raise_for_status().json()
#

the item_type variable is of interest here

#

if you hover over it, pyright should recognize it as being of type type[ItemT], but that information is lost in the get() method

#

thus, it incorrectly identifies x as BaseModel , instead of BaseModel*

viscid spire
#

oh btw, suggest to skip the typehint on self. It is known.

#

here's a simplified version of what I understand to be your issue (?)

class C[T]:
    def __init__(self, x: type[T]):
        self.x = x

    def get(self) -> T:
        return self.x()

This works

#

Did I simplify the problem correctly?

spice locust
#

i think i found the problem

#

my self type hints should be Base[ItemT]

viscid spire
#

or just omit them

spice locust
#

ruff complains

viscid spire
#

then use typing.Self instead of Base[ItemT]

frigid jolt
spice locust
#

thanks

viscid spire
#

(of course without the typing. part, bcuz blegh)

spice locust
#

is there a way to use Protocol or another typing construct to describe "any object, whose every attribute is of type X, or a mapping whose values are of type X"

viscid spire
#

working on it

spice locust
#

i tried using __dict__: [str, X] in the Protocol, which seemed to do nothing

viscid spire
spice locust
#

let me simplify it to "any object, whose every attribute is of type X"

viscid spire
#

thing is that X could be a union

spice locust
#

X in this case would be Union[Mapping[str, "DataType"], Sequence["DataType"], str, int, float, bool]

viscid spire
# 
class MyClass:
    x: int
    y: str

every attr is of type str | int

viscid spire
spice locust
#

it will be dumped to json

viscid spire
spice locust
#

same code as before, i just want to restrict ItemT

#

so that the user of the library can't just put any type, but only one that matches the protocol

viscid spire
#

well you can restrict it to stuff which would be JSON-able at minimum

# 
from collections.abc import Mapping, Sequence
type JSON = (
    Mapping[str, JSON]
    | Sequence[JSON]
    | str
    | int
    | float
    | bool
    | None
)


def f[T: JSON](x: T) -> T:
    return x

(3.12, sorry)

spice locust
#

yes, that works for mapping, but how would i do the same for a class?

#

basically, type hint that every attribute of the class must be of type X

viscid spire
#

which attributes do you care about?

#

are you dynamically retrieving attributes with the getattr function?

spice locust
#

the user defines a structure, like

@dataclass
class UsersClass:
  x: int
  y: str
  z: Foo  # undesirable

and passes that over to be used in the library

#

it could be a dataclass, pydantic model, regular class, typeddict, dict, etc

viscid spire
#

I've got no ideas

spice locust
#

i see, for the time being i will use the existing mapping type hint and basemodel

#

anything that isnt desired would raise a typeerror during serialization

#

thanks for the help

chrome hinge
storm cloak
#

Does anyone know if exhaustive pattern matching (with assert_never) works with Generic Subexpressions?

from typing import assert_never

class A: pass
class B: pass

def foo() -> A|B:
    return A()

value = foo()
match value:
    case A():
        print("A")
    case B():
        print("B")
    case e:
        assert_never(value)

> Success: no issues found in 1 source file
This code works fine with no issues

But if I try to wrap the expression in a class, it doesn't properly get rid of the matched patterns from the union

from typing import Generic, TypeVar, assert_never
from dataclasses import dataclass

class A: pass
class B: pass

T = TypeVar("T")
@dataclass
class Wrapped(Generic[T]):
    __match_args__ = ("error",)
    error: T

def foo() -> Wrapped[A|B]:
    return Wrapped(B())


value = foo()
match value:
    case Wrapped(A()):
        print("A")
    case Wrapped(B()):
        print("B")
    case e:
        assert_never(value)

Argument 1 to "assert_never" has incompatible type "Wrapped[A | B]"; expected "NoReturn" [arg-type]

#

I don't know if this just isn't supported or I'm doing it wrong

chrome hinge
storm cloak
#

The main reason was I was trying to do pattern match with a result object like this

match get_user():
    case Ok(user):
        ...
    case Err(A()):
        ...
    case Err(B()):
        ...
#

And I keep getting return type errors because it's not exhaustive

trim tangle
#

though it is more verbose

storm cloak
#

Yeah that's the problem

stiff nova
#

what's the difference between Type[Any] and Any? mypy is giving me an error if I put a custom type for Type[Any], but with Any it's not complaining

vivid ore
#

Is there some way to express that I’m using a list immutably? I want to have covariant types but use a list. collections.abc.Sequence doesn’t support appending and concatenating.

brisk hedge
chrome hinge
#

But... appending to a list generally isn't type-safe.

vivid ore
#

I mean just concatenating

#

I do not append

#

I need concatenation of two immutable covariant sequences

#

I could just do [*a, *b] but I’m told that’s slow

chrome hinge
#

i don't think that's slow actually, but yeah, I'm unsure how to do this without using the not-yet-implemented type intersection

#

you could make your own ListLike protocol which is like a Sequence plus can be concatenated, but you'd need to write all the sequence boilerplate

#

(because you can't just inherit from Sequence because protocols can't inherit from non-protocols)

vivid ore
#

OK

#

Thanks

#

I also just noticed I ran into a pyright bug

#

it seems that if a: T, pyright will refuse to treat literal [a] as being list[U] if T is a subtype of U, even though that’d be correct (I think)

chrome hinge
chrome hinge
vivid ore
#

it’s a literal

viscid spire
vivid ore
#

at any rate mypy accepts it

#

so one of them is wrong

viscid spire
#

x: type[int] = bool

storm cloak
chrome hinge
vivid ore
#

also it only does this in a very convoluted situation

chrome hinge
#

IIRC in only one of them you could do f([5]) if f wanted a list[int|str], something like that.

stiff nova
chrome hinge
#

bool is an int subclass, so bool fits the typehint type[int].

hallow flint
vivid ore
#

but it doesn’t accept mine

chrome hinge
#

with an explicit typehint it should really be fine always

vivid ore
#

aha

#

I have found the problem

#

I have a minimal reproducible example

#

The following code is rejected:

class A: pass
class B(A): pass

def g() -> list[B]:
    return [B()]

a: list[list[A]] = []
a += [[a] for a in g()]

However, the following code, which does the same thing, is accepted:

class A: pass
class B(A): pass

def g() -> list[B]:
    return [B()]

def h(a: list[list[A]], b: list[list[A]]) -> None:
    a += b

a: list[list[A]] = []
h(a, [[a] for a in g()])
#

those two examples should both be accepted, right?

brisk hedge
#

the first is accepted if you annotate the type of the comprehension'

#

by assigning it to a variable or by passing to a function like in the second

vivid ore
#

ok

loud zodiac
#

I am trying to type hint an AST for a toy language, it's encoded as JSON. I want to type the corresponding Python dicts

import typing as t

SyntaxTerm: t.TypeAlias = t.Union[
    'LiteralSyntax',
    'LiteralBooleanSyntax',
    'TupleSyntax',
    'IdentifierSyntax',
    'LetExpressionSyntax',
    'PrintSyntax',
    'FirstSyntax',
    'SecondSyntax',
    'BinaryExpressionSyntax',
    'CallExpressionSyntax',
    'ConditionalExpressionSyntax',
    'FunctionExpressionSyntax'
]

class SyntaxLocation(t.TypedDict):
    start: int
    end: int
    filename: str

class LiteralSyntax(t.TypedDict):
    kind: t.Literal['Int', 'Str']
    value: t.Union[int, str]
    location: SyntaxLocation

class LiteralBooleanSyntax(t.TypedDict):
    kind: t.Literal['Bool']
    value: t.Literal['true', 'false']
    location: SyntaxLocation

class TupleSyntax(t.TypedDict):
    kind: t.Literal['Tuple']
    first: SyntaxTerm
    second: SyntaxTerm
    location: SyntaxLocation

class IdentifierSyntax(t.TypedDict):
    kind: t.Literal['Var']
    text: str
    location: SyntaxLocation

class ParameterSyntax(t.TypedDict):
    text: str
    location: SyntaxLocation

class LetExpressionSyntax(t.TypedDict):
    kind: t.Literal['Let']
    name: ParameterSyntax
    value: SyntaxTerm
    next: t.Optional[SyntaxTerm]
    location: SyntaxLocation

class PrintSyntax(t.TypedDict):
    kind: t.Literal['Print']
    value: SyntaxTerm
    location: SyntaxLocation

class FirstSyntax(t.TypedDict):
    kind: t.Literal['First']
    value: SyntaxTerm
    location: SyntaxLocation

class SecondSyntax(t.TypedDict):
    kind: t.Literal['Second']
    value: SyntaxTerm
    location: SyntaxLocation

class FunctionExpressionSyntax(t.TypedDict):
    kind: t.Literal['Function']
    parameters: t.List[ParameterSyntax]
    value: SyntaxTerm

mypy is struggling to infer the typed dicts by looking at the "kind" key, they all have a literal value defined

#

while parsing the JSON mypy is not able to infer the correct typed dict here

    def bind_term(self, term: SyntaxTerm) -> Expression:
        kind = term['kind']
        # ...
        elif kind == 'Tuple':
            # mypy should infer "term" is TupleSyntax typed dict
            return Tuple(
                types=TUPLETYPES,
                first=self.bind_term(term['first']),
                second=self.bind_term(term['second'])
            )
        # ...
#

mypy error is error: TypedDict "LiteralSyntax" has no key "first" [typeddict-item] at term['first']

#

is there a way to fix it or mypy just won't work?

viscid spire
#

oh wait! interesting

loud zodiac
#

thats small example

viscid spire
#

if I save key as a variable, it doesn't understand that as an if elif chain

loud zodiac
#

in my full typehint pyright fails

viscid spire
#

but if I compare it directly, it does

loud zodiac
#

ah lol

#

so I need match

#

instead of if

viscid spire
#

the issue is in saving it to a variable apparently, which is so weird

#

but match would make it cleaner anyways so use it lol

loud zodiac
#

how do I perform "or" in match?

viscid spire
#

use the | operator

#

I think

#

as so

loud zodiac
#

thanks so

viscid spire
#

you're welcome so

loud zodiac
#

with match pyright can infer all types, but mypy cant

viscid spire
#

if those classes don't exist, that could cause an issue

loud zodiac
#

all classes exist, but discord wont let me paste everything inline

#

I also can't change the structure, it is set by a thirdparty

#

but pyright is correctly inferring all types, it's mypy issue now

#

I am trying to use mypy because I would like to compile this python module with mypyc

viscid spire
#

!paste

rough sluiceBOT
#
Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

loud zodiac
#

this AST is really bad structured but I have to work with that haha

#

a workaround for this is explictly cast the type with typing.cast()

#

but I expected mypy to infer it since the typed dict union at the key "kind" is kinda obvious

trim tangle
#

I would honestly recommend modelling the AST as dataclasses and then serializing/deserializing them separately

#

this way you are not coupled to the format in which you're saving and transmitting them

#

and it's less typing, you don't have to repeat the kind

loud zodiac
#

I have another typed structure, I am doing that, parsing the JSON

#

and converting to a structure based on classes

viscid spire
#

mypy and pyright both understand this (I mean, of course)

# 
from typing import Union, reveal_type
from dataclasses import dataclass

AB = Union["A", "B"]

@dataclass
class A:
    ...

@dataclass
class B:
    ...

def f(x: AB):
    match x:
        case A():
            reveal_type(x)  # A
        case B():
            reveal_type(x)  # B
loud zodiac
#

I will just manually cast types where mypy struggles

#

I wont convert dict to dataclasses which is another dict, I have speed constraints too

viscid spire
loud zodiac
#

you can run mypy against it

viscid spire
#

you don't have matching in there or anything

loud zodiac
#

try with your sample?

def f(x: SyntaxTerm):
    match x["kind"]:
        case "Int" | "Str":
            reveal_type(x)  # LiteralSyntax
        case "Tuple":
            reveal_type(x)  # TupleSyntax
viscid spire
#

I see that it is only failing on the first for Pylance

#

(I wrote the match cases out)

loud zodiac
#

wrong right

#

I had to hardcode casts in many places for mypy to be happy

viscid spire
#

however, for each other case, it correctly infers the type

loud zodiac
#

do pip install mypy and call mypy test.py

viscid spire
#

can just use myply playground in browser

#

mypy

viscid spire
#

someone's gotta explain dataclasses.InitVar to me man

#

seems Pylance doesn't belive they are what I think they are

image.png
#

is this just Pylance being stupid?

heady flicker
#

Though it confuses me too.

heady flicker
stray summit
#

is there any way to have the concrete input type of a union be considered for a output

right now if i have soemthingl ike

def maybe_get(foo: str|None = None) -> str|None:
   ...

i have to add 2 overloads to ensure the type of foo can correctly pass to the output - is ther any way to make a alias for a union that automatically drops the missmatch?

trim tangle
#

though the default will not work

#

maybe typevar defaults can somehow help, but i don't know tbh

fossil sorrel
#

Does anyone know a lxml stub library that is not potato?

image.png
# 
(method) def xpath(
    _path: _AnyStr,
    namespaces: _NonDefaultNSMapArg | None = ...,
    extensions: Any = ...,
    smart_strings: bool = ...,
    **_variables: _XPathObject
) -> _XPathObject
``` https://pypi.org/project/lxml-stubs/ shows this method returns a  _XPathObject, and i cannot iterate over it
PyPI
lxml-stubs

Type annotations for the lxml package

#

through, there is no problem with it runtime.

# 
    soup: etree._Element = ...
    iframes = soup.xpath("//iframe")

    for i in range(len(iframes)):
``` like, this works just fine in runtime.
#

but shows error with lxml-stubs

heady flicker
# trim tangle maybe typevar defaults can somehow help, but i don't know tbh

Yes, they can. Here's how you do it:

from typing import TypeVar, TYPE_CHECKING

if TYPE_CHECKING:
    MaybeStr = TypeVar("MaybeStr", str, None, default=None)


def maybe_get(foo: "MaybeStr" = None) -> "MaybeStr":
    ...


reveal_type(maybe_get())  # None
reveal_type(maybe_get(None))  # None
reveal_type(maybe_get("Hello"))  # str

@stray summit fyi

stray summit
stray summit
heady flicker
#

Ooh you were using mypy?

#

It's a feature from future python so I'm not sure whether mypy already supports it. I know for a fact that pyright does because it always implements new stuff quickly

stray summit
#

mypy reveals it as str only

stray summit
soft matrix
#

TypeVar defaults don't work for functions

stray summit
#

i created https://discuss.python.org/t/typing-firstresolved-as-mechanism-to-refer-to-the-first-resolved-passed-typevar/34003 to discuss having something to make this nicer, not yet sure if its a good approach

Discussions on Python.org
typing.FirstResolved as mechanism to refer to the first resolved pa...

the mypy issue on typevars and optional triggered this idea In utility helpers dealing with data transformation and/or loading the return value can be a product of multiple input parameters for example def get( self, key: str, default: _D | None = None, convert: Callable[[str], _T] | None = None, ) -> _D ...

#

but this hopefully reduces the need for certain types of overloads

torpid turret
#

guys, do i really need to import Self inside TYPE_CHECKING or not?

if t.TYPE_CHECKING:
    from typing_extensions import Self

or just simply import it globally?

#

also, if im using typing_extensions in my module, do i need to include it in my dev requirements?
cuz i've seen someone who said he/she doesn't have that module

trim tangle
#

If you're using typing-extensions, you need to include it in your runtime requirements

#

I think that's a perfectly fine requirement if your library uses typing

#

It is not a standard library module

torpid turret
trim tangle
#

Yes, the import will run at runtime just like any other import

#

and you don't need to use TYPE_CHECKING in that case

torpid turret
#

my library does import typing
but im using py3.8

torpid turret
trim tangle
torpid turret
#

ok, got it
thanks

noble elm
rough sluiceBOT
#

postgrest/_async/request_builder.py lines 203 to 208

BaseSelectRequestBuilder[_ReturnT].__origin__.__init__(
    self, session, headers, params
)
AsyncQueryRequestBuilder[_ReturnT].__origin__.__init__(
    self, session, path, http_method, headers, params, json
)```
noble elm
#

pyright complains here because __origin__ only exists at runtime. if I do typing.get_origin(Base[T]).__init__(...), i get Expected 0 positional arguments
should I just cast(type[Base[T]], get_origin(Base[T]))?

undone saffron
#

The comment near that line seems incorrect unless something weird is happening in this code base elsewhere I'm not seeing at a glance.

#

It's perfectly fine to initialize generic specialized classes

#

for instance, set[int]().union(*other_sets)

#

(which that's required for that to properly infer the type as set[int] in that case, it won't be inferred from other_sets even if those are all Iterable[int])

#

this looks like it should just be using super().__init__()

#

and you have incompatible inheritence issues here too...

noble elm
#

yeah you're right, it should be just super(). let me see how i can fix the weird inheritance going on. thanks!

green abyss
#

keep up

green abyss
trim tangle
#

truly one of the of all times

#

oh wait, I already have 50 messages

vivid ore
#

Is this a mypy bug?
Suppose I have:

from typing import Generic, Protocol, TypeVar
from collections.abc import Iterator
from typing_extensions import Self


T_Co = TypeVar("T_Co", covariant=True)


class ComposableSequence(Protocol[T_Co]):
    def __iter__(self) -> Iterator[T_Co]:
        ...

    def __add__(self, x: Self, /) -> Self:
        ...

This is accepted:

def bar() -> ComposableSequence[tuple[list[int | str], int]]:
    return []

This is not:

def foo() -> ComposableSequence[tuple[list[int | str], int]]:
    return [([], 0)]

test.py:18: error: Incompatible return value type (got "list[tuple[list[<nothing>], int]]", expected "ComposableSequence[tuple[list[int | str], int]]")  [return-value]
bleak imp
# vivid ore Is this a mypy bug? Suppose I have: ```py from typing import Generic, Protocol, ...

I'm still not good with typing, however you might find this useful: this code ```py
from typing import Generic, Protocol, TypeVar
from collections.abc import Iterable
from typing_extensions import Self

T_Co = TypeVar("T_Co", covariant=True)

class Addable(Protocol):
def add(self, x: Self, /) -> Self: ...

class test(Addable, Iterable, Protocol[T_Co]):
pass

def foo() -> test[tuple[list[int | str], int]]:
return [()]
gives
main.py:19: error: List item 0 has incompatible type "tuple[]"; expected "T" [list-item]``` Which I'm don't really understand why, but it is different. You could probably also get this to be done with NewType to aviod the class overhead, but I'm too small brain to do it.

#

I got back to the exact same issue by moving the [T_Co] py class ComposableSequence(Addable, Iterable[T_Co], Protocol): pass

bleak imp
#

If a variable length tuple is used, then the empty case passes py def foo() -> ComposableSequence[tuple[tuple[int | str, ...], int]]: return [((), 0)] # Passes

tranquil turtle
bleak imp
hallow flint
vivid ore
vivid ore
vivid ore
vivid ore
vivid ore
#

I wish Python typing had a way to make type parameters *variant at the usage site, like Java does it

#

e.g. if you use a list of T but never append to it, in Java, you could represent that as List<? extends T>, and then it just doesn’t allow you to write to it

#

that is, in Python typing terms, the only type that a parameter of type ? extends T accepts is Never

#

similarly if you only add to a list you can write List<? super T> and this will only produce values that can be converted to Object, nothing more specific

#

I think these are the top and the bottom of the type hierarchy, respectively

#

And you can convert an invariant type to a *variant one

#

but not back

pastel egret
#

Well, you can sorta do that via Sequence[T]? Though that's not generically applicable.

spice locust
#

is there a type i can use or create that has the attribute access of NamedTuple combined with the Mapping compatibility of TypedDict?

tranquil turtle
#

can you subclass them both at the same time?

noble pilot
# 
def _get_identifier(self, obj: T) -> Any:
        value = None
        try:
            value = getattr(obj, self._field_name)
        except AttributeError:
            try:
                value = obj[self._field_name]
            except (TypeError, KeyError):
                raise AttributeError
        finally:
            return value
#

How should I properly define T or bind it? Essentially looking for it to be a custom class or a dict

#

I tried two protocols of __getitem__ and __get_attribute__ and bound=Union but that still through off pyright

hallow flint
#

getattr with dynamic attribute cannot be typed. so just use Any

#

if you knew the field then it would be a union of a dict and a protocol with that field (not a type var)

grave fjord
#

mypy considers this valid:

from typing import NamedTuple

class Example(NamedTuple):
    foo: int
    bar: int

def foo() -> Example:
    return Example(1, 2)[:2]
rough sluiceBOT
#

mypy/types.py line 2422

self.partial_fallback,```
grave fjord
#

it's a bit horrible though but the test cases should be correct

terse swallow
#

does mypy check stuff in the TYPE_CHECKING as well?

if TYPE_CHECKING:
    from slodon.slodonix.slodonix.slodonix_windows import _Info

like is the module going to be executed when I run mypy ...

soft matrix
#

modules arent executed or imported with mypy

primal sapphire
#

is it correct that mypy returns different errors for these two?

def return_value_error() -> int:
    value = max([1, 2], default=None)
    return value

def arg_type_error() -> int:
    return max([1, 2], default=None)

I was expecting the second one to also have a return type error instead of None being invalid for max.

chrome hinge
#

Interesting, same in pylance. I'm guessing there's order-of-inference weirdness here, with them first inferring that for T1|T2 to be int, T2 must be int, and then noticing that None doesn't fit such a T2.

primal sapphire
#

yeah I was looking at the GH issue 6692 which suggests its supposed to do that but not sure why 🤔

oblique urchin
primal sapphire
#

Ah, yes makes more sense with an example with T1 | T2 and seeing it deduce valid arg types, thanks both

trim tangle
#

In both cases it's not clear whether the return type is wrong or the argument to the function is wrong, yeah

digital isle
#

Hey!
Should this be supported by mypy?

class Base(ABC):
    pass

class Derived(Base):
    pass

class Handle(Generic[T]):
    def __init__(self, t: T):
        self.t = t

def foo(h: Handle[Base]):
    print(h.t)

d = Derived()
foo(Handle(d))
trim tangle
digital isle
#

I am getting an error that Outer[Derived] != Outer[Base], but it should not matter

trim tangle
#

you're passing in Derived but the function accepts Outer

#

Do you know about variance? Maybe that's the problem

digital isle
#

Sorry. Fixed.

trim tangle
#

If you want your generic class to be covariant, you need to specify that in the typevar ```py
T = TypeVar("T", covariant=True)

digital isle
#

I SEE. Thank you so very much.

#

My google search history looks incredebly stupid having googled this issue in a hundred different ways. And yet talking to a human I understand the issue within a minute.

#

Yes. My generic is covariant, had to specify in the TypeVar. I'll read the tutorial a bit more

rustic gull
#

I also have a question related to variance, if that's ok!

TSchema = TypeVar("TSchema", bound=TypedDict)
TSelf = TypeVar("TSelf", bound="SerializesTo")

class SerializesTo(Generic[TSchema]):
    def serialize(self) -> TSchema: ...
    @classmethod
    def deserialize(cls: Type[TSelf], data: TSchema) -> TSelf: ...

class SerializedAnimal(TypedDict):
    legs: int

class SerializedElephant(SerializedAnimal):
    trunkLength: float

class Animal(SerializesTo[SerializedAnimal]):
    pass

# Base classes of Elephant are mutually incompatible
# Base class "SerializesTo[SerializedElephant]" is incompatible with type "SerializesTo[SerializedAnimal]"
class Elephant(Animal, SerializesTo[SerializedElephant]):
    pass

I can't have TSchema be invariant here, because of the above error. Making TSchema covariant would fix it, but then I would be able to do stuff like this, which doesn't make sense:

def factory(data: SerializedAnimal):
    return Elephant.deserialize(data)

The solution, suggested by Eric Traut the pyright developer, is to have Animal be generic:

TAnimalSchema = TypeVar("TAnimalSchema", bound=SerializedAnimal)

class Animal(SerializesTo[TAnimalSchema]):
    pass

This works beautifully, but pyright now can't determine the expected type of my deserialize parameter:

image.png
#

Apologies for the essay, if anyone is able to offer any thoughts I'd appreciate it very much :)

undone saffron
trim tangle
undone saffron
#

it also doesn't have to be public for this to be the case

trim tangle
#

because it's essentially adding this to your class ```py
def set_t(self, t: T) -> None
def get_t(self) -> T

undone saffron
# rustic gull I also have a question related to variance, if that's ok! ```py TSchema = TypeV...

I'm not 100% sure I'm following what your needs on this are or why the design appears to couple a dataclass with an empty subclass of that dataclass. I'm not saying that this must be indicative of an issue, but it's hard to reccomend a specific path towards making this understood without understanding the full context, and that could range from knowing immediately how to add the right type info to considering if other struture suits your task better.

rustic gull
#

Many thanks for the reply! That makes sense.
I have JSON-serializable types in my project, and I'd like the types to be aware of their serialised form. That lets me catch mistakes, for example in incorrectly expecting certain keys to be present, or forgetting to include required things in the class's serializer

undone saffron
#

oop, yep messages at same time seems to confirm this to an extent. If you want to do this yourself, it's still possible, but I would consider looking at options already designed to help in this space as well

digital isle
rustic gull
#

Haven't heard of structs in python before, thank you I'll do some reading :)

I also don't quite follow what you said here though: a dataclass coupled with an empty subclass of that dataclass? Are TypedDicts dataclasses at runtime?

It also doesn't feel like this problem would be particularly niche - base class uses a generic type parameter in a method signature, but we can't infer that type in a subclass 🤔

undone saffron
#

No, sorry I could have been slightly clearer. I mean a class that's only intended to hold data, not the literal dataclasse(es) module

rustic gull
#

Ahh gotcha - the actual context is a game. The objects do have their own state and behaviour

undone saffron
#

Hmm. I'm wondering more about if this is a "change your structure more" situation or if we would need to focus more on how to ensure the type system has enough information.

Ideally, all the behavior that is actually shared could be composited in, but you really need anything that diverges to be simple in divergence and isolate well to a generic parameter or be added on a per-type basis if it can't.

rustic gull
#

Should I give a different example with more context?

undone saffron
digital isle
#

Is pyright overall preferable for new projects in 2023 over mypy? Or is it a matter of requirements with no clear answer.

#

The pyright page - surprise surprise - appears to make it seem surperiour in many ways. But most projects I've seen so far for reference have used mypy.

undone saffron
#

I find pyright does a better job currently, but that the end goal should be that type checkers will eventually all agree. mypy is much older, and has a lot of inertia as a result.

#

@rustic gullsorry, I had to step away for a moment, the following appears to all type check as I would want it to:

from typing import Generic, Self, TypedDict, TypeVar, reveal_type


class BaseDict(TypedDict):
    pass


T = TypeVar("T", bound=BaseDict)

class Serializable(Generic[T]):

    def serialize(self: Self) -> T:
        ...

    @classmethod
    def deserialize(cls: type[Self], data: T) -> Self:
        ...


class ElephantDict(BaseDict):
    trunk: int

class SomeOtherSharedBehaviorClass:
    pass

class Elephant(Serializable[ElephantDict], SomeOtherSharedBehaviorClass):
    pass


edict: ElephantDict = ElephantDict(trunk=1)
elephant: Elephant = Elephant()
reveal_type(Elephant.deserialize(edict))  # Type of "Elephant.deserialize(edict)" is "Elephant"
reveal_type(elephant.serialize())  # Type of "elephant.serialize()" is "ElephantDict"
undone saffron
# digital isle The pyright page - surprise surprise - appears to make it seem surperiour in man...

for slightly more detail that I think I probably should have included originally.

  1. pyright has a page which shows the known differences between it and mypy here

  2. users of visual studio code will (by default) get the basic type checking from pyright indirectly through VSCs language server, and could get strict with one small change, so you or your users may inadvertently end up using both type checkers as has been pointed out in some discussions regarding the long term future of type checking and (Currently) diverging behavior

  3. I specified pyright on strict here in this specific case because pyright with strict is currently the only type checker I'm aware of to get this specific variance check correct, so it was more a means of confirming that you didn't have something that a typechecker still could see a reason for invariance which could either mean needing to modify structure or a bug report if pyright is tripping up on it.

rustic gull
# undone saffron <@456226577798135808>sorry, I had to step away for a moment, the following appea...

Yep, this works on my end!

I don't think this method would fit my particular project, because my classes are very heirarchical. I'd rather have my serializing be heirarchical too in that case:

class SerializedElephant(SerializedAnimal):
  trunkLength: int

class SerializedCat(SerializedAnimal):
  furLength: int

class Elephant(Animal, SerializesTo[SerializedElephant]):
  def serialize(self): return {**super().serialize(), "trunkLength": self.trunkLength}

class Cat(Animal, SerializesTo[SerializedCat]):
  def serialize(self): return {**super().serialize(), "furLength": self.furLength}
#

you may have been right all along :)

I'm wondering more about if this is a "change your structure more" situation

#

Wow, I just found my one brain cell deep in my back pocket!
I said that Eric Traut recommended making Animal generic:

TAnimalSchema = TypeVar("TAnimalSchema", bound=SerializedAnimal)

class Animal(SerializesTo[TAnimalSchema]):
    pass

I was using it like this:

class Elephant(Animal, SerializesTo[SerializedElephant]):
    pass

Instead of this:

class Elephant(Animal[SerializedElephant]):
    pass

The hierarchical serialized schema works as expected now, god i'm an idiot 😂 thank you so much for your time Sinbad :)

cosmic plinth
hallow flint
#

no. if something false to __getattr__ it'll usually end up as whatever the return type of __getattr__ is, which is usually Any (this means you won't have false positives, but also won't get much actual type checking)

#

as in that PR, if TYPE_CHECKING is a useful tool to lie to type checkers while doing whatever arbitrarily dynamic thing you want to do out of sight

cosmic plinth
hallow flint