#type-hinting

1 messages · Page 40 of 1

stiff acorn
#

Pylance: Type of "func" is "Task[(foo: str | int, bar: str | int), None]"

#

mypy errros on func.delay though

trim tangle
#

maybe mypy just ignores the decorator

#

sounds like both type checkers exhibit some amount of buggy behaviour here

stiff acorn
#

seems to be it, I'm running mypy in ci so i'll go with that to atleast get typechecking in spots that use func() syntax

#

What even happens to other decorators in overload? Do they get executed? I would assume they don't affect runtime?

trim tangle
#

!e

from typing import overload

counter = 0

def decorator(fn):
    global counter
    print("called decorator on", fn)
    counter += 1
    c = counter
    def wrapped(*args, **kwargs):
        print("called wrapped", c)
        return fn(*args, **kwargs)
    return wrapped

@overload
@decorator
def foo(a: int) -> int: ...
@overload
@decorator
def foo(a: str) -> str: ...

@decorator
def foo(a: int | str) -> int| str:
    return a

print(foo(42))
rough sluiceBOT
stiff acorn
#

That's incredibly helpful, thank you

stray summit
#

i need some kind of combine(Foo[T], Foo[U], Foo[V]) -> CombinedFoo[T|U|V] - is anyone aware of a example of something like that?

feral wharf
stray summit
#

hmm, i kinda wish i could define something like def combined[*Ts](inputs: *(Foo[T] for T in Ts) -> CombinedFoo[Union[*Ts]: ...

trim tangle
#

Actually, depends on what Foo is. Is it covariant or invariant?

# 
# `Box` is covariant

def f[T](*boxes: Box[T]) -> Box[T]:
    return random.choice(boxes)

strbox = Box("foo")
intbox = Box(42)
lbox = Box([1, 2, 3])

reveal_type(f(strbox, intbox, lbox))
``` ^this works because, since `Box` is covariant, `Box[int]` is assignable to e.g. `Box[int | str]`, and so pyright just resolves `T` to be `int | str | list[int]`
#

simpler example: ```py
def pick[T](a: T, b: T, /) -> T: ...

thing = pick(42, "hello")
``` this call is allowed, and thing is inferred as int | str by pyright

trim tangle
jolly cipher
#

have you ever done something like that?

rare scarab
#

I say use a @dataclass(frozen=True) instead

green bluff
#

it shouldn't be allowed since they don't share a base type

oblique urchin
green bluff
#

T should be str OR int, not either or in the same call site

#

if it's observed as object or Any, then it's pretty meaningless

trim tangle
green bluff
oblique urchin
#

int | str seems like a pretty reasonable inferred type in @trim tangle 's example above

green bluff
#

if you want them to be different types, you should have more generics

#

[T, U]

trim tangle
green bluff
#

ehhhhhh this feels like a bad semantic argument

trim tangle
#

What would you define as different types then?

green bluff
trim tangle
#

You mean, at runtime?

green bluff
#

correct

trim tangle
#

Well, int | str is not a class, so you cannot have an instance of the class int | str

#

There's a difference between "Python class" and "type-system type", just like in TypeScript

oblique urchin
#

you mean like this?

trim tangle
#

that's cheating

green bluff
#

yeah, this is a bit frustrating from a tooling respect since you're now forced to rely on another system

#

I've never really dug into "what" int | str is before, and now I'm seeing it's actually a types.UnionType and not a typing.Union

oblique urchin
green bluff
#

I didn't even know types exists

green bluff
#

I think my disagreement with the conversation stems with the lack of native ADTs in Python so you can't express what StrOrInt is

trim tangle
#

~~In a type system with subtyping, you generally want this: if A is a subtype of B, and an operation f is allowed on x: A, then doing ```py
y: B = x
f(y)

#

or rather, if f(x) is not allowed, then f(y) shouldn't be allowed either

#

Wait, I got it backwards.

#

Yeah, if f(A) is not allowed, it doesn't make sense for f(B) to be allowed

restive rapids
# trim tangle Wait, I got it backwards.

i think it makes more sense for f to be taking B (the supertype) in the example?

class B:
  ...

class A(B):
  ...

x: A = ...
y: B = x
# if f(x) is not allowed, so shouldnt be f(y)
green bluff
#

it would have to be the other way around

trim tangle
#

well, that's not strictly related to the inference of type variables

#

you could define a type checker where pick(42, "hello") would not be allowed, requiring you to explicitly do pick[int | str](42, "hello")

#

(well... the latter is not a thing in Python)

restive rapids
#

unless you make pick a class and make its __new__ funny

green bluff
#

yeah, I just think the union inference shouldn't be allowed for generic positions and should be required to be explicit?

green bluff
restive rapids
#

im mostly in favor of the union strategy except in cases where i actually want the types to have a common concrete base (thats not just object)
its just practical, idk

trim tangle
#

!e

def f[T]():
    pass

print(f[int])
rough sluiceBOT
# trim tangle !e ```py def f[T](): pass print(f[int]) ```

:x: Your 3.13 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 4, in <module>
003 |     print(f[int])
004 |           ~^^^^^
005 | TypeError: 'function' object is not subscriptable
green bluff
#

oh sad

image.png
trim tangle
#

also this will never work with functions whose types are in stubs only

green bluff
#

but you can do that with classes, which is weird

trim tangle
# trim tangle We could benefit from something like `NoInfer` in TypeScript: <https://www.types...

It would work like this ```ts
function pick<T>(first: T, second: NoInfer<T>): T {
return Math.random() < 0.5 ? first : second
}

const _test1 = pick("Foo", 42) // ok: Argument of type '42' is not assignable to parameter of type '"Foo"'.
const _test2 = pick("Foo", "Bar") // not terribly helpful: Argument of type '"Bar"' is not assignable to parameter of type '"Foo"'
const _test3 = pick<string>("Foo", "Bar") // allowed
const _test4 = pick<string | number>("Foo", 42) // allowed

`_test2` is an example of unhelpfully inferring a literal type
rare scarab
#

Or you can annotate the variable.

#

I think?

trim tangle
#

Actually, typescript seems to have a weird opinion on this inference. ```ts
function pick<T>(first: T, second: T): T {
return Math.random() < 0.5 ? first : second
}

const _test1 = pick("Foo", 42) // error: Argument of type '42' is not assignable to parameter of type '"Foo"'.
const _test2 = pick("Foo", "Bar") // _test2: "Foo" | "Bar"

rare scarab
#

it unionizes literals if they don't include the bottom type unknown in the shared type heirarchy

trim tangle
#

I've heard Amazon banned the use of | in their TypeScript code. They don't want any unions

rare scarab
#

or I guess the simpler way to put it is if they return the same typeof

green bluff
trim tangle
#

||/s||

trim tangle
#

(like in Python)

#

I don't know what the benefit of adding tagged unions to a language that already has unions would be

green bluff
#

because the unions aren't real at runtime

rare scarab
#

but rust unions can be values, structs, or tuples

#

how are those real but objects aren't?

green bluff
#

you cannot ignore variants

restive rapids
#

you can't ignore kind's if you are switching on them in a function which is supposed to return something non-void either

trim tangle
green bluff
restive rapids
#

it is with match-case

green bluff
rare scarab
#

I'm sure there's a ruff rule to enforce exhaustive match statements

#

maybe it's in redknot

green bluff
restive rapids
#

sure it does
maybe not in mypy

green bluff
#

no, it literally doesn't lol

#

you can ignore it at runtime just fine

restive rapids
#

typecheckers are not runtime

#

i literally have exhaustiveness errors from pyright until i finish writing all the cases every day

green bluff
#

which is a large part of my problem with the bolted on types semantics in Python

#

at least typescript forces you to go through a compiler

restive rapids
#

why are you pushing without passing CI anyways

green bluff
rare scarab
# 
match x {
  MyEnum:X => {},
  _ => {
    println!("hehe, non-exaustive");
  }
}
green bluff
rare scarab
#

it's explicitly non-exaustive

green bluff
#

and it's enforced byt he compiler

trim tangle
rare scarab
#

make a encoding that gives match a default _ branch that raises an error

green bluff
#

sqlalchemy is especially painful

trim tangle
#

You can enable this one rule selectively

#

mypy unfortunately doesn't have such a rule

green bluff
#

I dropped mypy from my set of tools since it was always lagging on new Python features

#

so I had a fight with my coworkers over this and I'm curious what people think. for each table, I create a NewType over its id: pk attribute and that is kind of annoying since each table = newtype = type DbId gets longer, but it gives me nice ergonomics that I can't accidentally pass the wrong ids across relationships

image.png
#

and as a result, I also need to create this giant type_annotation_map since sqlalchemy doesn't resolve newtypes natively

#

do you think this is a sound idea? is there a way to make it more ergonomic?

#

something I was thinking about was rather than using NewType, create my own newtype abstraction using inheritance, but I'm not sure if there's a solution for the type_annotation_map

rare scarab
#

Are you sure the type_annotation_map is required for that?

green bluff
rare scarab
# 
MyTableId = NewType("MyTableId", int)

class MyTable(Base):
  id: Mapped[MyTableId] = mapped_column(primary_key=True)
green bluff
#

it seems sqlalchemy is not calling the newtypes to get the inner type

restive rapids
# rare scarab make a encoding that gives match a default `_` branch that raises an error 
from codecs import register, CodecInfo, utf_8_encode
from collections.abc import Buffer
from ast import Match, NodeVisitor, match_case, MatchAs, Raise, Call, Name, Constant, parse, unparse, fix_missing_locations

class Transformer(NodeVisitor):
    def visit_Match(self, node: Match) -> None:
        if not isinstance(node.cases[-1].pattern, MatchAs):
            node.cases.append(match_case(
                MatchAs(),
                body=[Raise(Call(
                    Name("TypeError"),
                    [Constant(f"Non-exhaustive match-case (@{node.lineno})")]
                ))]))

def decode(buf: Buffer, errors: str = "strict") -> tuple[str, int]:
    txt = buf.__buffer__(0).tobytes().decode(errors=errors)
    tree = parse(txt)
    Transformer().generic_visit(tree)
    txt = unparse(fix_missing_locations(tree))
    txt += "\n"
    return txt, len(txt)

@register
def search(name: str) -> CodecInfo | None:
    if name == "strict": return CodecInfo(
        name=name,
        encode=utf_8_encode,
        decode=decode,
    )
rare scarab
green bluff
#

very cool, but disgusting

rare scarab
#

exhaustive-match-at-home

restive rapids
#

smh its just a file-wide rust procedural macro

green bluff
rare scarab
#

reading

#

By “loose matching” does it mean types without being defined in type_annotation_map?

green bluff
#

well sqlalchemy has an internal, default type_annotation_map

#

so things like datetime and int, etc. resolve natively

rare scarab
#

except datetime is naive

#

it's not sqlalchemy's fault.

green bluff
#

is there a DateTimeUTC type in sqla? I haven't checked yet

rare scarab
#

I made a TypeAdapter for it.

# 
from datetime import UTC, datetime

from sqlalchemy import TIMESTAMP, Dialect, TypeDecorator


class UTCDateTimeType(TypeDecorator[datetime]):
    impl = TIMESTAMP
    cache_ok = True

    def process_result_value(
        self, value: datetime | None, dialect: Dialect
    ) -> datetime | None:
        if value is None:
            return None
        # datetime is stored internally as a naive UTC timestamp. It's safe to make it
        # tz-aware.
        return value.replace(tzinfo=UTC)
green bluff
#

this is my id definition, and everything works flawlessly in practice, but working in the tables can be kind of annoying because of the mix of what's allowed since Mapped["Type"] and relationship("Type") etc. are eval()'d while it seems this inherited column is not

image.png
green bluff
rare scarab
#

once or twice

#

Done reading. NewType for each table does seem to create a bit of boilerplate.

#

ideally, sqlalchemy should be able to infer the internal type of the NewType

#

Maybe you can reduce it with a UserDict object implementing _missing_

#

NewType.__supertype__/TypeAliasType.__value__

radiant jetty
#

Hello, is there any way how to get correct type hints on this?



class Decoder[T]:
    # I would like this to support also something like TypeUnion[T]
    def __init__(self, t: type[T]) -> None:
        pass

    def decode(self, v: bytes) -> T:
        raise NotImplementedError("")


NON_UNION_DEC = Decoder(int)  # works fine
UNANNOTATED_DEC = Decoder(str | int)  # need type annotation
ANNOTATED_DEC: Decoder[int | str] = Decoder(int | str)  # incompatible type "UnionType"; expected "type[int] | type[str]

if __name__ == "__main__":
    reveal_type(NON_UNION_DEC.decode(b"1"))  # correctly inferred as "builtins.int"
    reveal_type(UNANNOTATED_DEC.decode(b"1"))  # inferred as "Any" (should be str | int)
    reveal_type(ANNOTATED_DEC.decode(b"1"))  # correctly inferred as "Union[builtins.int, builtins.str]"```
trim tangle
terse sky
# 
from dataclasses import dataclass, field, MISSING


def named(
    *,
    default=MISSING,
    default_factory=MISSING,
    init=True,
    repr=True,
    hash=None,
    compare=True,
    metadata=None,
    kw_only=MISSING,
):
    return field(default, default_factory, init, repr, hash, compare, metadata, kw_only)


@dataclass
class MyArgs:
    first_arg: int
    second_arg: int = named(default=0, init=False)


a = MyArgs(5, 7)
#

how should I annotate named so that mypy correctly gives an error here?

#

hmm, when I specify that named returns a Field, I get other errors

#

ah, I see

#

actually, yeah, I don't 😂

#

I guess what I need is something like dataclass_transform but for field

polar aurora
#

I may be missing something, but can't you just declare the return value of named(...) to be Field?

#

for metadata you might want Optional[dict[str, Any]] = None or something? Not sure what shape that arg has in your mind

terse sky
#

i tried, but it does not type check

#

metadata is just forwarding an argument that field already requires

#

I mean I could just leave it as th edefault I suppose

#

okay, I realize there were some unrelated errors. but here's the most boiled down version:

def named(
    *,
    default=MISSING,
) -> Field[int]:
    return field(default=default)

@dataclass
class MyArgs:
    first_arg: int
    second_arg: int = named(default=0)


a = MyArgs(5)
#

this "should" type check but does not

#

it seems like the way to go is just to make named generic function and return _T itself, and do # type ignore inside the body

terse sky
#

i assume I'm missing something stupid, but why does this not type check

# 
_T = TypeVar("_T")
class Factory(Generic[_T]):
    def __init__(self, func: Callable[[], _T], /) -> None:
        self.func = func

def f() -> int:
    return 5

def glug(x: _T | Factory[_T]):
    pass

glug(Factory(f))
trim tangle
#

If you add a return type annotation, it works out somehow ```py
def glug[_T](x: _T | Factory[_T]) -> tuple[_T, _T]:
assert False

x: tuple[int, int] = glug(Factory(f))
reveal_type(x)

#

though in general, this function signature is potentially... unclear

terse sky
#

hmm, I guess the _T itself could be Factory[_T]

trim tangle
#

yeah

terse sky
#

so, to explain why I'm doing this weird stuff

#

I am working on this function that forwards to dataclasses.field

#

i got it working but it's pretty horrifying

#

because you have to use @overloaded, to handle how default vs default_factory is done

#

I want to define a bunch of these helper functions, and make it easy for users to do so too, so having to write every function signature four times is pretty bad

#

I think the solution here is to wrap both sides of the union

#

which I didn't really want to do; I wanted to keep the common case nice

# 
class Factory(Generic[_T]):
    def __init__(self, func: Callable[[], _T], /) -> None:
        self.func = func

class Value(Generic[_T]):
    def __init__(self, val: _T, /) -> None:
        self.func = val

def f() -> int:
    return 5

def glug(x: Value[_T] | Factory[_T]):
    pass

glug(Factory(f))
#

this works, but now people would have to write e.g.

@dataclass
class MyArgs:
   first_arg: list[int] = argparse_field(default=Value(0), ...)
#

i guess one alternative is to just give up on statically enforcing that exactly one of default and default_value are passed

terse sky
#

how much would you hate if a library had you write, instead of

@dataclass
class foo:
    x: int = field(default=0)
    y: list[int] = field(default_factory=list)

, rather

@dataclass
class bar:
    x: int = my_field(de=fault(0))
    x: list[int] = my_field(de=fault_factory(list))
trim tangle
terse sky
#

a little

trim tangle
#

Maybe you get away with just default_factory? like ```py
class bar:
x: int = my_field(default=int) # or lambda: 0
x: list[int] = my_field(default=list) # or lambda: []

#

tbh I find it weird that int supports a 0-argument call producing 0

terse sky
#

yeah, that is pretty weird

#

Hmm I could also just support default because... that's what argparse does

# 
In [8]: ap = argparse.ArgumentParser()

In [9]: ap.add_argument("--foo", type=list, default=[])
Out[9]: _StoreAction(option_strings=['--foo'], dest='foo', nargs=None, const=None, default=[], type=<class 'list'>, choices=None, required=False, help=None, metavar=None, deprecated=False)

In [10]: ns1 = ap.parse_args([])

In [11]: ns2 = ap.parse_args([])

In [12]: ns1
Out[12]: Namespace(foo=[])

In [13]: ns1.foo.append(5)

In [14]: ns1
Out[14]: Namespace(foo=[5])

In [15]: ns2
Out[15]: Namespace(foo=[5])
#

😂

frigid jolt
#

every primitive data type has a default empty arguments constructor

tranquil turtle
#

bool() == False also feels weird

trail kraken
#

!e print(bool() == int(), bool() == str())

rough sluiceBOT
terse sky
# 
from dataclasses import fields
...
def parse_args(t: type[_T]) -> _T:
    for f in fields(t):
        ...

if I have code like this, mypy complains

scratch2.py:108: error: Argument 1 to "fields" has incompatible type "type[_T]"; expected "DataclassInstance | type[DataclassInstance]"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)

What's the most appropriate way to resolve this? I found this definition

_DataclassT = TypeVar("_DataclassT", bound=DataclassInstance)

but _DataclassT and DataclassInstance (imported from _typeshed) both don't seem to be for public usage

#

it seems like maybe I should be importing from _typeshed?

#

I see, I guess _typeshed only has an underscore as a sort of "warning" that they are type checking only types

lunar dune
terse sky
#

Is there a way to use a boolean function in a bound for a generic?

brazen jolt
terse sky
#

just the example above to work

#

I want a generic function that accepts a type, and for the type to be constrained such that it's a dataclass

#

alex suggested using a function that returns a bool

#

I'm asking how I can incorporate that into a generic bound

# 
from typing import TYPE_CHECKING
from dataclasses import fields

if TYPE_CHECKING:
    from _typeshed import DataclassInstance

def parse_args(d: type[_D]) -> _D:
    for f in fields(d):
        ...
#

currently the code looks like this and it works

#

so I'm asking in response to Alex's suggestion, if there's a way to use is_dataclass instead of DataclassInstance and have it work the same way

feral wharf
#

Protocol?

terse sky
#

what would that look like

brazen jolt
#

would this not work then?

def parse_args[T: DataclassInstance](d: T) -> T:
   ...
terse sky
#

more or less

terse sky
brazen jolt
#

and what's wrong with using DataclassInstance?

feral wharf
# 
class LikeDataclass(Protocol):
    is_dataclass: Literal[True]

def func[T: LikeDtataclass](dc: T) -> T: ...

Not sure if that's what you want but basically, that LikeDataclass can have the same stuff as the class that fields() wants

terse sky
#

nothing - alex just suggested something else, and I asked how to make it work, lol

brazen jolt
#

well, the is_dataclass check just returns a TypeIs[DataclassInstance]

#

so it can be used to narrow down the type in a condition

terse sky
terse sky
feral wharf
#

Yes

brazen jolt
#

well yeah, because this doesn't help you to solve the issue

feral wharf
#

If dt is a class with a is_dataclass attribute

brazen jolt
#

if you wish to use is_dataclass, I guess you could like do this:

def func[T](dc: type[T]) -> T:
    if not is_dataclass(dc):
        raise TypeError
#

but that's not ideal

#

just using DataclassInstance for the type is much more type safe

terse sky
# feral wharf If `dt` is a class with a `is_dataclass` attribute 
In [1]: from dataclasses import dataclass

In [2]: @dataclass
   ...: class Foo:
   ...:     pass
   ...: 

In [3]: Foo.is_dataclass
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[3], line 1
----> 1 Foo.is_dataclass

AttributeError: type object 'Foo' has no attribute 'is_dataclass'
brazen jolt
#

I wouldn't go with a protocol here

feral wharf
terse sky
brazen jolt
#

no

#

that's the point

terse sky
#

then yeah, not very useful

brazen jolt
#

that's why I find that suggestion weird

#

and why I was confused for why you're avoiding DataclassInstance

terse sky
#

yeah, I didn't think that python's type system could do that

#

most constrained generics systems cannot just use arbitrary boolean functions that way

brazen jolt
#

I mean, the function acts similarly to an isinstance check

feral wharf
terse sky
#

Not really trying to avoid it, like I said, just trying to understand Alex's suggestion.
It would be a little nicer to avoid using _typeshed and needing the if TYPE_CHECKING thing but eh

#

not a big deal

brazen jolt
#

inside of an if is_dataclass(x) a type-checker can assume that x is a DataclassInstance

terse sky
#

yeah, that's a branch inside, but what I need is to actually constrain a generic function

brazen jolt
terse sky
#

which there isn't a way to do in python's generic system (or most constrained generics systems)

brazen jolt
#

but it doesn't restrict the function calls to just dataclasses

terse sky
#

yeah

#

I was also thrown off at first by it being _typeshed - now that I read it, I understand the motivation but initially it's a lot more confusing that way

#

(that is, the leading underscore made me assume it was package internals)

brazen jolt
#

well, it is generally a good idea to try and avoid _typeshed imports if you can, but in this case, there isn't really anything better, dataclasses don't have any common parent class, they're built through dynamic code generation, and there's no easy way to distinguish between a dataclass and a regular class.

The is_dataclass actually does this in the back: ```python
def is_dataclass(obj):
"""Returns True if obj is a dataclass or an instance of a
dataclass."""
cls = obj if isinstance(obj, type) else type(obj)
return hasattr(cls, _FIELDS)


`_FIELDS` being `'__dataclass_fields__'`
#

and since the dataclass.fields method expects a DataclassInstance, you need to convert that type into it somehow

#

even if you defined a protocol with __dataclass_fields__ on your own, it wouldn't work

terse sky
#

interesting; why not?

brazen jolt
#

because that's not what dataclass.fields is expecting as an input, it expects DataclassInstance which afaik isn't a protocol

#

okay, it actually is

terse sky
#

Ah, ok

brazen jolt
#

in that case, a protocol should actually work too

# 
class DataClassType(Protocl):
    __dataclass_fields__: ClassVar[dict[str, Field[Any]]]
terse sky
#

but the protocol has the downside that _FIELDS is an implementation detail, strictly speaking

brazen jolt
#

that import will give you that same thing

terse sky
#

that seems a little worse to me on balance since I'm accessing implementation details directly now, no?

#

I'd probably only do this if I really badly wanted to avoid the typeshed dependency

brazen jolt
#

well, you're kinda also doing that through _typeshed, which on it's own could be considered an implementation detail

#

but I'd probably go with _typeshed too

terse sky
#

yeah, I figure if dataclass changes something then the typeshed authors will too 😛

brazen jolt
#

precisely

vivid ore
#

Pyright and Mypy have the exact opposite behaviour

Screenshot_From_2025-04-11_23-50-26.png Screenshot_From_2025-04-11_23-50-12.png
cinder bone
#

The mypy one strikes me as a wrong annotation on the return value

#

not sure why pyright is being funky

trim tangle
#

!e

from typing import NamedTuple

Foo = NamedTuple("Foo", [])
print(Foo)
rough sluiceBOT
polar aurora
#

Hmm, does it want NamedTuple[T] where T is a TypeVar?

plain dock
#

i have a function that returns a NamedTuple without complaint, but it's the class form instead

trim tangle
polar aurora
#

Hmm, yeah, I guess it doesn't make sense. I'm just trying to parse the pyright error

#

and isn't it saying it's expecting an instance of a type, and getting that type instead?

trim tangle
#

Yes

#

it's like if you did:

def banana() -> int:
    return bool
polar aurora
#

but how is NamedTuple(...) not an instance of NamedTuple?

trim tangle
#

are you familiar with collections.namedtuple?

polar aurora
#

Oh, duh, sure

#

Yeah I wasn't very, haven't run into it much

trim tangle
#

!e ```py
from typing import NamedTuple

print(NamedTuple)
Foo = NamedTuple("Foo", [])
print(Foo)

rough sluiceBOT
polar aurora
#

I get it now, cheers

trim tangle
#

It's also subclassable to make class-based NamedTuples

rough sluiceBOT
#

Lib/typing.py lines 3025 to 3031

_NamedTuple = type.__new__(NamedTupleMeta, 'NamedTuple', (), {})

def _namedtuple_mro_entries(bases):
    assert NamedTuple in bases
    return (_NamedTuple,)

NamedTuple.__mro_entries__ = _namedtuple_mro_entries```
trim tangle
#

hacks

vivid ore
#

!e

from typing import NamedTuple

class Foo(NamedTuple):
    x_1: int
    x_2: float

print(f"{Foo(x_1=10, x_2=2.9)!r}")
rough sluiceBOT
trim tangle
# vivid ore but you can subclass NamedTuple

!e
It creates a subclass of tuple that doesn't involve NamedTuple though (since NamedTuple is not a class):

from typing import NamedTuple

class Foo(NamedTuple):
    x_1: int
    x_2: float

print(Foo.mro())
rough sluiceBOT
vivid ore
#

I just want dependent types ):

polar aurora
vivid ore
#

what is pyright smoking

Screenshot_From_2025-04-12_12-57-37.png
#

im doing

y = x if isinstance(x, Sequence) else [x]

where x: float | Sequence[float]

#

where is it getting the int from

feral wharf
vivid ore
#

im sorry what

image.png
trim tangle
vivid ore
trim tangle
#

or rather, when you don't plan to have a subclass, just use the class name directly

trim tangle
vivid ore
#

yeah

#

but you can subclass Foo

#

in this example

trim tangle
#

right, the implementation doesn't match the signature, so pyright is complaining

vivid ore
#

oh

#

makes sense

#

I get it

trim tangle
#

In the enum example, it's not possible to subclass Foo, so technically the Self return type will never be violated, but it doesn't put the two pieces of knowledge together

vivid ore
#

ok

nocturne folio
#

is type hinting an art form

jovial gazelle
#

it certainly looks that way sometimes

oak ruin
#

Type hinting is helpful but you can go down rabbit holes when using certain frameworks. Luckily most popular Python frameworks have proper type hints or stubs made b my other developers, but doing type hinting with an unmaintained framework is so much pain

terse sky
# 
@overload
def foo(x: Literal[True] = ...) -> str:
    pass
    
@overload
def foo(x: Literal[False] = ...) -> int:
    pass

def foo(x = False):
    pass

reveal_type(foo())
#

the revealed type here is str

#

how would I do this correctly?

#

obviously, I believe the revealed type here should be int

#

actually I think I get it. I have to remove the default value from the first overload

signal oak
#

Is there any substitution for @override decorator in py < 3.12?
I can't use typing_extensions as well

oblique urchin
signal oak
# oblique urchin that's what typing-extensions is for. why can't you use it?

I am writing an extension for a certain application.
Application uses python 3.11 so no go for typing.override()

For the sake of typing_extensions it is quite weird.
Since I am writing an extension, I cannot just pip install packages, however, according to the API I can install .whl binaries. And also, software doesn't come with typing_extensions as site-package either.
For automated binary installation I have a class, that ... installs them lol. I also have a logger class.
logging.Formatter class has method format that I then override in my custom formatter.
The problem is that I use my logger inside of module installation class and thus cannot install typing_extensions module.
Hope it makes sense!

oblique urchin
signal oak
oblique urchin
# signal oak What do you mean by that?

the packaging tools you use are likely able to declare install-time dependencies, e.g. in pyproject.toml. I'm not sure exactly what tools you are using but what you're saying doesn't convince me that it's impossible to install third-party dependencies.

#

Though an alternative is to use a TYPE_CHECKING hack, something like ```from typing import TYPE_CHECKING
if TYPE_CHECKING:
from typing_extensions import override
else:
def override(f): return f

signal oak
stray summit
#

does anyone have any examples on using typevar tuples to take in a sequence of types and having multiple methods in turn be returning unions of the types

cinder bone
trim tangle
#

You cannot get a union from a typevartuple

#

It was initially allowed to do Union[*Ts], but it is now disallowed for some reason

#

So if you run e.g. ```py
class Foo[*Ts]:
def init(self, *items: *Ts) -> None:
self._items = items

def get_first(self):
    return self._items[0]

def f[*Ts](foo: Foo[*Ts]):
reveal_type(foo.get_first())
``` through pyright, it will reveal the type as Union[*Ts], but it will not allow you to name this type explicitly. A bit silly

#

To be fair, this is also true for this ```py
class Foo: pass
class Bar: pass

def f(x: Foo):
assert isinstance(x, Bar)
return x

f inferred as: (x: Foo) -> <subclass of Foo and Bar>

stray summit
#

i have something like

class Loader[*Ts]:
    loader_types: Sequence[type[Ts]

     def parse(self, input) -> Union[*Ts]
#

but no idea how to do the mapping

trim tangle
stray summit
#

list of types i pass in as valid to receive

restive rapids
# 
class Loader[T]:
  def __init__(self, loader_types: Sequence[type[T]]):
    self.loader_types = loader_types
  def parse(self, input: str) -> T:
    ...
``` would work on pyright because it unifies types (so, in `Loader([int, str])`, `T = int | str`) (not on mypy because it joins them so you'd probally end up with `object` or something)
stray summit
#

yup - pyright gets it, mypy botches it

trim tangle
#

You could make an atrocious API like this ```py
class Loader[Uni, *Ts]:
def init(self: Loader[Never]) -> None: # type: ignore
self._items: tuple[Any, ...] = ()

def add(self, new_type: type[T]) -> Loader[Uni | T, *Ts, T]:
    loader = Loader()
    loader._items = self._items + ()
    return loader

def parse(self, input: str) -> Uni:
    ...

loader = Loader().add(int).add(str)

trim tangle
median gulch
#

Is there a way of defining literals as "any combination of the options"?

Best example is tkinter direction constants:

from typing import Literal

W = LEFT = "w"
E = RIGHT = "e"
N = TOP = "n"
S = BOTTOM = "s"
NSEW = ALL_DIRECTIONS = N + S + E + W
Direction = Literal[LEFT, RIGHT, TOP, BOTTOM, LEFT+RIGHT, TOP+BOTTOM, TOP+LEFT, TOP+RIGHT, BOTTOM+LEFT, BOTTOM+RIGHT, ALL_DIRECTIONS]
brazen jolt
#

afaik, the answer is no. What you could do is define the combinations manually as actual separate literals, or take a sequence of the literals (or a set to ensure each option is only present at most once)

median gulch
#

and tuple would need be mutually exclusive based on type options

brazen jolt
#

but you had LEFT+RIGHT as an option too, a tuple of (horiz, vert) can't represent that

#

I'd just go with set[Direction] or write them out explicitly

#

depending on your use-case

#

in this case, writing them out doesn't look too bad

median gulch
#

I wanted to have this behavior at the type checker level as well

median gulch
#

Thanks

brazen jolt
#

and btw, I'd recommend an enum for this, if you need the str representations, you can use StrEnum

#

it's generally a better idea than a union Literal

median gulch
#

so it's better to keep these zero-cost simple things as is

brazen jolt
# 
class Direction(StrEnum):
    LEFT = "w"
    RIGHT = "e"
    ...

def my_wrapper(d: Direction):
    tkinter_method(d.value)
#

worrying about 0 cost abstractions when you're using python IMO isn't all that important

#

I'd generally prefer a clean design choice over something that might save you a few bytes of memory

median gulch
#

Keeping it simple is the main concern

brazen jolt
median gulch
median gulch
brazen jolt
#

I'd personally much rather see an API like Direction.UP than one where I'm passing "n" as a literal

#

I wouldn't say this harms how hidden the control flow is, but I suppose it depends on the kind of wrapper you're making

median gulch
brazen jolt
#

oh, I see

median gulch
#

If it isn't broken, don't fix it.

brazen jolt
#

yeah, if you're passing these into tkinter functions directly, this makes more sense

#

I thought by wrapper you meant more like a complete one, supplying your own functions that call tkiner internally

#

rather than just exposing some things to the user but still have them use tkinter calls too

median gulch
# brazen jolt I thought by wrapper you meant more like a complete one, supplying your own func...

No, no. Makes sense to think that. tkinter is just a python-ized version of the thought process behing the C API. It's a lot nicer, but simple things are verbose and can turn spaghetti real fast. Plus, many quality of life things that you barely touch after writing the first time require more ugly glue code, or a new dependency just to do very little, if your glue code is use-case based.

#

I want to make all these static things explicit and declarative. Keep it simple and leave what already is good as is. Improve customizability by abstracting and turning declarative the things you can only learn by reading 2 books, 3 documentations and the C implementation.

#

Explicit method calls if your property needs to be calculated. Regular properties if they are calculated before you're able to interact with them, or if they are constant/final.

#

dicts, tuples, NamedTuples, TypedDicts, literals, dataclasses sometimes. That it.

#

and an awful lot of type hinting

brazen jolt
#

yeah, that makes sense, the only annoying thing with it is that the tkinter method signatures won't direct you to your abstractions, which can make it hard to know which one to use from your lib

median gulch
#

I wish I had a counter of the amount of "find text" operations I did on those two PDFs.

#

Yet to finish so many more to go.

brazen jolt
#

Yeah, what I meant was that, if your lib just provides types for tkinter, but not wrapper methods, those types will indeed need to be compatible with what the tkinter built-in methods already take. That however means that even if the built-in tkinter methods have some type annotations for the variables, those annotations won't direct the devs towards your types.

And while your docs can be good, in pretty much all cases, devs prefer just following func signatures, since it's hard to remember everything, and and cross-referencing docs, even when it's good docs, can still be very annoying. With signatures, it'd be type enforced (at least with a type checker)

median gulch
brazen jolt
#

you wouldn't even be able to define tkinter stubs to direct the user towards your types, since the types are just string literals

#

but yeah, if the built-in tkinter doesn't even have stubs, and your goal is just providing them, then this does make sense

#

I haven't used tkinter in a long time (not for like 4 years), so I've got no idea on how type-complete it is

median gulch
brazen jolt
#

yeah, that makes sense then

median gulch
brazen jolt
#

lol

terse sky
#

is there a way to annotate with something that ends up as an Annotated, but to restrict the annotated values in relation to the type

#

Like, say I have some Config[T] class

#

I want users to be able to annotate with an Annotated[T, Config[T]] - i.e. enforce that the Config value that is passed to the Annotated has the same type parameter as the T

oblique urchin
#

!pep 746

rough sluiceBOT
oblique urchin
#

Which is kind of stuck

terse sky
#

interesting, thanks for the info

terse sky
#

if I'm understanding this correctly, then it means that I can currently do strictly more type checking with

@dataclass
class Foo:
    x: SomeType = my_field(...)

Than with

@dataclass
class Foo:
    x: Annotated[SomeType, MyInfo(...)]
#

but if this PEP were to be accepted, then the situation would be reversed

#

I could do strictly more type checking with the latter than the former

rapid brook
terse sky
#

sorry, not sure I follow

#

I was looking at two approaches where I wanted to add extra metadata to fields of a dataclass

#

One was using Annotated, one was writing my own my_field function. I was comparing the type checking possibilities of the two approaches

wooden cipher
#

hi
i have

empty = object()

Wrapped = TypeVar("Wrapped", Settings, UserSettingsHolder)


class LazyObject(Generic[Wrapped]):
    _wrapped: Wrapped | None | object = None
    
    def __init__(self):
        self._wrapped = empty

    def _setup(self) -> None:
        ...
        self._wrapped = Settings()

    def configure(self) -> None:
        ...
        self._wrapped = UserSettingsHolder()

so at the start of the class self._wrapped is None, then it's empty

but when using, self._wrapped is guaranteed to be either Settings or UserSettingsHolder

how can i tell python and mypy that the type will be one of these two?

#

the problem happens here:

    def register(self, settings_module: ModuleType) -> None:
        if self._wrapped is empty:
            self._setup()
        self._wrapped.register(settings_module)

    def clear(self) -> None:
        if self._wrapped is empty:
            return

        self._wrapped.clear()

where mypy errors saying empty object doesn't have a method register(), or clear()

polar aurora
# 
from typing import TypeGuard

def is_settings_thingy(obj) -> TypeGuard[Settings | UserSettingsHolder]:
  return (isinstance(obj, Settings) or isinstance(obj, UserSettingsHolder))
``` is one way I guess
#

maybe you could just say isinstance(obj, Wrapped) there? Not sure if TypeVars let you do that.

hushed panther
#

Would removing the default value of _wrapped work?

polar aurora
#

I don't think so, because I don't think mypy can figure out that it can't be None

hushed panther
# 
class LazyObject(Generic[Wrapped]):
    _wrapped: Wrapped | object
    ...
polar aurora
#

Why not just Wrapped? What other types do you want it to be able to be?

hushed panther
polar aurora
#

Oh sure, yeah so we don't know yet.

#

I guess that's my main question; why does this need to be a union type at all?

wooden cipher
#

it's first assigned to None because some infinite recurssion would happen without that, it has some discussion behind it in django

then it's assigned to empty, because there is nothing to wrap when the instance starts
it needs to be provided

pastel egret
wooden cipher
pastel egret
#

The checker knows enums are singletons, so an is check can remove it from the variable type. Same with string equality.

trim tangle
cinder bone
#

Any updates on PEP 728?

simple sundial
#

can someone help me understand how to use sockets

trim tangle
warm topaz
#

i have package which uses type stubs for type definitions where a particular class is generic and the generic type parameters aree defined in the pyi files using default= for TypeVar
we can also however provide the generics ourselves in user side code like A[MyType] is there a way to determine the source of these generic type params whether its stub defined or user defined

rare scarab
#

!pep 728

rough sluiceBOT
rare scarab
#

it's in typing-extensions 4.13

warm topaz
# rare scarab !pep 728

currently my class aren't really typeddicts its just a generic class inheriting from t.Generic, for reference this is the issue I am facing
https://github.com/typeddjango/django-stubs/pull/2590#issuecomment-2812769225

GitHub
Feat/builtin models types by FallenDeity · Pull Request #2590 · t...

I have made things!
This PR aims to add type hints to builtin model fields, i.e for example models in contrib, admin, auth etc.
Base generic fields are modified to use the default= from PEP-696 to ...

rustic gull
#

Hello, why does calling _get work but get doesn't?

from typing import TypeVar, reveal_type
from collections.abc import Callable

class Base: ...

class Derived(Base): ...

T = TypeVar("T", bound=Base)

def _get(cls: type[T]) -> T:
    return cls()

def func(get: Callable[[type[T]], T]) -> None:
    reveal_type(get(Derived))  # incompatible type
    reveal_type(_get(Derived))  # Derived
trim tangle
# rustic gull Hello, why does calling `_get` work but `get` doesn't? ```py from typing import...

Callable[[type[T]], T] is not the type of a generic function. It might be easier to see with new-style generics:

from collections.abc import Callable

def func[T](get: Callable[[type[T]], T]) -> None:
    reveal_type(get(Derived))  # incompatible type
    reveal_type(_get(Derived))  # Derived
``` this means that you can call this function with e.g. `Callable[[type[int]], int]`, and `T` will be bound to `int`. `T` is bound to a specific type, which isn't necessarily `Derived`. In your case it's more like ```py
def func[T: Base](get: Callable[[type[T]], T]) -> None:
    reveal_type(get(Derived))  # incompatible type
    reveal_type(_get(Derived))  # Derived
#

If you want to express the type of a generic function, you need a callable protocol

from typing import Protocol

class GetFn(Protocol):
    def __call__[T: Base](self, cls: type[T], /) -> T: ...

def func(get: GetFn) -> None:
    ...
rustic gull
trim tangle
#

if you are in this position: py def func(get: Callable[[type[T]], T]) -> T: # here you can't really call get in a type safe way, because you don't know what T is. (you can call it with an Any argument)

rustic gull
#

I think I am following, how would you suggest I go about this if I want type hints?

trim tangle
rustic gull
#

thank you, I will use that then

ruby yew
#

Why it is saying it is posibaly unbound

image.png
trim tangle
ruby yew
#

If that if is_main is not in the finally then there is no problem but the problem is just when it gose in the finnaly block.

trim tangle
#

Can you show the code where it doesn't show the error?

ruby yew
#

yea you are right it is not smart enough

#

my bad

trim tangle
#

you could open an issue on github if it hasn't been brought up yet

ruby yew
#

but I see it many times detect it in complex code

#

whatever

#

Hey I found the problem

#

the variables are global

#

not local

#

so other thread can change it

trim tangle
#

well, socketio.run could also change it

#

(in theory)

ruby yew
#

yea

trim tangle
ruby yew
#

they can do like get_all_gloabl_vars["is_main"] = True

#

I forgot what how we vchange the global vars

trim tangle
#

An alternative would be keeping the things that depend on is_main as a singular entity, something like ```py
@contextlib.contextmanager():
def app_resources():
chat_history_file = ...
chat_history.load_from_json(chat_history_file)
...
try:
yield
finally:
chat_history.load_from_json(chat_history_file)
...

if is_main:
ctx = app_resources()
else:
ctx = contextlib.nullcontext()

with ctx:
socketio.run(app, ...)

ruby yew
#

I need to do research on it

edgy stirrup
#

Hi, how can I solve the following typing error on 'arg' in the following code? Pylance says Type of "arg" is unknown

def func(data: dict[str, Any]):
    args = data.get("args")
    if args is None:
        raise ValueError("ERROR: Arguments not found")

    if not isinstance(args, list):
        raise ValueError("ERROR: Arguments must be a list")

    if not all(isinstance(arg, str) for arg in args):
        raise ValueError("ERROR: All arguments must be strings")
hushed panther
#

You can annotate the args as a list of string (my first thought)

args: list[str] = ...
terse sky
#

What I would probably do is just write your for loop that processes each arg, and put your check and raise there

#

Then the narrowing to str type should work

#

Type narrowing cannot work with arbitrarily complex logic even if it's obvious to a human

# 
for arg in args:
    if not isinstance(arg, str):
        raise ...
    # arg should be treated as a str here
#

If you add more details about what you want to do, a better recommendation can be made though

edgy stirrup
#

Thanks, but I still get the Type of 'arg' is unknown. The updated code:

def func(data: dict[str, Any]):
    args = data.get("args")
    if args is None:
        raise ValueError("ERROR: Arguments not found")

    if not isinstance(args, list):
        raise ValueError("ERROR: Arguments must be a list")

    for arg in args:
        if not isinstance(arg, str):
            raise ValueError("ERROR: All arguments must be strings")

the code is intended to validate some data from a JSON. The full code contains other checks for values coming from other keys

#

and indeed, if I add after the for loop for example len(args) as a separate statement, I get:

Argument type is partially unknown
  Argument corresponds to parameter "obj" in function "len"
  Argument type is "list[Unknown]"
spiral fjord
# 
from typing import Any, TypeGuard

def is_string_list(lst: list[object]) -> TypeGuard[list[str]]:
    return all(isinstance(ele, str) for ele in lst)

def func(data: dict[str, Any]):
    args = data.get("args")
    if args is None:
        raise ValueError("ERROR: Arguments not found")

    if not isinstance(args, list):
        raise ValueError("ERROR: Arguments must be a list")

    if not is_string_list(args):
        raise ValueError("ERROR: All arguments must be strings")
    
    len(args)```
edgy stirrup
#

nice, thanks! Now what is left is to solve the error:

Argument type is partially unknown
Argument corresponds to parameter "lst" in function "is_string_list"
Argument type is "list[Unknown]"

updated code:

from typing import Any, TypeGuard


def is_string_list(lst: list[Any]) -> TypeGuard[list[str]]:
    return all(isinstance(ele, str) for ele in lst)


def func(data: dict[str, Any]):
    args = data.get("args")
    if args is None:
        raise ValueError("ERROR: Arguments not found")

    if not isinstance(args, list):
        raise ValueError("ERROR: Arguments must be a list")

    if not is_string_list(args):
        raise ValueError("ERROR: All arguments must be strings")

    len(args)
stiff acorn
# 
from dataclasses import dataclass

@dataclass
class Foobar:
    baz: int | None = None

    def is_baz_int(self) -> bool:
        return self.baz is not None
        
obj = Foobar(20)
reveal_type(obj.baz)
if obj.is_baz_int():
    reveal_type(obj.baz)

is it possible to narrow down obj.baz like that?

#

Mypy reports both as int | None

#

thanks, guess I'll do without it

hushed panther
edgy stirrup
hushed panther
terse sky
#

and using reveal_type shows that the type of arg inside the loop is str, like I said

#

If you can add some representative code that actually errors in a mypy playground it'll be easier to help

#

I also wouldn't use a type guard unless it's necessary or you really want to reuse some bit of code. If you can just make use of type narrowing in the function body easily, then that's preferable

polar aurora
cinder bone
#

yeah as a casual type hinting person I never really understood why we would want TypeGuard if we have TypeIs, and the documentation isn't really helpful for this lol

oblique urchin
#

TypeGuard came first, then we discovered some problems with how it was specified. There was a proposal to make TypeGuard work differently, but we rejected that for compatibility reasons and instead I proposed adding TypeIs.

polar aurora
#

does the type system have a special way to indicate that a function has a 'divergent' or 'terminal' type? like, something that makes the program exit?

#

or do we just say -> None for that?

#

like in Haskell you can say your return type is ⊥

#

aha typing.Never

#

y'all thought of everything, lovely

rapid mesa
#

Hello, have anyone here used pylance in VSCode. I am working on a project that relies heavily on "imgui" and we get literally 100's of "errors" that are incorrect.
Writing # type: ignore at every single line will imho only bloat the code. Can I whitelist e.g. "imgui" somewhere?

#

This error occurs for every single use of imgui.X everywhere. I want the linting to be on by default, but want to whitelist all imgui.X usage. Is it possible?

mossy rain
#

otherwise, a quick fix that should work for all situations is doing #type: ignore for the whole file, after your done with coding (so normal errors still get shown)

polar aurora
#

I think you need extraPaths configured for pylance maybe, to tell it where imgui is?

#

Like, your site-packages path probably?

oblique urchin
mossy rain
hazy rain
#

Hi folks,
I am trying to annotate a function with particular types -

@overload
def func(a: int) -> int = ...
@overload
def func(a: float) -> float = ...

Now the issue here is that an element of type int can be used to construct an element of type float, ie, b = float(int(0)) is valid. Now this above snippet shows me an error saying overloads overlap due to incompatible return types. But I believe that these return types are compatible due to float being constructible from int. Is there any way around without having to use # type: ignore throughout?

oblique urchin
hazy rain
#

Interestingly, if I change this to -

from typing import final, overload, Union

@final
class mpz:
    @overload
    def __add__(self, other: Union[int, "mpz"] ) -> "mpz": ...
    @overload
    def __add__(self, other: Union[float, "mpfr"]) -> "mpfr": ...

    def __add__(self, other):
        return other

@final
class mpfr:
    pass

This seems to work with mypy but pyright (or whatever is the vscode default, i ain't sure) gives the overload overlap error. Note that mpz and mpfr are gmpy2 types represting int and float respectively.

#

Could it be a pyright bug??? 🤔

mossy rain
oblique urchin
hazy rain
mossy rain
oblique urchin
mossy rain
#

?

#

Why then?

oblique urchin
hazy rain
#

Yeah that is an amazing explanation. You guys are superb! joe_salute

floral spruce
#

any tips for bigginers?

#

and some types for biggerner??

feral wharf
#

Wdym "types for beginners" lol

#

Just start annotating your code i guess

grave fjord
#

Don't start annotating without using a checker in CI

mossy rain
feral wharf
grave fjord
feral wharf
#

Why is that necessary tho? Aren't you running the checker locally anyways

grave fjord
#

No people forget

trim tangle
# grave fjord No people forget

I've seen a senior coworker annotate every function lacking a return statement with a -> NoReturn annotation in a project, because PyCharm didn't complain apparently.

#

(that project in particular was lacking mypy in CI for some reason)

polar aurora
#

I always install it as a pre-commit hook.

grave fjord
#

People then forget to run pre-commit in CI

feral wharf
#

Fair

#

Is there any news on ruff's type checker

lunar dune
# feral wharf Is there any news on ruff's type checker

We're hoping to have an alpha release out in a few weeks or so to demonstrate some of its capabilities and features that other tools don't have. But the alpha will still be very far from feature-complete. It'll be at least a few months before it's something approaching parity with mypy or pyright

feral wharf
#

That's understandable. I'll be looking out for the release!

rare scarab
trim tangle
#

like you know, it's not a generator function if it doesn't have yield

rare scarab
#

which is the same as void in other languages

trim tangle
# 
def foo():
    if bar():
        return
    func1()
    func2()
oblique urchin
#

That should be annotated as -> SomeReturn (joking)

trim tangle
# 
type SomeReturn[T] = T | None
#

new replacement for Optional

oblique urchin
#

I've also seen this confusion about the meaning of NoReturn though, another reason to move to Never

rare scarab
#

MaybeReturn

trim tangle
#

I think Python docs are somewhat missing a formal notion of "soft deprecation". Like, X is not recommended anymore, but we won't delete it to break thousands or programs.

rare scarab
#

The Return suffix is actually redundant

#

So -> No

trim tangle
oblique urchin
#

https://discuss.python.org/t/formalize-the-concept-of-soft-deprecation-dont-schedule-removal-in-pep-387-backwards-compatibility-policy/27957

Discussions on Python.org
Formalize the concept of "soft deprecation" (don't schedule removal...

Hi, I would like to specify clearly a new “soft deprecation” status in PEP 387: Backwards Compatibility Policy for functions which should be avoided for new code since these functions will not be developed further, but it’s ok to keep them in old code since their removal is not scheduled. Over the least years, whenever I saw anything mar...

trim tangle
#

ah, it has been discussed

#

I was thinking of %-formatting as an example. There are at least two (three?) newer ways of formatting strings now, but % is not going anywhere, and on occasion can be used

oblique urchin
trim tangle
oblique urchin
#

Yeah he walked away from that

#

I think we ended up with soft deprecated meaning docs-only

oblique urchin
trail kraken
#

Why does the docs require a soft deprecation to say a newer syntax is preferred?

oblique urchin
trail kraken
#

Clearer, like collecting into a "soft deprecation" list of some sort?

trail kraken
pallid swan
#

hello everyone. I am using unittest and mypy together.I find that unittest's self.assertIsInstance cant narrow the type correctly,how can i solve it?

#

like this. I have to write many redundant isinstance to stop the error message.I am using vscode's Mypy Type Checker
extension.

1.png 2.png
terse sky
#

pytest works with the built in assert and gives equally good error messages without remembering all this stuff

bronze juniper
#

Hello, looking for help/advice.
When decorating a function with functools.lru_cache, the signature is lost
I mean:

@lru_cache(None)
def myfunc(arg1: Type1, arg2: Type2):
    ....

becomes

myfunc(*args, **kwargs)

How can we have it back again ?

#

I tried to use functools.update_wrapper but then the cache_clear, cache_info and cache_parameters are not available anymore

#

I simply came up with

@lru_cache(None)
def _myfunc(arg1: Type1, arg2: Type2):
    ...

def myfunc(arg1: Type1, arg2: Type2):
    _myfunc(x, y)

but I am not really satisfied as you need to call _myfunc.cache_clear() instead of myfunc.cache_clear()

#

I could assign myfunc.clear_cache = _myfunc.clear_cache but such dynamic assignements are not recognized by pylance

#

I finally came up with

from functools import lru_cache as _lru_cache
from typing import Any, Callable, ParamSpec, TypeVar

P = ParamSpec("P")
T = TypeVar("T")

def lru_cache(size: int | None = 128, typed: bool = False):

    def cache_wrapper(function: Callable[P, T]) -> Callable[P, T]:

        class CachingWrapper:

            @_lru_cache(size, typed)
            def __call__(self, *args: P.args, **kwargs: P.kwargs) -> T:
                return function(*args, **kwargs)

            def cache_clear(self):
                self.__call__.cache_clear()

            def cache_info(self):
                self.__call__.cache_info()

            def cache_parameters(self):
                self.__call__.cache_parameters()

        return CachingWrapper()

    return cache_wrapper

Which I think have all expected behaviors but Idk it feels too complicated

bronze juniper
#

Now for some reason pylance do not know the cache_clear, cache_info and cache_parameters methods

pallid swan
meager slate
#

Hello, I've recently found that the order in which you overload a function/method actually matters? Especially with ones having default arguments. So I have this method-

    @overload
    async def get_account(
        self, auto_create: Literal[False] = ...
    ) -> None | User: ...

    @overload
    async def get_account(self, auto_create: Literal[True] = ...) -> User: ...

    async def get_account(self, auto_create=True):
        async with BaseData.make_session() as session:
            get_user_query = select(User).where(User.id == self.id)
            result = await session.execute(get_user_query)
            data = result.fetchone()

            if data is None:
                if auto_create:
                    await self.post_account()
                    return await self.get_account()
                return None
            return data[0]

Now even though the default value of auto_create is True, when I try calling the method like so-

        player_data = await player.get_account()

VSC indicates the type of player_data as User | None.

But if I interchange the places of the overloads to like so-

    @overload
    async def get_account(self, auto_create: Literal[True] = ...) -> User: ...

    @overload
    async def get_account(
        self, auto_create: Literal[False] = ...
    ) -> None | User: ...

    async def get_account(self, auto_create=True):
        async with BaseData.make_session() as session:
            get_user_query = select(User).where(User.id == self.id)
            result = await session.execute(get_user_query)
            data = result.fetchone()

            if data is None:
                if auto_create:
                    await self.post_account()
                    return await self.get_account()
                return None
            return data[0]

Everything becomes alright and player_data does indeed turn out to be just User and not User | None

#

I wanna know if this is how overloads are meant to be or is there some bug with vsc or am i tripping hard on smt

trim tangle
#

In your case the correct overload is

    @overload
    async def get_account(self, auto_create: Literal[False]) -> None | User: ...
    @overload
    async def get_account(self, auto_create: Literal[True] = ...) -> User: ...
``` or ```py
    @overload
    async def get_account(self, auto_create: Literal[True] = ...) -> User: ...
    @overload
    async def get_account(self, auto_create: Literal[False]) -> None | User: ...
meager slate
#

ohh

#

i see

#

thanks

trim tangle
#

Now even though the default value of auto_create is True
overloads do not peek at the actual signature, it is considered an implementation detail as far as overload resolution is concerned

cunning plover
#

is there a type for attribute that may or may be not defined (without None)?
Basically smth working for hasattr level of class attribute

#

Optional is supposed to be value that is still defined, it is just going to be None?
Is there smth fitting the case, working with hasattr case?

#

oh well, i can be just using like it has normal attributes

#

and not forgetting to make checks for hasattr 😄 may be even going for just Optional, and remembering to do hasattr instead of check for None

#

optionally NotRequired type could be a nice hint, even if it is not typeddict may be

winter lintel
#

Here's my dilemma: I want a typed data structure with all of the attributes optional, I want it to be fast, I don't want it to eat a lot of memory, and I want to be able to address it without literal keys. TypedDict is failing at this last part

oblique urchin
feral wharf
oblique urchin
#

There have been some discussions around adding some feature like this, but it's difficult to create a sound spec for it

cunning plover
oblique urchin
#

Yes, that sounds like a good strategy

cunning plover
#

a thing to try which one works nicer here

#

ultimately under the python hood arbitary in amount of atttributes class instance is dictionary somewhere inside

#

so question is only which out of those variables will work better

winter lintel
#

sorry, my requirements were wrong, would like for the structure to be non-total, not for the values to be optional, as in attributes not set

cunning plover
winter lintel
#

this was why i was using TypedDict instead of dataclass or NamedTuple

#

that does sound good

#

also need to be able to iterate over the attrs pretty quick

cunning plover
#

and we can't defined types for not set attrs, i guess it will be impossible to type 🤔 unless u define typed getters in it as i noticed just a moment above

feral wharf
#

Are there any docs on this "typed getters" that you're talking about? Sounds interesting

cunning plover
# feral wharf Are there any docs on this "typed getters" that you're talking about? Sounds int...

https://www.geeksforgeeks.org/getter-and-setter-in-python/
getters in python are stuff like this

class Geeks: 
    def __init__(self): 
        self._age = 0
    
    # using property decorator 
    # a getter function 
    @property
    def age(self): 
        print("getter method called") 
        return self._age 
    
    # a setter function 
    @age.setter 
    def age(self, a): 
        if(a < 18): 
            raise ValueError("Sorry you age is below eligibility criteria") 
        print("setter method called") 
        self._age = a

U just add to them types as from
https://docs.python.org/3/library/typing.html

#

We can inherint SimpleNamespace (highly likely)

#

and define getters to it

feral wharf
#

Ahh I see. Thank you!

winter lintel
#

!e

from typing import Iterator

class PropStruct:
    __slots__ = (
        "a",
        "b",
    )
    a: int
    b: str

    def has(self, name: str) -> bool:
        """Check if the attribute exists."""
        return hasattr(self, name)

    def __iter__(self) -> Iterator[tuple[str, int | str]]:
        """Iterate over the attributes."""
        for name in self.__slots__:
            if hasattr(self, name):
                yield name, getattr(self, name)


s = PropStruct()
print(s.has("a"))
s.a = 1
s.b = "test"
print([(k, v) for k, v in s])
rough sluiceBOT
winter lintel
#

mypy doesn't choke on it, might work 👍

#

iter is probably gonna be really slow unless i do some more work

#

and mypy probably gonna have issues with iter output

#

feeling like going down this road gonna be expensive, when there's 29 possible attrs and usually 0 are set..

#

maybe i'll just make a kwarg factory for the TypedDict for users

#

and then maybe just use this approach for the other end.. it's message properties for a networking lib

#

and keep using TypedDict internally

cunning plover
#

it is just Dictionary with wrapping to have attrs accessed in class way

#

ultimate if u have 29 Possible attrs which are may or may not defined, i think hashmap does have point but

#

but... to be fair i prefer always having defined explicitely Data Structs 😄

#

There is always a room to optimize memory usage in other ways than hacking hashmaps

#
  1. using smartly bufferization principles usually. making sure that at the moment of a time in a program at maximum N instances of smth
    Each time when it reached N, the buffer is flushed, to free the memory
    and when u finished working with big set of data, u call flush one last time (it can be done automatically in python by context manager)
winter lintel
#

thing is i don't mind so much about memory and cost for the end user interface, internally i'll probably always just use dict

cunning plover
#
  1. just using actually manual GC to clean up memory faster if necessary
winter lintel
#

user might have 1 or 2 of these alive at a time, library might have 65000

cunning plover
# winter lintel user might have 1 or 2 of these alive at a time, library might have 65000
  1. Also when u work with relational databases, u can be not storing all data at the same time in memory
    U can be using cursors and reading chunks in portions and handing data as u go i think?
    At least i used this trick in django orm with queryset.iterator
    https://docs.djangoproject.com/en/5.1/ref/models/querysets/#iterator
Django Project
QuerySet API reference | Django documentation

The web framework for perfectionists with deadlines.

#

it allowed me handling data memory well even if having half of millions objects in the query without memory problems

#

may be your database engine (if u are going to do it with databases) supports similar feature / and code logic fits such thing
same as bufferization but at db engine client basically

winter lintel
#

i don't expect the persistence will be anything fancier than sqlite, just persisting state of MQTT session

winter lintel
#

ended up using a dataclass with optional fields on the user side, shipping it

stray summit
#

anyone aware of a way to blend typeddict and normal dicts - i'm sorting out some legacy code that shoehorns some data into a different mapping and i'd like to defer refactoring it to something correct without foregoing better typing

a minimal example would be something like

Data = dict[str, DataItem]
class ErrorMess(TypedDict, total=False):
    _errors_parsing: list[ErrorRecord]

DWIM = magic[Data, ErrorMess]
feral wharf
#

I wonder if intersections will support that

terse sky
#

on the dataclass

#

if you can use that internally, then that will save memory relative to a dict

grave fjord
#

It's not supported in mypy yet

winter lintel
# terse sky btw, you mentioned you care about memory usage - so you should also use slots=Tr...

Because I need fast iteration over non-empty attrs (for encoding) I really need to have access to a dict in the library. That same iteration is required to create a TypedDict from the dataclass. When I benchmarked this, a non-slots dataclass was much much faster to produce a dict, because I don't have to call getattr twice to check for the None sentinel value 29 times (which is required to solve this problem with a stock dataclass, which is always total). It was also faster than a custom non-total slotted class checking with hasattr.

Maybe there's some kind of slotted class I can make that's faster than non-slotted dataclass, which has all the iteration performance and typing I need, but I haven't found it yet. Using more memory (by a lot! but never retained by the library) and marginal cost (only when this optional feature is used) to complete the typing story in the user interface seems like a pretty good win over forcing users to use TypedDict which has some obnoxious limitations.

This is the only dataclass use case in this project where slots weren't preferable!

#

empty TypedDict is 64B
slotted dataclass with 29 None is 264B
non-slotted dataclass with 29 None is 2048B

terse sky
#

is there no way to iterate a slots dict?

#

also, a named tuple should be both fast and have fast iteration

#

sorry, should both be small and have fast iteration

#

I think, at any rate

winter lintel
#

how would you iterate over a slots dataclass without iterating over __slots__?

terse sky
#

fields

winter lintel
#

sure, but practically the same thing, you get a list of attr names and you need to check if the value is None

#

and need to use getattr to do anything in that iteration

#

non-slotted dataclass:

return {key: value for key, value in self.__dict__.items() if value is not None}

slotted dataclass:

return {key: getattr(self, key) for key in self.__slots__ if getattr(self, key) is not None}

custom slotted class:

return {key: getattr(self, key) for key in self.__slots__ if hasattr(self, key)}
winter lintel
#

i may still eliminate the TypedDict and just use the dataclass, since it's just gonna get encoded to bytes up front shrug

terse sky
#

but I admit these kinds of microoptimizations generally aren't on my radar

slender timber
cosmic plinth
#

I have what is equivalent to the following code:

if sys.platform == "win32":
    from my_pkg._pty.windows import PtySession
else:
    from my_pkg._pty.unix import PtySession

On my current platform Windows, Mypy continues to perform checking inside my_pkg._pty.unix and when I import PtySession from this module going to the declaration in VS Code for some reason takes me to the implementation inside my_pkg._pty.unix. Why is this happening?

tranquil turtle
#

pyright is what provides "go to definition" features in vs code

#

maybe something about it is misconfigured

cosmic plinth
#

Mypy is behaving in the same (incorrect?) manner as well

grave fjord
cosmic plinth
grave fjord
#

If you do;

WINDOWS: Final = sys.platform == "win32"
if WINDOWS: ...
#

It doesn't work as you night expect

tranquil turtle
#

btw mypy supports treating some vars as always true, so you can do if WINDOWS:... else:... and mypy --always-true=WINDOWS (or something like that) on windows and mypy --always-false=WINDOWS on non-windows
I think that would work, but that is clunky

grave fjord
#

It's annoying it won't propagate them to other finals

stiff acorn
# 
class Foo:
    num: int

Is it possible to type that num will always be greater than or equal to 1

#

it can't be negative, it can't be 0

trim tangle
#

why though? and what would you do with that information?

polar aurora
#

(Could a TypeIs do this?)

restive rapids
#

you could make a "newtype wrapper" int subclass but it would just be a separate symbol and the typechecker couldnt really do anything with it
i guess you could override __gt__(Literal[0]) -> Literal[True]

feral wharf
#

int: Literal[0,1,2,3,4,5,6,7,8,9,10,<all numbers in the world>] Kappa

mossy rain
#

maybe a range(1, max_int) would work?

trim tangle
#

No, a range object is not valid as a type

feral wharf
#

Imagine

stiff acorn
#

If it's not possible then int does the job too

cosmic plinth
grave fjord
oblique urchin
#

It's important because type checkers recognize only specific patterns

cosmic plinth
#

I haven't pushed my branch yet but the full file is dda.utils.platform._pty.session:

from __future__ import annotations

import sys

if sys.platform == "win32":
    from dda.utils.platform._pty.windows import PtySession
else:
    from dda.utils.platform._pty.unix import PtySession

__all__ = ["PtySession"]

and other files only import from this file that exposes the PtySession. dda.utils.platform._pty.__init__ is empty, and the only part of my code that imports that is dda.utils.process:

class SubprocessRunner:
    def __run(self, ...) -> tuple[int, str]:
        from dda.utils.platform._pty.session import PtySession
#

isn't that weird/unexpected behavior?

cosmic plinth
oblique urchin
#

I guess the thing that comes to mind is that you run with something like a --platform argument

cosmic plinth
oblique urchin
#

config can also be in mypy.ini/pyproject.toml

#

(also unrelated but --install-types is evil)

cosmic plinth
# oblique urchin config can also be in mypy.ini/pyproject.toml 
[tool.mypy]
disallow_untyped_defs = true
disallow_incomplete_defs = true
enable_error_code = ["ignore-without-code", "truthy-bool"]
follow_imports = "normal"
ignore_missing_imports = true
pretty = true
show_column_numbers = true
warn_no_return = true
warn_unused_ignores = true

[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false
check_untyped_defs = false
no_implicit_optional = false
strict_optional = false
warn_return_any = false
warn_unused_ignores = false
# 
❯ hatch run types:uv pip freeze | rg mypy
Using Python 3.12.7 environment at: C:\Users\ofek\AppData\Local\hatch\env\virtual\dda\6xDqlXVy\types
mypy==1.15.0
mypy-extensions==1.1.0
feral wharf
#

Is there anyway to overload *args: Enum?

warm topaz
#

Why does this work for pyright

  • pyright
from __future__ import annotations

import typing as t

T = t.TypeVar("T", default=str)

class A(t.Generic[T]):
    @t.overload
    def __new__(cls, *, null: t.Literal[True]) -> A[T | None]:
        ...

    @t.overload
    def __new__(cls, *, null: t.Literal[False] = False) -> A[T]:
        ...

    def __new__(cls, *, null: bool = False) -> A[T] | A[T | None]:
        return super().__new__(cls)
    
    def __init__(self, *, null: bool = False) -> None:
        super().__init__()


a = A(null=False)
reveal_type(a)
b = A(null=True)
reveal_type(b)
c = A()
reveal_type(c)

c:\Users\Triyan Mukherjee\VsCodeProjects\imc\test.py
  c:\Users\Triyan Mukherjee\VsCodeProjects\imc\test.py:63:13 - information: Type of "a" is "A[str]"
  c:\Users\Triyan Mukherjee\VsCodeProjects\imc\test.py:65:13 - information: Type of "b" is "A[str | None]"
  c:\Users\Triyan Mukherjee\VsCodeProjects\imc\test.py:67:13 - information: Type of "c" is "A[str]"
0 errors, 0 warnings, 3 informations

but not for mypy

  • mypy
test.py:54: error: "__new__" must return a class instance (got "A[T] | A[T | None]")  [misc]
test.py:63: note: Revealed type is "test.A[builtins.str]"
test.py:65: note: Revealed type is "test.A[builtins.str]"
test.py:67: note: Revealed type is "test.A[builtins.str]"

the __new__ overload and return type both seem to be inferred weirdly

warm topaz
#

maybe try out enum.IntFlag with | and ~& for multiple options or settings depending on your use case

humble quail
#

i was wondering why json.loads returns Any type instead of some better typing?

for example something like

Json = dict[str, Json] | list[Json] | str | int | float | bool | None
rare scarab
#

Plus annotating with a typeddict you expect it to be is usually good enough. Other libraries can provide validation like pydantic

winter lintel
#

i was wrong about dataclass, it doesn't have to be total! you just write your own init and conditionally set the attrs

rare scarab
#

Conditionally setting attributes will lead to AttributeErrors

terse sky
#

And improving the type would be a pretty big breaking change - I'm actually dealing with that at work now

#

And probably the value isn't high enough, as unalive said

#

Typically people should anyway be pumping the json into a data class and doing validation there and then you have the data class which is more strictly typed

cosmic plinth
winter lintel
rare scarab
winter lintel
#

what would i use?

rare scarab
#

A type that doesn't define those attrs

winter lintel
#

the thing i want is a typed data structure that can quickly iterate over all non-default key: value pairs, and not storing the default values in an internal dict

rare scarab
#

Default value instances are shared

winter lintel
#

they are shared when they are __slots__

rare scarab
#

No, always

#

Anyway, this is #type-hinting and typing doesn't support a "maybe defined" attribute.

#

Outside of TypedDict of course.

winter lintel
#

is this the most cursed thing that could possibly do what i want?

class PropertiesDict(TypedDict, total=False):
    one: int
    two: str
    three: bytes
    four: float

@dataclass(init=False, repr=False)
class PropertiesDataclass:
    one: int
    two: str
    three: bytes
    four: float

    def __init__(
        self,
        one: int | None = None,
        two: str | None = None,
        three: bytes | None = None,
        four: float | None = None,
    ) -> None:
        if one is not None:
            self.one = one
        if two is not None:
            self.two = two
        if three is not None:
            self.three = three
        if four is not None:
            self.four = four

    def __getattr__(self, name: str) -> Any:
        return self.__dict__.get(name, None)

    def __repr__(self) -> str:
        return f"PropertiesDataclass{self.__dict__}"

    def to_dict(self) -> PropertiesDict:
        return cast(PropertiesDict, self.__dict__)
rare scarab
#

Don't do that.

#

!d dataclasses.asdict this also exists

rough sluiceBOT
#
dataclasses.asdict 

dataclasses.asdict(obj, *, dict_factory=dict)```
Converts the dataclass *obj* to a dict (by using the factory function *dict\_factory*). Each dataclass is converted to a dict of its fields, as `name: value` pairs. dataclasses, dicts, lists, and tuples are recursed into. Other objects are copied with [`copy.deepcopy()`](https://docs.python.org/3/library/copy.html#copy.deepcopy).

Example of using `asdict()` on nested dataclasses...
winter lintel
#

thing is i'd rather incur the cost of filtering out None in __getattr__ than when iterating over all the key value pairs

rare scarab
#

Might as well use types.SimpleNamespace

winter lintel
#

is there a way to get that to type check, and reject unspecified attrs? this passes mypy strict:

class PropertiesNamespace(SimpleNamespace):
    one: int
    two: str
    three: bytes
    four: float

    def __repr__(self) -> str:
        return f"PropertiesNamespace{self.__dict__}"

    def to_dict(self) -> PropertiesDict:
        return cast(PropertiesDict, self.__dict__)

PropertiesNamespace()
PropertiesNamespace(one=1)
PropertiesNamespace(one=1, two="two")
PropertiesNamespace(one=1, two="two", three=b"three")
PropertiesNamespace(one=1, two="two", three=b"three", four=4.0)
PropertiesNamespace(one=1, two="two", three=b"three", four=4.0, five=5.0)
rare scarab
# 
import types
import typing

@typing.dataclass_transform()
class Namespace(types.SimpleNamespace): pass


class PropertiesNamespace(Namespace):
  one: int | None
#

Make a base class decorating with @typing.dataclass_transform()

winter lintel
#

it's also inexplicably slower to render the dict using the same method as the cursed dataclass

rare scarab
#

Why are you rendering the repr so often?

#

!d types.SimpleNamespace

rough sluiceBOT
#
types.SimpleNamespace 

class types.SimpleNamespace```
A simple [`object`](https://docs.python.org/3/library/functions.html#object) subclass that provides attribute access to its namespace, as well as a meaningful repr.

Unlike [`object`](https://docs.python.org/3/library/functions.html#object), with `SimpleNamespace` you can add and remove attributes.

[`SimpleNamespace`](https://docs.python.org/3/library/types.html#types.SimpleNamespace) objects may be initialized in the same way as [`dict`](https://docs.python.org/3/library/stdtypes.html#dict): either with keyword arguments, with a single positional argument, or with both. When initialized with keyword arguments, those are directly added to the underlying namespace. Alternatively, when initialized with a positional argument, the underlying namespace will be updated with key-value pairs from that argument (either a mapping object or an [iterable](https://docs.python.org/3/glossary.html#term-iterable) object producing key-value pairs). All such keys must be strings...
rare scarab
#

NamedTuple can also be useful for immutable data. It's much faster than dataclasses

winter lintel
#

i put @dataclass_transform() on it and it still passes type check, nothing is jumping out at me in the docs as a way to fix that?

rare scarab
#

Works fine for me.

# 
import types
import typing

@typing.dataclass_transform()
class Namespace(types.SimpleNamespace): pass


class PropertiesNamespace(Namespace):
  one: int | None


PropertiesNamespace(one="one") # Argument of type "Literal['one']" cannot be assigned to parameter "one" of type "int | None" in function "__init__"
winter lintel
#

oh ok, put it on a parent class

rare scarab
#

works in mypy too

winter lintel
#

i get this though

bench/prop_struct.py:71: error: Missing positional arguments "one", "two", "three", "four" in call to "PropertiesNamespace"  [call-arg]
bench/prop_struct.py:72: error: Missing positional arguments "two", "three", "four" in call to "PropertiesNamespace"  [call-arg]
bench/prop_struct.py:73: error: Missing positional arguments "three", "four" in call to "PropertiesNamespace"  [call-arg]
bench/prop_struct.py:74: error: Missing positional argument "four" in call to "PropertiesNamespace"  [call-arg]
bench/prop_struct.py:76: error: Unexpected keyword argument "five" for "PropertiesNamespace"  [call-arg]
#

with kw_only_default it causes "Missing named argument" instead

rare scarab
#

!e this should work. ```py
import types
import typing

@typing.dataclass_transform()
class Namespace(types.SimpleNamespace): pass

class PropertiesNamespace(Namespace):
one: int | None = None

PropertiesNamespace(one=1)
_ = PropertiesNamespace()
print()
print(.one)
_ = PropertiesNamespace(one=1)
print()
print(.one)```

rough sluiceBOT
rare scarab
#

Note that PropertiesNamespace().__dict__ is empty

winter lintel
#

dang, that is super fast and good

#

i think it's exactly what i need

rare scarab
#

Also experiment with namedtuple

winter lintel
#

@rare scarab thanks, that's worth about 14% more throughput when properties are used 😂

winter lintel
#

except it doesn't work in Python 3.10 agony

#

i think it's worth the typing_extensions

terse sky
#

I get that using another language is annoying but dang it kind of hurts to see this level of micro optimization in python when simply writing the code in any mainstream statically typed language will give you a 10x speedup

#

(or more)

winter lintel
#

i don't think using another language is annoying, i just want to have the fastest pure python implementation

#

and this SimpleNamespace trick is really very clean

#

most of the optimizations i'm doing are resulting in cleaner python, because the less python you write the faster it runs

#

in 3.13 the differences are more marginal than 3.10, so things are moving in the right direction

#

in fact pybind11 is really very pleasant

cosmic plinth
#

follow-up from last night with more context:
I have what is equivalent to the following code:

if sys.platform == "win32":
    from my_pkg._pty.windows import PtySession
else:
    from my_pkg._pty.unix import PtySession

On my current platform Windows, Mypy continues to perform checking inside my_pkg._pty.unix and when I import PtySession from this module going to the declaration in VS Code for some reason takes me to the implementation inside my_pkg._pty.unix. Why is this happening? I can reproduce this on CI.

terse sky
#

Pybind is nice but mixing languages is significantly more involved than not is what I meant

cosmic plinth
#

I'm positive this a bug and VS Code's pylance has the same issue, it makes no sense otherwise

terse sky
#

In that sense I don't actually find this that surprising

cosmic plinth
#

they do, it works in other circumstances

terse sky
#

Gotcha, fair enough then

#

Can you give an example where it works out of curiosity?

cosmic plinth
grave fjord
#

this program should typecheck, but I get:

main.py:49: error: An overloaded function outside a stub file must have an implementation  [no-overload-impl]
main.py:65: error: Name "walk_breadth_first" already defined on line 49  [no-redef]
Found 2 errors in 1 file (checked 1 source file)
#

but the thing is, if I delete the first function; walk_depth_first - this issue goes away

cosmic plinth
winter lintel
tranquil turtle
grave fjord
rare scarab
#

imo overloads inside a type_checking block shouldn't need an implementation, just be defined or imported somewhere in the file.

#

It makes you need a reference to typing.overload at runtime

humble quail
#

is there a better way to type annotate a function that accepts a string of certain length than just str ?

rare scarab
#

try using the annotated-types package

#

!pep 746 also

rough sluiceBOT
rare scarab
#

!pep 736 didn't realize this got rejected.

rough sluiceBOT
terse sky
rare scarab
#

syntactic sugar? In MY language? It's more likely than you think.

cosmic plinth
grave fjord
pliant vapor
#

anyone know of a way to make this work properly with basedpyright?

from typing import TypedDict
import httpx

class Params(TypedDict):
    foo: str
    bar: int

params: Params = {
    'foo': '',
    'bar': 1,
}
httpx.get('/', params=params)

right now basedpyright is giving me

Argument of type "Params" cannot be assigned to parameter "params" of type "QueryParamTypes | None" in function "get"

on the last line

#

i would paste this here but idk how to copy it

Screenshot_2025-05-02_at_22.55.51.png
trim tangle
#

mostly because typeddicts are allowed to have extra items:

class Params1(TypedDict):
    foo: str
    bar: int

class Params2(TypedDict):
    foo: str
    bar: int
    baz: socket.socket

def f(p: Params1):
    ...

def g(p: Params2):
    f(p)  # allowed
#

There's a new PEP728 that allows one to create closed typeddicts (which is what most people want most of the time probably, but for some reason has not been added yet)

#

However, pyright still doesn't think that Params is assignable to Mapping[str, str | int] when Params is closed=True. I don't know if it's a bug or not. You can file an issue on the pyright repo when PEP728 is accepted

upper vortex
#

Can anyone help me ignore certain warnings from basedpyright?

Diagnostics:
1. Return type is Any [reportAny]
2. Type `Any` is not allowed [reportExplicitAny]
#

This is the default config I used from the basedpyright website.

require("lspconfig").basedpyright.setup {
  settings = {
    basedpyright = {
      analysis = {
        diagnosticMode = "openFilesOnly",
        inlayHints = {
          callArgumentNames = true
        },
      }
    }
  }
}
vernal agate
#

[Resolved] #type-hinting message
What's the correct way to bind the generic type of Node to its parent class DoublyLinkedList? I've read in mypy docs that the same type var T used in nested classes does not mean its type has to follow the parent T, but there's nothing saying why or how to make this happen.

from __future__ import annotations

from typing import Generic, TypeVar

T = TypeVar("T")

class DoublyLinkedList(Generic[T]):
    class Node(T):
        value: T

        def __init__(self, value: T):
            self.value = value
            self._prev: DoublyLinkedList.Node = None
            self._next: DoublyLinkedList.Node = None

    head: Node[T] = None
    tail: Node[T] = None
trim tangle
vernal agate
#

yup that's what i meant to do

oblique urchin
trim tangle
#

ah hmm

oblique urchin
#

But this may be easier if you don't make it a nested class and make it independently generic

trim tangle
terse sky
#

yeah I definitely wouldn't nest classes

oblique urchin
#

pyright accepts ```class DoublyLinkedList[T]:
class Node:
value: T

trim tangle
terse sky
#

this sort of thing works differently in different languages, as well

#

If someone is used to C++ or Rust, vs if they're used to Java or Kotlin, they will expect very different behavior from the nested class

#

in python if you don't want anyone except DoublyLinkedList to use Node, you would just define it as _Node as top level and call it a day

vernal agate
terse sky
#

yeah, for sure

#

just use module/package level encapsulation

#

_Node is the idiomatic way to do this

#

really, _Node is "more" encapsulated than what you wrote, because anybody can use that Node type anyway and you haven't indicated that it's an implementation detail with a leading underscore

trim tangle
#

Alternatively you can use two different type variables ```py
class DoublyLinkedList[T]:
class _Node[U]:
value: U

    def __init__(self, value: U):
        self.value = value
        self._prev: DoublyLinkedList._Node[U] | None = None
        self._next: DoublyLinkedList._Node[U] | None = None

head: _Node[T] | None = None
tail: _Node[T] | None = None
vernal agate
#

does this force T and U to be the same type?

trim tangle
#

essentially you're defining two standalone classes and then doing DoublyLinkedList._Node = _Node

terse sky
#

this is how it works in Java/Kotlin, fwiw

#

that's kind of what I expected tbh

#

this is the more "type erasure" kind of model

trim tangle
#

I thought there was some impact on access to private fields in Java

terse sky
#

there is, I'm just talking about the generic aspect of it

#

in Java/Kotlin nested classes of generic classes don't have any kind of menaingful access to the type parameter

#

so if you were doing something similar to this, this is how you would do it

trim tangle
#

!e
actually, I wonder

class Foo:
    class Bar:
        def __init__(self):
            self.__baz = 42

bar = Foo.Bar()
print(bar.__dict__)
rough sluiceBOT
trim tangle
#

makes sense

vernal agate
#

thanks for the input, this seems to work for me:

from __future__ import annotations

from typing import Generic, TypeVar

T = TypeVar("T")

class Node(Generic[T]):
    value: T

    def __init__(self, value: T):
        self.value = value
        self._prev: Node[T] | None = None
        self._next: Node[T] | None = None


class DoublyLinkedList(Generic[T]):
    head: Node[T] | None = None
    tail: Node[T] | None = None
terse sky
#

Like, can you access _Node as DoublyLinkedList[int]._Node ? is that even valid syntax?

trim tangle
terse sky
#

!e

class Foo[T]:
    class Bar:
        pass
print(Foo[int].Bar)
rough sluiceBOT
terse sky
#

interesting, it's allowed, but I wonder if type checkers will actually distinguish

trim tangle
terse sky
#

is there a way to run mypy jobs here

trim tangle
terse sky
#

!mypy

trim tangle
#

we do not have mypy or pyright playgrounds

terse sky
# 
class Foo[T]:
    class Bar:
        pass

x: Foo[int].Bar = Foo[float].Bar()
#

mypy doesn't even like this as a type annotation

trim tangle
#

I was thinking of making a side-by-side type checker thing, but I'm too lazy

#

the UX of my pyright playground was so bad that microsoft had to make their own

terse sky
#

at any rate I haven't fully understood how it works but yeah the bottom line I think is just to really really avoid nested classes in python.
in the era of type checking more than ever

winter lintel
#

trying to make a decorator for a class instance method, what's wrong with this picture?

P = typing.ParamSpec("P")
R = typing.TypeVar("R")

def wrapper(
    func: typing.Callable[typing.Concatenate[C, P], R],
) -> typing.Callable[typing.Concatenate[C, P], R]:
    """A decorator to wrap a function with additional functionality."""
    @functools.wraps(func)
    def wrapped(c: C, *args: P.args, **kwargs: P.kwargs) -> R:
        print(f"Wrapped function called with {c}")
        return func(c, *args, **kwargs)
    return wrapped

class C:
    @wrapper
    def f(self, a: int, b: str) -> None:
        print(f"Function f called with {a}, {b}")
# 
bench/wrapper.py:17: error: Incompatible return value type (got "_Wrapped[[C, **P], R, [C, **P], R]", expected "Callable[[C, **P], R]")  [return-value]
bench/wrapper.py:17: note: "_Wrapped[[C, **P], R, [C, **P], R].__call__" has type "Callable[[Arg(C, 'c'), **P], R]"
Found 1 error in 1 file (checked 1 source file)
#

friend or footgun?

R = typing.TypeVar("R")

def wrapper(
    func: typing.Callable[..., R],
) -> typing.Callable[..., R]:
    """A decorator to wrap a function with additional functionality."""
    @functools.wraps(func)
    def wrapped(c: C, *args: typing.Any, **kwargs: typing.Any) -> R:
        print(f"Wrapped function called with {c}")
        return func(c, *args, **kwargs)
    return wrapped

class C:
    @wrapper
    def f(self, a: int, b: str) -> None:
        print(f"Function f called with {a}, {b}")
rare scarab
#

Footgun.

#

make the typevar bound Callable[..., Any]

# 
TFunc = TypeVar("TFunc", bound=Callable[..., Any])

def wrapper(func: TFunc) -> TFunc:
  @functools.wraps(func)
  def wrapped(*args: Any, **kwargs: Any) -> Any:
    print(f"Wrapped function called with {args[0]}")
    return func(*args, **kwargs)
  return wrapped

class C:
  @wrapper
  def f(self, a: int, b: str) -> None: ...

reveal_type(C.f) # C.f is (self: C, a: int, b: str) -> None
reveal_type(C().f) # C().f is (a: int, b: str) -> None
#

With your solution, C.f would be (c: C, *args: Any, **kwargs: Any) -> None

#

C().f would be (*args: Any, **kwargs: Any) -> None

trim tangle
rare scarab
#

Is types.FunctionType generic now?

#

or would it not need to be in this instance?

trim tangle
#

ParamSpec is the right approach, but mypy might be confused about methods

rare scarab
#

This was the way to go before paramspec.

#

also paramspec involves more boilerplate, and if you're extracting the first argument, you also have to use Concatenate

#

but I don't think that works with P.args

#

Unless...

trim tangle
rare scarab
# 
import functools
from typing import Callable, Concatenate, TypeVar, ParamSpec
C = TypeVar("C")
R = TypeVar("R")
P = ParamSpec("P")

def wrapper(func: Callable[Concatenate[C, P], R]) -> Callable[Concatenate[C, P], R]:
  @functools.wraps(func)
  def wrapped(self: C, *args: P.args, **kwargs: P.kwargs) -> R:
    return func(self, *args, **kwargs)
  return wrapped
trim tangle
#

wtf, I'm only getting an error in strict mode

rare scarab
#

Add a / after self

trim tangle
#

Oh right

rare scarab
trim tangle
#

Yeah, I just realized that as well

#

the error is extremely unhelpful

# 
main.py:15: error: Incompatible return value type (got "_Wrapped[[C, **P], R, [C, **P], R]", expected "Callable[[C, **P], R]")  [return-value]
main.py:15: note: "_Wrapped[[C, **P], R, [C, **P], R].__call__" has type "Callable[[Arg(C, 'c'), **P], R]"

things wrong with this error:

  • unclear what _Wrapped is and what module it's even from
  • Arg is some internal mypy entity whose representation is not explained
#

well, 1 is a minor complaint

polar aurora
#

It's weird to see it talk about Arg though, given that I thought that was internal

trim tangle
#

What's missing from this error is a link to some explanation on assignability of callables, specifically that you need to remember how keyword arguments work. If you have a function like py class SomeClass def foo(self, *args, **kwargs): ... you cannot call SomeClass().foo(bar=1, baz=2, self=42) because the self argument (which is both positional and keyword) is already supplied. But this would accept a self kwarg: ```py
class SomeClass
def foo(self, /, *args, **kwargs): ...

#

The wrapper function promises that it will transform a function with any shape whatsoever, including those shapes that expect c as a keyword argument. So it must not reserve any particular name for the first parameter

#

In retrospect, Callable[[Arg(C, 'c'), **P], R] is not assignable to Callable[[C, **P], R] is kind of saying that, but in a way where you need to know the above to understand it

rough kettle
#

is it possible to specify "I'll return the same size of tuple you gave me but with these types instead" with the current typing spec?

#

while also specifying that the input tuple must hold values of some type

rare scarab
#

No mapping types yet

#

What's your impl look like?

rough kettle
# 
def exclusive_true(
    *set_funcs: Callable[[bool], None]
) -> tuple[Callable[[bool], None], ...]:
    return tuple(
        (
            lambda pressed: (
                None
                if not pressed
                else consume(func(i == my_index) for i, func in enumerate(set_funcs))
            )
        )
        for my_index in range(len(set_funcs))
    )

It's basically a function that when a radio button is set to true, it sets all the others false

#

it returns the functions you can then pass as button press event handlers

#

consume here is more_itertools.consume

#

I was thinking of using TypeVarTuple, but that doesn't allow you to bind the values to some type

#

so it'd just accept any tuple

rare scarab
#

Current solution is to use overloads

rough kettle
#

for each number of arguments?

rare scarab
#

Yes

rough kettle
#

ugh

#

I'd rather just leave it as is lol

winter lintel
winter lintel
#

since it's meant to be a base class

#

it's a class that lets you acquire its lock in a with block and use a decorator to guarantee a method is called with that lock

#

and when applying it to thread-safe code, i find all the places i wasn't locking access to resources 😂

rare scarab
green shoal
#

I'm trying to build some type checking for my config parser. It's relatively simple, but I'm wondering how I should get this type checked properly:

from typing import TypedDict

class InnerConfigBase(TypedDict):
  foo: str

class ConfigBase(TypedDict):
  inner: InnerConfigBase

class ExpandedInnerConfig(InnerConfigBase):
  bar: str

class Config(ConfigBase):
  inner: ExpandedInnerConfig

To me it feels this should work, but mypy complains about overwriting the inner field in Config.

oblique urchin
green shoal
#

I've just tried adding it, and that doesn't appear to change anything

#

I don't know too much about type hinting though 😅

oblique urchin
#

That seems to be a bug in mypy

rare scarab
#

Using a generic over InnerConfigBase may be better.

# 
class InnerConfigBase(TypedDict):
  foo: str

class ConfigBase[T: InnerConfigBase](TypedDict):
  inner: T

class ExpandedInnerConfig(InnerConfigBase):
  bar: str

class Config(ConfigBase[ExpandedInnerConfig]): pass
#

unless your config parser doesn't support generics this way.

green shoal
#

It probably will, I'll try that out

#

Thanks

undone birch
#

How do a I do a generic type where it could be a list, dict, or any json value, and I process it different for each type? (simplifying my code a little bit)

def diff_json_data[T](old: T, new: T) -> tuple[T, T]:
    if isinstance(old, dict) and isinstance(new, dict):
        olddiff_dict: T = {}
        newdiff_dict: T = {}
        for key in list(old.keys()) + list(new.keys()):
            diff_value = diff_json_data(old.get(key), new.get(key))
            if diff_value[0] or diff_value[1]:
                olddiff_dict[key], newdiff_dict[key] = diff_value
        return olddiff_dict, newdiff_dict
    if isinstance(old, list) and isinstance(new, list):
        olddiff: T = []
        newdiff: T = []
        oldstart, newstart, oldend, newend = 0, 0, len(old) - 1, len(new) - 1
        while oldstart <= oldend or newstart <= newend:
            diff_value = diff_json_data(
                old[oldstart] if oldstart <= oldend else None,
                new[newstart] if newstart <= newend else None,
            )
            if diff_value[0] or diff_value[1]:
                olddiff.append(diff_value[0])
                newdiff.append(diff_value[1])
            oldstart += 1
            newstart += 1
        return olddiff, newdiff
    if old != new:
        return old, new
    return None, None
pliant vapor
#

is there a way to use typing.overload with contextlib.contextmanager? something like

from collections.abc import Iterator
from contextlib import contextmanager
from typing import overload

@overload
def f(x: int) -> Iterator[int]: ...

@overload
def f(x: str) -> Iterator[str]: ...

@contextmanager
def f(x: int | str) -> Iterator[int | str]:
    yield x

with f(1) as y:
    print(y)

doesn't seem to work

oblique urchin
pliant vapor
#

ah thx

austere pebble
#

I like the type anootations since they provide me better understanding via editor supoort for arguments and their types. I am feeling that I am using it wrong.
I have a seperate folder named typing for type annotation imported all over the application. Can you please provide useful suggestions to better approach for using type annotation in my project.
Project: https://github.com/sparrowsurya/whisper

GitHub
GitHub - SparrowSurya/whisper: A chat application in python.

A chat application in python. Contribute to SparrowSurya/whisper development by creating an account on GitHub.

cinder bone
feral wharf
#

Wooooe

tranquil turtle
oblique urchin
keen flicker
tranquil turtle
#

having X.typing submodule is a common practice

austere pebble
trim tangle
tranquil turtle
#

wasn't there a x: str @ doc('docstring for x') proposal? I don't see it being mentioned in the pep 727

grave fjord
#

I want to add a typeshed TypedDict type for asyncio.Task's kwargs (available at loop.create_task, asyncio.create_task and asyncio.TaskGroup.create_task somewhat on 3.13.3 and completely on 3.14b1)

how do I add it so it's available to typeshed to import but it's private outside typeshed?

rare scarab
#

Name it _TaskArgs

#

or put it in a private module

#

i.e. in _typeshed

grave fjord
#

if I call it asyncio.tasks._TaskKwargs can I still import it in asyncio.base_events ?

rare scarab
#

Put it in asyncio._types

grave fjord
#

oh cool

rare scarab
#

If only we had a ParamsOf(func) utility

grave fjord
#

well create_task isn't the same params of asyncio.Task

#

loop and coro are different

rare scarab
#

base class?

grave fjord
indigo halo
#

How does pyright conclude there's a positional argument missing??

rare scarab
#

You're missing kw_only=True

#

or...

#

Callable[] syntax doesn't know about kwargs.

indigo halo
rare scarab
#

no, ignore that.

indigo halo
#

I think you are onto something actually

#

**rec is not [RecordType]

rare scarab
#

Try using Calable[[Unpack[RecordType]], T]

indigo halo
#

Callable[..., T]

#

oh, Unpack!

trim tangle
#

Yeah, you might want cls: Callable[..., T] | None = None

#

I'm not sure Unpack is legal with Mapping

rare scarab
#

A limitation of the current typing system is we can't separate ParamSpec.args from ParamSpec.kwargs

trim tangle
#

Since this is impossible to strictly type, might as well do the YOLO option (... in Callable)

trim tangle
# indigo halo

Do you have an example of how you'd use this? Maybe there's a type-safer way

rare scarab
#

Guard the types at creation time

#

Use a factory function.

indigo halo
#

yeah

#

thanks so much, this was very helpful. I stared at this 2 hours and you guys took 5 minutes 😉

rare scarab
#

typing is the foxhole of python

#

maybe even a rabbit hole

#

And at the bottom are punji sticks

indigo halo
#

the good thing is that it remains optional. Compare to C++ where I remember spending nights over pre-processer errors.

trim tangle
rare scarab
#

Unless iterable is a generator

#

in which case, just yield the object

trim tangle
# 
def unfun[T](*fns: Callable[[], T]) -> Iterator[T]:
    return (fn() for fn in fns)

TPAdapter(unfun(
    lambda: Foo(bar=1),
    lambda: Foo(bar=1, baz=5),
    lambda: SpecialFoo(bar=1, debug=True),
))
hazy adder
#

Just a second, booting up VSCode.

#

Issue being:

if div := response.find(
    "div",
    {"class": "sessions-new system session region"},
):
    tag = div.find("p")
    if isinstance(tag, Tag) and "This work is only available to registered users" in tag.get_text(strip=True):
        raise RuntimeError(
            "The page which you are trying to reach is restricted to authenticated sessions; use `session.authenticate`"
        )
    # I have NO clue what Pyright is complaining about.
    elif div.find("div", {"class": "flash alert"}):  # type: ignore ???
        raise RuntimeError(
            "The authentication process has failed. Are you certain you inputted the correct credentials?"
        )

The elif gives the following type error:

The argument of type "dict[str, str]" cannot be assigned to the parameter "start" of type "SupportsIndex | None" in the function "find"
"dict[str, str]" cannot be assigned to type "SupportsIndex | None"
"dict[str, str]" is incompatible with the protocol "SupportsIndex"
"index" is not present
"dict[str, str]" cannot be assigned to "None"
Pylance(reportArgumentType)
#

type of response is BeautifulSoup, div being Tag

#

At runtime both end up being Tag though.

#

@trim tangle

#

Is there any more information that I could provide you here?

indigo halo
trim tangle
hazy adder
#

According to the documentation, it should by all accounts be fine.

#

¯_(ツ)_/¯

trim tangle
#

(with basedpyright)

hazy adder
#

Oh yeah. I'm using types-beautifulsoup4 to mitigate the worst of it

# 
[dependency-groups]
lint = [
    "pre-commit==3.6.0",
    "pyright==1.1.353",
    "types-beautifulsoup4==4.12.0.20250204",
    "ruff==0.2.2",
]
trim tangle
#

I replaced that line with ```py
assert isinstance(div, Tag)
tag = div.find("p")

hazy adder
#

:thonk: Huh. "beautifulsoup4==4.13.4",

#

Let me try getting rid of it.

trim tangle
#

presumably response.find can return either of PageElement | Tag | NavigableString, but here I'm assuming you're sure that it's a Tag

hazy adder
#

Yeah, I am

#

Both their runtime types end up being Tag

#

Lowkey strange. But oh, well. That fixed it

#

Thank you

#

So whenever I encounter this type of error, just asserting with isistance does the trick. Good to know

trim tangle
#

Sometimes you can eliminate this by refactoring the code, e.g.: ```py

before

class Widget:
def init(self):
self.parser: Parser | None = None # multi-step initialization
def run(self):
assert self.parser #???
self.parser.parse(self.things())
widget = Widget()
widget.parser = MyParser()
widget.run()

after

class Widget:
def init(self, parser: Parser):
self.parser = Parser # single-step initialization
def run(self):
self.parser.parse(self.things())
widget = Widget(MyParser())
widget.run()

#

or alternatively ```py

using the Null Object pattern

class Widget:
def init(self, parser: Parser | None = None):
self.parser = parser or NoOpParser()

def run(self):
    self.parser.parse(self.things())
#

but sometimes you can't avoid it

#

If beautifulsoup wanted, they could have an assertive method like find_tag(...) or find(Tag.DIV, ...) that narrows down the possibilities -- that would be the refactoring route

hazy adder
#

I see. That clarifies a lot, actually.

#

Thank you very much for helping me squash out this error. I appreciate you

indigo halo
rare scarab
#

Pre-commit runs mypy in an isolated environment

#

Your dependencies will not be installed

#

Consider using github actions instead