type-hinting

[oblique urchin]

PEP 646 Unpack isn't supported yet. If you're asking about something else, you should give more detail because I don't understand your question

[cedar sundial]

Ah, 646 is what I want

[granite plover]

if faker https://github.com/joke2k/faker is annotated, why is mypy giving me this error error: Cannot find implementation or library stub for module named "faker"?

[clear lily]

How would one go about defining overloads on a function that gets returned from somewhere?
i.e.

def create_thing() -> ???:
    @overload
    def thing(a: str) -> str:
        ...

    @overload
    def thing(a: int) -> int:
        ...

    def thing(a: Union[str, int]) -> Union[str, int]:
        ...

    return thing

Callable[[str | int], str | int] would obviously work, but that completely voids the benefit of using overloads in the first place

[oblique urchin]

might not have a py.typed file, or it might not be installed in the right environment

[blazing nest]

If you only ned the yield type, just use Iterator!

[oblique urchin]

you can use callable protocols

[blazing nest]

Union together two Callable for the two overloads nope, see below

[oblique urchin]

though first I'd try to redesign the API to not need overloads

[brisk hedge]

A TypeVar("T", str, int, str | int) may also be enough for the specic case

[oblique urchin]

conceptually overload is more like an intersection

[blazing nest]

Hm, true, and I can also think of an issue here where type checkers would probably complain no matter what you pass it in.

[clear lily]

yeah, I know how I'd to it in TS with intersection types, but no such thing in Python unfortunately

[blazing nest]

Since it doesn't understand that it is the same callable, just that it can either be a callable that takes a string or one that takes an integer

Passing Union[str, int] would fail because in the type checkers world there might be a mismatch.

[granite plover]

yea wasnt using my poetry venv, oof, cheers

[clear lily]

oh, a callable protocol isn't something I thought of yet, completely forgot about those

The actual snippet that I'm looking at is a bit more complex with several typevars and a mess of callables (pending redesign ), but that should be a good start, ty!

[granite plover]

if I have

T = TypeVar("T")
Shape = Dict[str, T]
shape: Shape = {"name": str, "age": int}

shouldnt this error?

[hearty shell]

No, the way this is handled depends on the type checker, but you can assume an empty declaration of Shape to just be Shape[Any]

In pyright it takes the union it seems

[hearty shell]

Oh right, in pyright anything that would normally be inferred as Any becomes Unknown, which will try to match whatever types you throw at it

[granite plover]

Im using mypy
Im not sure i get it, if i say that the value in a dict should be generic, doesnt that mean that the type could be anything but it has to be the same across all instances of value?

[hearty shell]

In Python generics are generally not meant to add constraints like this, Shape here is not a generic dict, dicts are already generic, when you define something like X = Y[..., ..., T] You are just partially filling the generic and making a TypeAlies in which you can complete the type. It is just for convenience

[tranquil turtle]

maybe Callable[[str], str] | Callable[[int], int]

[hearty shell]

as an example

def foo(a: T, b: T) -> T: ...

foo(3, "str")

works just fine, because T will try to match the argumetns

In mypy T becomes object and in pyright it becomes Union[int, str]

[granite plover]

this looks very wrong tho lol
it looks to me that the function wants two of the same type, whatever the type, why accept this and why not request something like

def foo(a: T, b: U) -> V: ...

[hearty shell]

The issue is that the constraint "must be the same type" is a little meaningless in languages with subtyping. Any term is an instance of object, and any two terms are both instances of their union, a boolean is an instance of an int, so if you return a boolean from a function that returns an int you now only know that the return was an int, and if you were to use that result with another int that would be a function that takes two values of the same type even though one of them is a boolean

[hearty shell]

You can add constraints to make so T can only be a subtype of X but you still have free range within all the types that are subtypes of X to be matched

Although on a matter of practicality, there are instances when type checkers will actually complain when they can't resolve T to anything other then a Union or object. I am not sure what the rules are on that

[oblique urchin]

constrained TypeVars must be solved to one of the constraints

[hearty shell]

Nvm, I thought I had seen an example of an unconstrained tyepvar that would still complain even though It could match on object, but the problem was that it was resolving to Any I think

Yeah, it it is just that it cant be inferred as Any, even though that is "lawful" https://github.com/python/mypy/issues/12653

[cunning raven]

from typing import Union

def foo(x: Union[int, float], y: Union[int, float]) -> Union[int, float]:
  ...

In these cases is better to assign a variable this typehint?

[trim tangle]

float is actually the same as Union[int, float]

[cunning raven]

oh

why that though

def foo(x: float, y: float) -> float:
  ...```

[trim tangle]

This is legal: ```py
x: float = 42

<https://peps.python.org/pep-0484/#the-numeric-tower>
> ...when an argument is annotated as having type `float`, an argument of type `int` is acceptable; similar, for an argument annotated as having type `complex`, arguments of type `float` or `int` are acceptable
 I personally think it's a mistake, but well, I didn't write PEP 484 🙂

[soft matrix]

it makes sense to me

x: real = 42

[cunning raven]

same

[cunning raven]

thx so much

[hearty shell]

That would make sense but floats are not real since they cant hold arbitrarily large numbers while ints can, so the space of possible ints is larger then the space of possible floats, which is odd

[trim tangle]

Right, but it's not really a subclass of float. It's surprising IMO, especially when you can only get this information by reading the PEP or other documentation.
And what about this: if x is float | str should if not isinstance(x, float) narrow x to int | str or to str?

def f(x: float | str) -> None:
    if isinstance(x, float):
        print(x + 3.14)
    else:
        print(x + "!!!")

f(42)
``` Both `mypy` and `pyright` are okay with the above snippet

[cunning raven]

do you recommend a site to learn docstrings?

[trim tangle]

?

[cunning raven]

documenting the function

"""
Blah blah
"""```

[trim tangle]

what specifically about docstrings do you want to know? what are the popular accepted formats inside the docstrings?

[cunning raven]

nvm, it's solved. thx for asking back 👍

[trim tangle]

I didn't say your question was wrong, I'm just asking for details 😄

[cunning raven]

i don't know where to ask this, sorry if it's the wrong channel

[cunning raven]

what does it mean by documenting an attribute, why, if its not even visible i guess

[trim tangle]

If it's about our guide, probably #dev-contrib

[cunning raven]

i didn't say you said my question was wrong, i just meant that you replied

some people don't.

[cunning raven]

https://google.github.io/styleguide/pyguide.html#381-docstrings

i do have seen code like ```py

class Foo:
"""The docstring"""

var1: int
"""The var1 description"""

var2: int
"""The var2 description"""

but i don't see the use of it.

I haven't been able to get those

print(Foo.__doc__)``` fetches main one above.

[trim tangle]

I think what it means is: if you have a @property, you should document it like this: py @property def name(self) -> str: """The artist's name, as reported by the Soviet intelligence""" # good return self._intel().fetch_name(self._uid) and not like this: py @property def name(self) -> str: """Fetch the artist's name from the the Soviet intelligence cloud database""" # bad return self._intel().fetch_name(self._uid) ...just like you would describe an attribute: ```py
name: str
"The artist's name, as reported by the Soviet intelligence"

[trim tangle]

PEP257 https://peps.python.org/pep-0257/#what-is-a-docstring:

...two types of extra docstrings may be extracted by software tools:

1. String literals occurring immediately after a simple assignment at the top level of a module, class, or init method are called “attribute docstrings”.
   ...

[cunning raven]

ohhhhh

[trim tangle]

some editors recognize these:

[cunning raven]

oh wait

wow, i haven't seen it

ohhhhh, wait, if i document the arguments

i'd not need to document above

let me explain myself better

[soft matrix]

I think the good ones do 😎

[cunning raven]

import attr

@attr.define
class Example:
    x: int
    """X the first unknown"""

    y: int
    """y the second unknown"""

    z: int
    """z the third unknown"""

Would i need to do like

@attr.define
class Example:
   """
   Something about this class

   Args:
      ...
   """
   ...```

or just leave the attributes' documentation

[rustic gull]

foo = Foo(a=0, b='', c=[])
bar: foo
bar.a  # int
bar.b  # str
bar.c  # list

Is there any way I can make something like this possible?

[oblique urchin]

make Foo a dataclass?

[rustic gull]

I want to be able to pass some **kwargs into a constructor, and then be able to use the returned object in type annotations with kwargs as the object's __dict__

If that makes sense

[rustic gull]

No, foo is not intended for run-time usage.

Type-annotating stuff in Python is kinda weird, I was wondering if there's a simple and intuitive way to get this done

If not, I will just go the more verbose route

[uneven ginkgo]

Ok so if i'm type hinting a generator expression then i should use the Iterator[] type hint instead of Generator[] IF i am not using a send type nor a return type?

[acoustic thicket]

Generator[T, None, None] and Iterator[T] are both ok

[uneven ginkgo]

thx

[indigo locust]

Anything interesting happening in the world of type hinting?

[blazing nest]

An Iterator is easier because you don't need to type the send or return typr

[indigo locust]

Actually — while I'm here

Lemme ask you guys. How exactly does one write a static checker?

I've come so far in my own thoughts as to infer that each type annotation can be thought of as a parsable tree, and hence, it can be parsed and a structure of nodes of different types built

When checking a type, you call the root, it calls its children, and so on. Eventually you reach a terminal node and the type is valid, or you don't and its invalid

Given all the different typehints and ways they're used, I'll be actually writing the messages that report a bad type is a nightmare QQ

[blazing nest]

To actually answer this, yeah somewhat. The biggest thing is TypeVarTuple as well as discussion on alternate syntax for TypeVars... the syntax for callables was denied afterall

[oblique urchin]

It's basically like writing a compiler, but skipping the part where it actually compiles things. Sounds like you're writing about the type compatibility check specifically, which is only one part of what a Python type checker needs to do (checking attributes is another big one), for example.

[indigo locust]

I know its a pretty rarified subject, but do you have any idea where I could see a guide on how its all implemented? That, or manybe some kind of simplified, miniature checker I could check out? (hehe)

[oblique urchin]

my typechecker, https://github.com/quora/pyanalyze, is relatively simple and has some (incomplete) implementation documentation

[indigo locust]

I'm writing a tool for runtime enforcement of typehints. This means, or course, that the tool needs to be able to understand the typehints. You're right though, in this context I don't need to worry about the "big picture". I just need to be able to compare an object against its annotation

[indigo locust]

Boss

https://tenor.com/view/not-today-dont-avoid-boss-bossy-devil-wears-prada-gif-14569465

[oblique urchin]

in pyanalyze the code that ingests runtime type hints lives in annotations.py and most of the type compatibility logic is in value.py

[indigo locust]

If this server had an upvote system, I'd be smashing that like button

Thanks, like, a tonne 😄

[cold ridge]

Is there any typing method that allows us to add requirements onto types? For example, a function accepting an object of type T would mean that that object has a .read() method. Is there a way to encode that requirement into the type?

Or would we just use an abstract class?

[summer berry]

Maybe binding a typevar to a protocol?

Though I don't think you need a typevar

I think I just got confused by you using T

[cold ridge]

Is that not the right type name to use? I understand it's not specific.

Can I know what I should use instead here?

[cold ridge]

I'll look into that, I don't know about protocols

Thanks

Oh yeah, protocol is exactly what I want, thank you

[summer berry]

I just immediately associated T with a generic

[cold ridge]

Yeah I was also wondering if I should use that while writing the question, but no other name came to mind so I just went with T

[wispy spindle]

Does mypy flag use of underscored module variables? As an example, mypy on a file that contains only:

from flask import _app_ctx_stack

gives Module "flask" has no attribute "_app_ctx_stack". Which is kind of crummy since that's used a lot, not sure whether it's a quirk with mypy or if flask should expose a non underscored variable

[oblique urchin]

might be a version mismatch, looks like mypy should pick it up

maybe mypy runs in an environment with a different version of flask installed

it's present on flask master: https://github.com/pallets/flask/blob/main/src/flask/__init__.py#L14

[rough sluice]

src/flask/__init__.py line 14

from .globals import _app_ctx_stack as _app_ctx_stack```

[wispy spindle]

I guess that must be the case, just tested in a container and it passes. Weird because it's been in flask forever

Ok now that's shooting myself in the foot, this years-old project still had types-Flask as a dev dependency, which hasn't been required since flask 2

Thanks for the hint

[devout barn]

A = TypeVar("A", bound=tuple)
K = TypeVar("K", bound=dict)


class Arguments(Generic[A, K]):
    def __init__(self, *args: A, **kwargs: K) -> None:
        self.args: A = args
        self.kwargs: K = kwargs

    def __repr__(self) -> str:
        name = self.__class__.__name__
        args = ", ".join(map(repr, self.args))
        kwargs = ", ".join(f"{k}={v}" for k, v in self.kwargs.items())
        return f"{name}({args}, {kwargs})"


foo = Arguments(1, 2, 3, a=4, b=5) # <- problems occur here
"""
Argument of type "Literal[2]" cannot be assigned to parameter "args" of type "A@Arguments" in function "__init__"
  Type "int" cannot be assigned to type "tuple[Unknown, ...]"
    "int" is incompatible with "tuple[Unknown, ...]

Argument of type "int" cannot be assigned to parameter "a" of type "K@Arguments" in function "__init__"
  Type "int" cannot be assigned to type "dict[Unknown, Unknown]"
    "int" is incompatible with "dict[Unknown, Unknown]"
"""
print(foo)

not bounding to tuple or dict gives errors like object does not have items attribute, so what's wrong here?

[acoustic thicket]

*args: A means args: tuple[A, ...]
**kwargs: K means kwargs: dict[str, K]

[devout barn]

How do i make it so that args is considered as the tuple?

And not the constituent elements

[acoustic thicket]

not sure but i don't think you can

[soft matrix]

You can probably use pep 646

But that won't work for kwargs

[devout barn]

so what do I do about this?

[acoustic thicket]

i'm not sure why you're using a typevar there in the first place
args will always be a tuple, i don't see any way it can be a subclass

[trim tangle]

I assume the class will be used to specify particular positional and keyword arguments for a callable

so it's important to know what the arguments are exactly

Seems like this use case is not supported

[acoustic thicket]

[trim tangle]

Although, maybe you can do this with a ParamSpec?

[devout barn]

I initially thought of using typevar there because I intended to implement new methods which returned the args and kwargs (in my case, I was going to implement __iter__) so that the instances can be unpacked into args, kwargs = arguments

[devout barn]

I wished we could do a return type like -> P.args but unfortunately that's not possible (as per pyright)

that seems like a such a convenient thing, I wonder why it's not implemented

[trim tangle]

Would this work for you?
https://pyright-playground.decorator-factory.su/?gzip=H4sIAPykpGIC_3VSwa7bIBC88xXb9IIt6qrtzVIqVT30Vlmq1UsUIWzjlBcMCHAj_30XnOc4lR4XxO7szuwOo7cT9FZr2UdlTahE14OanPURvgutRaclGRMoLk6Zy2uuXZz8LTyDRngx_XKyZ_BDGulVTwhp4fiKoIf2UJAGAxuSHhoMEdJrEQJ885dA76Wn5lzUBPAMcgTOlVGRc5oj6QSpR7a9BFbWEGen5cl2LzgBg6qqzgw2yPW2ggbVx1OIqHcFnh9dBmsi73FUHv-owAflMa-XGjprNQN4D87LGBdAGhHku1xYwIev8NMaWT9pq3iiw1nT9V8mS8HUKolsUwrn9ELzZDCaetv6qWHQ4iwfM1f7IEI1szeIpeWDk0FZ7nhCkYSjZbIGdTHWS9x3ZsMcH62n42yy48-E9-1kyn08mYTmnFcVdwVaTN0goBSJ_HqrVyvxdb2xt9Z6bP0s3xKnVYjczFMnfaAl1pnknTLoa5AOv0-0vga0MevD-0nOBqlerDJ0Eo5mx9c-RfpxO3fyEvaEBf3E4DODLzuy44FB-qpe_pUCJ0HJNG83hcKs471ZtZr41O6paEUX5B8m4SoAbwMAAA%3D%3D

You can also make a thing like this: py @args_for def number_and_string(number: float, string: str, /): ... and then use it like number_and_string(3.14, "foo")

although it is quite verbose

[devout barn]

Thanks!

[trim tangle]

btw I made a typo, s/self._kwarg =/self._kwargs =.

aka # type: ignore considered harmful 😛

[knotty wren]

is there a type that encompasses both int | str and t.Union[int, str]?

[fluid ice]

I'm finally getting off my butt and writing some type-hinted code. so far: smoother than I was anticipating. I foresee a lot more dataclasses and a lot less big gross dicts in my future

[blazing nest]

That's the same thing, use the latter if you need to support older versions

[knotty wren]

i need the type of both unions

to be one type

[blazing nest]

You want an intersection type? As in, a type which resembles an integer and a string together?

[knotty wren]

no

i want to annotate a param that is a Union object

the problem is that type(int|str) is not type(Union[int, str])

[soft matrix]

Yes but to type checkers they are

Oh wait you want the runtime _UnionGenericAlias?

You probably want duck types if that's the case

[knotty wren]

what are those

[soft matrix]

!d typing.Protocol

[rough sluice]

typing.Protocol

class typing.Protocol(Generic)```
Base class for protocol classes. Protocol classes are defined like this:

```py
class Proto(Protocol):
    def meth(self) -> int:
        ...
```  Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing), for example...

[fervent sierra]

Did you want to check if a particular value was either an int or a str? What was your intention when comparing those types in runtime?

[knotty wren]

no

those are dummy values for the example

what i need is to pass a union to a func

and i need to type hint that func

and i compared them just to show that these 2 things dont give the same type back

TypeA | TypeB retuns something of type types.UnionType

thats fine

but i cant understand the type of typing.Union[TypeA, TypeB]

[fervent sierra]

Do you mean the type of the runtime expression typing.Union[TypeA, TypeB]? Or the static type that will be seen by the the type checker?

[knotty wren]

whats the difference? sorry im not pro at typing

i can show my use case

def do_action_on_union_args(union): ...

some_union = TypeA | TypeB
another_union = typing.Union[TypeC, TypeD]
do_action_on_union_args(some_union)
do_action_on_union_args(another_union)

i need to annotate that param

some_union's type is easy - types.UnionType

but another_union idk

[fervent sierra]

so, you gotta be careful there. Python is super weird because it allows you to make the mistake you're making there
You indicate the type of a variable by adding a colon after it, then the type:

def fo_action_on_union_args(union: Union[int, str]): ...

when you use this "colon" syntax, you're not actually affecting anything in your code. In fact, you can still run your code even if your type hints are completely bogus.

What it does, though, is communicate to the type checker (mypy or pyright or whatever) that you intend to only call that function with either int or str

On the other hand, if you do this:

my_variable = Union[int, str]

that acually affects something in your code; you're assigning a weird, essentially useless thing to my_variable, which is some object that represents a type annotation. There is very very rarely any reason to do this kind of thing unless you're going for something very exotic, like interpreting the type hints yourself

[buoyant swift]

i think he knows that

[soft matrix]

What you're attempting to do isn't possible, typecheckers think that some_union is a union of the two types not types.UnionType

Umm I don't think you can do any better than object and then actually checking if it's a UnionType or _UnionGenericAlias at runtime

[blazing nest]

You actually can't represent this in the current Python typing system. There's been suggestions for a TypeForm type, but it's not gotten a PEP yet.

Personally I just use Any here, I have no other choice.

[knotty wren]

I see, rip

thanks

[trim tangle]

Using dicts, lists etc. for structured data is indeed a bad idea

Where I work, we type-annotated everything, and one of my senior colleagues does this and just uses Mapping[str, Any] as the type

[knotty wren]

how can i type hint the return value? the func tries to convert argument to the first type in union that doesnt fail

def convert(argument: Any, union: UnionType) -> ??? | None: ...

[summer berry]

Wouldn't the return type just be that same union then?

[knotty wren]

yeah...

my brain is dying

i had it like so

T = TypeVar("T", bound=UnionType)
def convert(argument: Any, union: T) -> T | None: ...
``` but thought it was wrong

returning T (UnionType) implies that we will return the same Union object, no?

[blazing nest]

Yeah

It's really tricky and I recommend changing your API to work around this

[knotty wren]

i had it like so

T = TypeVar("T", bound=UnionType)
def convert(..., union: T) -> Union[typing.get_args(T)]: ...

[blazing nest]

That won't work, no, sadly.

[knotty wren]

f

ill just not type hint return value

[summer berry]

Maybe that would be possible with variadic generics?

[knotty wren]

uhhh whats that

[summer berry]

https://peps.python.org/pep-0646/

I have never used them so I am not very familiar with them. They just came to mind.

Doesn't seem possible to use unless you have 3.11

Because of lack of support for unpacking in subscripts

I was thinking something along the lines of```py
T = TypeVarTuple("T")
def convert(argument: Any, union: UnionType[*T]) -> Union[*T] | None: ...

[knotty wren]

yeah that would be useful

[knotty wren]

UnionType isnt subscriptable

[summer berry]

ooh

[knotty wren]

yeah its just an arbitrary Union

[summer berry]

Maybe like this then? ```py
T = TypeVarTuple("T")
def convert(argument: Any, *types: *T) -> Union[*T] | None: ...

Shot in the dark cause I don't have a Python 3.11 environment to test with

[oblique urchin]

Static type checkers won't accept that

I think you don't need TypeVarTuple actually. If you take variadic args this should work def convert(argument: Any, *types: type[T]) -> T | None:

https://mypy-play.net/?mypy=latest&python=3.10&gist=37ef6cf8854080096d6a96459f16b58e

unfortunately mypy likes to infer object

pyright infers a Union for the same code though

[summer berry]

Yeah good thinking. This works https://pyright-playground.decorator-factory.su/?gzip=H4sIAHo_pWIC_zXNQQrCMBAF0H1O8XGVSCy4LSh4AVfBjYjEdloC7aQksaTg4W2rnd0M_79pgu-RpsFxC9cPPiSYaaCbDRoXnoQwOG0XuTM7JWpqUHkeKSRpQ_vuiVO5ZDX2M0SxXDy6m4fC4QyDD66eqRSYpygKIfJMbsJRw3HSeHnfKSFcgwwXwT6tLViu18VFxzFZrkjmf_onBhrJds_lpczqC7ngLNTPAAAA

[oblique urchin]

arguably that's a bug, int | bool shouldn't be accepted for type[]

[summer berry]

Oh oops. I forgot to change the call

[wispy spindle]

Is it possible to bind a "generic-style" typevar to the type of a class (or metaclass) member, without actually using a generic? So that, e.g., BaseClass().fn() can specify its return type to be that of BaseClass.member, and a child class can correctly determine that its (inherited) ChildClass().fn() method will return the type associated with its (overridden) ChildClass.member

I asked on stack overflow too, but have been struggling with the wording a bit https://stackoverflow.com/questions/72581534/type-hinting-use-type-of-a-class-member-as-function-return-type-for-inheritanc

This is also somewhat relevant to things like libraries that allow users to specify information in metaclasses, or with overridden variables, that drive return types of functions that are not usually overridden. As an example, FactoryBoy uses a meta class and defines a create() method that will return an instance of the self.Meta.model type https://github.com/FactoryBoy/factory_boy/issues/468

[steel birch]

How can I make an if statement to check if value is less than 0 in a dictionary

[wispy spindle]

This is all possible with generics, but that adds a bit of redundancy since the type is already defined in assigning to a member

[wispy spindle]

I think you have the wrong channel, but just something like if mydict[somekey] < 0: do_something()

[steel birch]

Awesome thanks @wispy spindle

Where should I go for general questions like that? Sorry!!

[wispy spindle]

I'm no expert on the discord and not really sure, but I think the relevant "Topical chat/help" channels only if you have a more upper-level question related to those topics. Otherwise the occupied help channels for general stuff

[pastel egret]

It's still possible to use variadic generics in earlier versions, by using typing_extensions.Unpack[T] - type checkers treat it as equivalent to *T for this purpose.

What's needed there is "TypeForm" (a hint indicating a type definition itself), but that doesn't yet have a PEP/implementation etc.

[wooden solstice]

from typing_extensions import TypedDict

class Base(TypedDict):
    name: str

class Options(TypedDict, total=False):
    number: int

class Child(Base, Options):
    ...

default_options: Options = {"number": 1}
child: Child = {"name": "foo"}

for k, v in default_options.items():
    child[k] = v
# error: TypedDict key must be a string literal; expected one of ("number", "name")

TypedDict is being a nuisance...
How can I merge an Option into a Child without hard-coding every key?

[trim tangle]

child ={**default_options, "name": "foo"}

Generally though, TypedDicts are for typing existing interfaces that use dicts. If you're building something new I would recommend dataclasses or something similar.

[wooden solstice]

I just tried that and now mypy gives me error: Expected TypedDict key to be string literal instead

or if it was to without the type hint (: Child), it's error: Argument 1 to "update" of "MutableMapping" has incompatible type "Options"; expected "SupportsKeysAndGetItem[str, str]"

[trim tangle]

That's one very clear error message :D

[trim tangle]

Hmm, maybe mypy doesn't support that pattern

Maybe it will work with |?

Or are you not on 3.9 yet?

[wooden solstice]

I have to stick with 3.7, unfortunately

Tried union operator in 3.10.5 and it's error: Incompatible types in assignment (expression has type "Dict[str, object]", variable has type "Child")

[wooden solstice]

Thank you for suggestion. I think, as you said, I should find another way, not TypedDict

[blazing nest]

There's Unpack[...] for this reason

[soft matrix]

class Query:
    _raw: tuple[str] | tuple[Query[Any], Operator, T_co]

    def __new__(
        cls,
        *raw: WhatGoesHere,
    ) -> Query[T_co]:
        self._raw = raw
```is there a way to actually type this (without overloads)? ive tried `Unpack[tuple[Query[Any], Operator, T_co] | tuple[str]]` but that doesnt work

[oblique urchin]

I think you need overloads. You can't Unpack a union

[soft matrix]

dang

also ```py
Q: TypeAlias = "Query[Q]"

class Query(Generic[T_co]):
def foo(self):
cls: type[Q] = self.class

well i guess pyright doesnt think this clears specialisation

but i think it should

[blazing nest]

Where and what

[soft matrix]

cls: type[Q] = self.__class__

[blazing nest]

Does it just say "invalid"?

[soft matrix]

well it says type[Query[T_co]] cannot be assigned to type[Query[Q]]

[blazing nest]

Hm, yeah I see. I think that makes sense why it'd complain. Any reason you're doing it like this?

[soft matrix]

well i want to support subclasses of Query

although thinking about it the returning Q doesnt really work

hm

[blazing nest]

Why have Query generic if you're gonna hardcode what you put into it?

[soft matrix]

i dont https://github.com/Gobot1234/steam.py/blob/main/steam/game_server.py#L272

[rough sluice]

steam/game_server.py line 272

def _process_op(self, other: T_co, op: Operator) -> Q:  # type: ignore```

[blazing nest]

Well you somewhat do because the return type won't support subclasses

Q is is hardcoded

[soft matrix]

yeah i realised

i dont really know what to do about this

[blazing nest]

Can't you just -> Self?

[soft matrix]

i could, but the whole idea is that i can type check the queries before they are ran

and that would erase the other type info i already have

maybe this just needs hkt

[blazing nest]

Yeah I am not sure

[soft matrix]

ill just type ignore for now then i guess

[blazing nest]

The typehints are wrong though because self.__class__ could be a subclass but you force it to always be Query

[tranquil turtle]

>>> class X(dict[K | int, V]): ...
...
>>> X[str,list]
__main__.X[str, list]

why repr(X[str,list]) is not __main__.X[str | int, list] ?

>>> X[str,str,str,str,str,str,str,str,str]
__main__.X[str, str, str, str, str, str, str, str, str]
``` weird

[oblique urchin]

it's a dict[str | int, list], but a X[str, list] because the int is not a type argument to X

[rare scarab]

Does it show up in the mro()?

[robust basin]

I think I have a pretty basic mypy issue I'm running into -- I'm getting stub not found errors importing code from another module in the same package

I tried tossing around py.typed files, and adding config sections in mypy.ini

Any tips?

Well I feel silly -- it was lack of __init__.py

[fervent sierra]

Does anyone know if there is such a thing as a Protocol inheriting form another Protocol ? Whenever I try the obvious class MySpecializedProtocol(MyGeneralProtocol), pyright seems to think that I'm trying to create a normal class that implements that protocol. I would also like to not have to copy all the contents of MyGeneralProtocol into MySpecializedProtocol

[oblique urchin]

multi-inherit from Protocol and your other protocol

[fervent sierra]

But I would also like to make references to methods implemented in the base protocol in my specialized protocol =/

that is, something like

class MySpecializedProtocol(???):
    def foo(self):
      self.some_method_from_base_protocol()  

[fervent sierra]

Ha, nevermind... Not sure why I was getting the pyright errors, but at least mypy addresses it in its documentation: https://mypy.readthedocs.io/en/stable/protocols.html#defining-subprotocols-and-subclassing-protocols

[proud brook]

Do I just not type hint enum.StrEnum enumerations? PyCharm warns when I have

from enum import StrEnum


class Whatever(StrEnum):
    NONE: str = ""


variable: Whatever = Whatever.NONE

but not when I don't write : str for the enumeration

[buoyant swift]

what's the warning

[proud brook]

Expected type 'Whatever', got 'str' instead

[buoyant swift]

probably just pycharm being dumb ¯_(ツ)_/¯

[proud brook]

Found out I can type hint with StrEnum.value

Also StrEnum.name

Those two are properties

[trim tangle]

I'm pretty sure you shouldn't make any annotations in enum members

[proud brook]

Okay

[trim tangle]

because the type of Whatever.NONE is Literal[Whatever.NONE]

[proud brook]

Is this how you recommend using enumerations by the way

With a default enumeration

[trim tangle]

I guess sometimes that's useful

[proud brook]

I could do | None

To specify I have not yet decided on a value from the enumerations

[trim tangle]

I recently had to integrate with a bug tracking system which has a bunch of values for status/resolution/etc.. It's pretty useful to have an "unknown" in that case, because they could add more in the future.

[proud brook]

Do I do that

Okay

Cool

[trim tangle]

Although the right solution would be something like ```rs
enum Status {
New,
Assigned { assignee: UserId },
Fixed,
Wontfix,
AsDesigned,
Unknown { raw: u16 }
}

[proud brook]

Isn't that the same thing

Having an unknown enumeration

[trim tangle]

No, in this case the "unknown" case contains some information about the raw data that couldn't be classified

[acoustic thicket]

i mean, nothing's stopping you from setting it as an attribute :p

[trim tangle]

enum values are singletons though

[acoustic thicket]

hmm true i forgot that

[trim tangle]

so something like... ```py
@dataclass(frozen=True)
class New: pass

@dataclass(frozen=True)
class Assigned:
assignee: UserId

@dataclass(frozen=True)
class Fixed: pass

@dataclass(frozen=True)
class Wontfix: pass

@dataclass(frozen=True)
class AsDesigned: pass

@dataclass(frozen=True)
class Unknown:
raw: int

Status = New | Assigned | Fixed | Wontfix | AsDesigned | Unknown

[proud brook]

Basically an enumeration saying "unspecified" or "unset"

Because, well, its value is an empty string and this is what it is supposed to represent

[trim tangle]

sometimes it makes sense, sometimes it doesn't

e.g. if you make a ColourChannel to represent a colour channel, it would be strange to have Red, Green, Blue and Idk

what enum are you making?

[proud brook]

These are enumerations for a type in a dataclass

So there is a value data in the dataclass too

The type data specifies what type the value data is

[trim tangle]

what does it represent, I mean?

[proud brook]

Which data does what represent

So I'm writing a lexer and I decided to make an object oriented "lexer machine" lol

I'm saving a token type and token value state as instance variables

I could probably save one Token instance variable

Yeah

[trim tangle]

If you either have no token or both value and data, then ```py
@dataclass(frozen=True)
class Token:
kind: TokenKind
value: str

and then have either a `Token` or `None`

[proud brook]

Um what

What does frozen=True do

[trim tangle]

makes it immutable

[proud brook]

Hm

[acoustic thicket]

how could a lexer lex a token of an unknown kind

[proud brook]

It's just a default value for when it starts generating tokens

Could probably just do self.__token: Token | None = None

I do keep line and column numbers as instance variables as well

But I think I won't merge them into the token because I want to keep counting them as I generate tokens

And I will make the token immutable

Like

I will have this

@dataclass(forzen=True)
class Token:
    type: TokenType
    value: str
    line: int
    column: int


class Lexer:
    def __init__(self, source_code: str) -> None:
        self.__iter_code: str = source_code
        self.__token: Token | None = None
        self.__line: int = 1
        self.__column: int = 1

What do you think?

Each token will have its own line and column numbers, but I will keep counting lines and columns for other tokens

I do define line and column numbers variables inside a function apart from the instance variables, because I save where I was before reading a token (to load it back if it's an identifier or a number, because those span several columns)

[runic sleet]

Tbh I don't really agree with | None of you will set None as default value

Most type checkers already infer the value as well

[acoustic thicket]

what does __token represent

[proud brook]

A Token

[acoustic thicket]

well yes

[proud brook]

But in the __init__ I set it to None so I add | None to the type hint

[runic sleet]

You could just do

self.__token: Token = None

[proud brook]

Wait I can?

[runic sleet]

Most type checkers will automatically make that into Token | None

[proud brook]

PyCharm doesn't like this

[runic sleet]

Hm?

I use pycharm also, what does it say?

[acoustic thicket]

i meant what does this specific token represent? the token its currently parsing?

[proud brook]

An instance variable that will hold a Token object that will be added to a list of Token objects

Wait

I do not even need this instance variable

[acoustic thicket]

yea

[proud brook]

And it removes a method I've defined

As well

A bonus.

Cool

I do have this but it's okay

As I said line and col save the current line and column numbers as they change (get incremented) when reading identifiers and numbers, so I know where the token has started

I am left with these static methods

I can remove them

I think I will

Now my class looks a lot cleaner

Thanks!

[runic sleet]

wait what's TokenType?

[proud brook]

Now it's this

class TokenType(StrEnum):
    INT = "integer"
    IDT = "identifier"
    OPR = "operator"
    OPP = "open parenthesis"
    CLP = "close parenthesis"
    OPB = "open bracket"
    CLB = "close bracket"
    COM = "comma"
    END = "end"

[runic sleet]

ah okay I thought it was an actual type

so you don't have a token dataclass anymore?

[trim tangle]

No, that's a bad idea IMO
https://github.com/python/mypy/issues/9091#issuecomment-899981237

[proud brook]

Python does have a standard library for generating AST (ast)

But I'm gonna make my own for this simple programming language

[trim tangle]

and I think type checkers are turning off implicit optional

[runic sleet]

no that's

specifically for int I think?

[trim tangle]

No

[runic sleet]

ah okay I see

well that's good to know at least

[trim tangle]

yep, that's what I meant

[proud brook]

Wait how do you use enum.StrEnum

I get this

    from enum import StrEnum
ImportError: cannot import name 'StrEnum' from 'enum' (C:\Users\Roie\AppData\Local\Programs\Python\Python310\lib\enum.py)

Both in PyCharm and in CMD

[oblique urchin]

https://docs.python.org/3.11/library/enum.html#enum.StrEnum

it's new in 3.11

[proud brook]

Oh come on

What do I do now though

[oblique urchin]

class MyEnum(str, Enum):

[proud brook]

Okay

[void panther]

May need to override the str method but I think that's mostly it.

Or you could port it from 3.11, it's a python module so should be fairly easy

[oblique urchin]

https://github.com/python/cpython/blob/main/Lib/enum.py#L1316

[rough sluice]

Lib/enum.py line 1316

class StrEnum(str, ReprEnum):```

[proud brook]

What is ReprEnum

[oblique urchin]

https://github.com/python/cpython/blob/main/Lib/enum.py#L650

[rough sluice]

Lib/enum.py line 650

if ReprEnum is not None and ReprEnum in bases:```

[oblique urchin]

it affects the __str__ method I guess?

[proud brook]

For some reason I had an exception trying to do this

self.__tokens: list[Token] = []
# somewhere else
self.__tokens += Token(token_type, token_value, line, column)

I have to use .append(Token(token_type, token_value, line, column))

It says Token is not an iterable

I thought list.__iadd__ just appends the other item

[oblique urchin]

list += iterates over the argument

[proud brook]

Weird

Oh because

Concatenating iterables

Right

list.__iadd__ is like list.extend

I guess

[brisk hedge]

because + is list concatenation

[proud brook]

Yes

[proud brook]

What should I type hint a parameter that is used on a string (similar to a key function in map or in filter but for strings specifically)

I narrowed down tokenizing integers and identifiers this way

The parameter is str.decimal and str.isalpha

[brazen jolt]

callable that takes a str?

[brazen jolt]

so, it's a function which takes in a single string and returns a bool?

[proud brook]

Yes

[brazen jolt]

Callable[[str], bool]

[proud brook]

A single character too

Okay

Cool

[brazen jolt]

well, you can't type-hint that

python doesn't have a character type, just str

[proud brook]

Yeah I'm fine with that

I renamed the parameter to check

Idk

[brazen jolt]

what is it supposed to do?

[proud brook]

Uses the argument (a function, one of the is methods from the str class) to use on a string (of length 1)

Or any Callable[[str], bool] for that matter

But str already has functions for these so I will just use them

Clear enough?

[brazen jolt]

sure, I just meant more like why do you need a function like this?

[proud brook]

To apply it in a sort of a take_while for strings

[brazen jolt]

you should probably call the function predicate then

[proud brook]

Yeah

Done

Thanks again

[indigo locust]

When typing a callable that is a method

Do I have to include a self argument in the signature?

Callable[[<Something Here>], int]

class Thing:
  def method(self) -> int:
    return 0

[acoustic thicket]

if you're typing Thing.method, then it needs self
if you're typing Thing().method, it doesn't

[void panther]

depends on how is it passed

you care about the call signature; if it's passed as unbound then the caller needs to know about self, if it's bound then self is not a part of what's needed

[indigo locust]

Basically, I'm writing a custom descriptor which works almost identically to a property. Naturally, its a typehinting nightmare

I had originally wanted something to the effect of

class MyProperty(Generic[Input, Output]):
  ...

But it was just a pipe dream. Between the complexity of eeking out the input and output typevars from the fget and fset methods passed, and the fact the many type checkers can't understand generic descriptors

Not worth the time

As an aside question — is the right way to type attributes on a class returned by a generic descriptor would be...

class Descriptor(Generic[T_Return]):

  def __get__(self, instance, owner) -> Any: # return type doesn't matter
    ...

class Thing():
  
  described : Something

  @Myproperty
  def described(self) -> Something: # return type here no recognized as 
                                    # being tied to the annotation for
                                    # instance attribute 'described'
    ...

[hearty shell]

Were you trying to make a property that would only set with the type Input and get with the type Output?

[indigo locust]

That's the theory, but but your question stems from the deeper problem

Which is that properties are pretty unpredictable

So as I've said, I'm not going to bother with any of this

[soft matrix]

considering property can't be typed anyway atm why bother?

Just do if TYPE_CHECKING Descriptor = property

[void panther]

PyCharm always manages to surprise with typehints; somehow resolved tuple(kwargs.items()) to tuple[str, ...]

[proud brook]

What is the difference between typing.Sequence and typing.Iterable

[void panther]

https://docs.python.org/3/library/collections.abc.html#collections-abstract-base-classes

[proud brook]

Thanks

[trim tangle]

btw, you should be using the imports from collections.abc

because the ones from typing are deprecated

(unless you're targeting 3.8 and lower)

See PEP 585

or (🔌) my TL;DR on this package's readme

!pypi flake8-pep585

[rough sluice]

flake8-pep585 v0.1.5.1

flake8 plugin to enforce new-style type hints (PEP 585)

[proud brook]

What about collections.abc.Callable? PyCharm says it's from typing

[trim tangle]

wdym it's from typing?

[brazen jolt]

aren't most (if not all) of the collections.abc imports that are supposed to replace the typing ones in there even in 3.8?

[trim tangle]

that's... PyCharm being PyCharm, I guess 🙂

[proud brook]

I guess I will remove this line

[trim tangle]

Yes, but they are not generic in 3.8

that's what PEP 585 changed

[brazen jolt]

Ah I see, I generally use __future__.annotations so I've never noticed, good to know

[proud brook]

I'm currently writing the parser btw

In the stage of making sense of tokens

I already planned out my expressions and statements layout

[void panther]

is there no way to not evaluate the type parameters to Generic? It yells at me when I pass strings of the names

[soft matrix]

Wdym if yells at you?

[void panther]

TypeError: Parameters to Generic[...] must all be type variables

[soft matrix]

And you're passing a string?

[void panther]

Yes; for a forwardref to paramspec

[soft matrix]

That should work

What version of typing extensions are you on?

Assuming this is typing extensions

[void panther]

should be latest, but it's not imported at runtime

basically this

from typing import Generic, TYPE_CHECKING
if TYPE_CHECKING:
    from typing_extensions import ParamSpec
    P = ParamSpec("P")
    
class A(Generic["P"]):
    ...

[soft matrix]

Weird what I can suggest for now is just adding else P = TypeVar(...)

[void panther]

Yeah thought of that now too when I wrote out the smaller example

the runtime checks typing tries to do are extremely annoying

[soft matrix]

They've become somewhat more relaxed in newer versions

But yeah I agree it's annoying

I can't run a replit with this for some reason but I can reproduce this on tio.run which runs 3.7

Anyway this seems like a bug I feel like this might be fixed on a newer version

[oblique urchin]

This is hard to address because Generic needs to know what its parameters are so it can do type substitution

[tranquil turtle]

does numba have stubs?

[deft merlin]

dict[str, str, ...] means a dict with key-value pairs being strs and there is more than 1 pair in the dict. Right?

[soft matrix]

No, the ellipse is invalid there

dict[str, str] doesn't encode any kind of length

[deft merlin]

so dict[str, str] is good enough to denote key and value are supposed to be str

[soft matrix]

Yep

[deft merlin]

aight

[pastel egret]

It's only tuples that have a specific length, since they're usually used as fixed length objects where each position is a different type. Every other collection the element types are the same. If you want to specify types for specific keys, there's TypedDict.

[tranquil turtle]

!e

def f() -> None:
    raise UnicodeDecodeError

f()

[rough sluice]

@tranquil turtle :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 4, in <module>
003 |   File "<string>", line 2, in f
004 | TypeError: function takes exactly 5 arguments (0 given)

[acoustic thicket]

the fuck

oh, UnicodeDecodeError takes 5 arguments not f

that makes more sense

[solar sphinx]

hey all, does anyone know how can I get rid of the following mypy error?

import codecs
title = job[title_s + len(title_string) : title_e].replace('"', "")
title = codecs.decode(title, "unicode-escape")  # decode \u002F
``` error is: `No overload variant of "decode" matches argument types "str", "str"mypy(error)`

basically title is a string with the code \u002F in between, and I'm trying to get rid of it, got the codecs.decode idea from stackoverflow, but mypy doesn't like it

[oblique urchin]

hm that might be a typeshed bug, we've struggled with those text-to-text encodings for a while

or maybe I fixed it recently? https://github.com/python/typeshed/commit/9660ee97ee901aa5f32e2fd7d57444d147afe3db

what version are you on?

[solar sphinx]

python 3.10, mypy==0.950

[oblique urchin]

looks like my fix hasn't made it into mypy release yet

so the advise is to # type: ignore and wait for mypy 0.970 🙂

[solar sphinx]

ok. will do that. since my code works fine, that's actually a great solution until 0.970 arrives

thanks @oblique urchin 😄

[tough mulch]

anyone know how to let a class type hint it's self
like

class Node:
  Value: str|int|None
  LeftNode: Node
  RightNode: Node

but if i do this it claims Node is undefined
and i tried self but same issue

[oblique urchin]

put it in quotes ("Node"), or use typing_extensions.Self

[tranquil turtle]

from __future__ import annotations

[rare scarab]

if you use Self, it will update itself if you extend the class.

mostly useful for factory classmethods. ```py
class AbstractThing:
@classmethod
def create(cls: type[Self]) -> Self:
return cls()

class Thing(AbstractThing):
pass

t = Thing.create() # It's a Thing

[oblique urchin]

you don't need to annotate cls with Self

[rare scarab]

I'm aware, just being explicit to show that's what it is implied

[oblique urchin]

(you do with typeshed's _typeshed.Self hack which we're using until mypy supports typing.Self)

[rare scarab]

plus, there's a mode in pyright where self and cls need to be annotated

[oblique urchin]

really? that's very pedantic

[rare scarab]

it's not enabled by default.

Anyway.
mypy: "Let's go slow so we get it right"
pyright: 🤠 "Yyyeeeee hhhhhaaawwwwww"

I also find it weird that pyright is written in javascript

the only python in the repo (51%) is type stubs and test samples

[oblique urchin]

TypeScript to be precise 😄

[summer berry]

I figured it made it was a decision to make it easier to integrate into VSC

[oblique urchin]

also because Eric Traut doesn't like Python

[summer berry]

Ironic

[hearty shell]

He also mentioned speed at one point

[rare scarab]

If speed was a concern, go might have been chosen. It's what esbuild did.

esbuild: compiles typescript into javascript using go, ignoring type checking.

[hallow flint]

eric had mentioned that integration with vsc was a big consideration

[rare scarab]

I guess it also makes es6 bundles, which is slow in webpack.

[rare scarab]

seems like a non-issue now that vscode has the language-server

[sacred crane]

Guys how can i change this long number to this short one

[rare scarab]

not sure what that has to do with type hints

[sacred crane]

I know there was like a percentage way in which u write %j or something

[summer berry]

This is off topic. Please claim a help channel. #❓｜how-to-get-help

[hearty shell]

Yeah this was from one of the slides where he talked about it

[rare scarab]

by "runtime performance" it's because there's overhead in starting a new process that has to parse the (potentially very large) python syntax.

[trim tangle]

2 is basically: "typescript has a better type system than Python"? 😛

tbh I don't really get why people love typing so much...

unless you're a data satanist, there are great statically typed languages out there

[grave fjord]

You mean data scientist?

Does TS have variadic generics and DataFrames?

[soft matrix]

ts has variadics

[trim tangle]

wdym by DataFrames?

You can definitely emulate variadic generics, because it's turing complete and supports many operation on tuple types

[grave fjord]

It's like a regular frame, but you can store data in it

[trim tangle]

like you could have Data<[string, string, number]> and it would be typeable

in a similar way to how asyncio.gather can be well-typed in TypeScript

[grave fjord]

But JavaScript can't implement asyncio.TaskGroup

[trim tangle]

I didn't say JavaScript was a great statically typed language 😛

if you want structured concurrency, there's definitely some stuff in Kotlin

[grave fjord]

!!!

[trim tangle]

not sure about concurrency in Haskell

[grave fjord]

Haskell SC needs -o not sure if it's landed yet

[trim tangle]

hm?

sorry I'm a noob in Haskell

[grave fjord]

It's a lollipop, it's Rust-like ownership

-o is a bit like ->

[trim tangle]

oh, you mean the linear arrow thing

[grave fjord]

But the o stands for ownership

[trim tangle]

I thought you meant a -o CLI flag because it's slow lol

[grave fjord]

This brings me back to my original question

What's a data satanist?

[trim tangle]

They come together in a circle, draw an ER diagram on the floor and travel back in time when there was no colour photography so all bears were black and white, like pandas

[grave fjord]

Eh

[dull lance]

Ok so I have a problem that could be illustrated by this example... how would I go about annotating the relationship between TContainer and TElement for the Unwrapper class?

from __future__ import annotations
from abc import ABC, abstractmethod
from typing import Generic, TypeVar

T = TypeVar('T')
class Container(Generic[T]):
    def __init__(self, obj: T) -> None:
        self._obj = obj

    def get(self) -> T:
        return self._obj

class NegativeIntContainer(Container[int]):
    def get(self) -> int:
        return -self._obj

TInput, TOutput = TypeVar('TInput'), TypeVar('TOutput')
class Function(ABC, Generic[TInput, TOutput]):
    @abstractmethod
    def apply(self, x: TInput) -> TOutput: ...

TContainer, TElement = TypeVar("TContainer", bound=Container), TypeVar('TElement')
class Unwrapper(Function[TContainer, TElement]):
    def __init__(self, container_type: type[TContainer]) -> None:
        pass

    def apply(self, x: TContainer) -> TElement:
        return x.get()

# Annotated as: Unwrapper[NegativeIntContainer, Unknown]; should be: Unwrapper[NegativeIntContainer, int]
unwrapper = Unwrapper(NegativeIntContainer)

# Annotated as: NegativeIntContainer; this is correct
container = NegativeIntContainer(1)

# Annotated as: Unknown; should be: int
unwrapped_value = unwrapper.apply(container)

The issue of missing type information in the variables can be solved by providing TElement through __init__, but it misses the requirement of TElement being the element type of TContainer

[trim tangle]

What you want is called "higher-kinded types" and it's not supported in Python

https://sobolevn.me/2020/10/higher-kinded-types-in-python

why do you need such a complex system though? Why not accept Container[TElement] in Unwrapper?

[dull lance]

I'm trying to write a wrapper around sqlalchemy ORM classes to expose only certain fields for read/write access, and also to control the transactions (such that external users can consider all operations on the wrapper as atomic). Naturally, the wrapper is Generic[TModel] where TModel is the ORM class. This corresponds to Container[TElement] in the above example.

I quickly realize that I am writing a large amount of boilerplate just to forward the attributes. To solve this, I have written some custom descriptors to map the values of the attributes between the ORM and the wrapper. This corresponds to Function in the above example.

The problem occurs for attributes that reference other records in the database - in that case, I need to wrap the referenced record as the input/output value (depending on the direction of the mapping). So, the class would be something like Function[Wrapper[TModel], TModel] which leads to the problem described above. I can't use Container[TElement] since I need to invoke the methods defined in the wrapper on the returned value, and doing so would erase such information from the type annotation for the returned value.

[soft matrix]

can probably return None

just use or to initialise with 1 if its none or something

yeah you could do that aswell

[quiet mulch]

Can use try catch

[trim tangle]

checking for None is more clear

[storm python]

what if you're a data stalinist?

[oblique urchin]

I'm looking at https://github.com/python/cpython/blob/ce4d11f98b30ec62f56a0653a212f0f78ca08b59/Modules/posixmodule.c#L13303 now to figure out when it will fail. Online info says the Linux branch doesn't work if you have non-glibc libc or an old version of glibc

[rough sluice]

Modules/posixmodule.c line 13303

int ncpu = 0;```

[trim tangle]

Then definitely go with TypeScript, you don't want people owning @property

[storm python]

though you may want to group them into a class

[trim tangle]

at least you can delete classes!

just like in Python though.

[storm python]

def

[opaque root]

Hello, I am having trouble with pylance and a default argument:

SceneObjectBound = TypeVar("SceneObjectBound", bound=SceneObject)

class Scene:
    def create_object(
        self, ...
        object_class: t.Type[SceneObjectBound] = PNFSprite, ...
    ) -> SceneObjectBound:
        ...

It interprets the true type of object_class to t.Type[SceneObjectBound] | t.Type[PNFSprite] although PNFSprite is a subclass of WorldObject which is a subclass of SceneObject.
Because of that, the results of calls where i do not explicitly specify object_class are being typed as Any, since I assume it just ignores the SceneObjectBound typevar when the default is used and evaluates the "other path" (t.Type[PNFSprite]).
I guess the question is: Is there a way to make pylance realize the subclass relation and leave the type at t.Type[SceneObjectBound]?

[trim tangle]

Are you sure you need a typevar here?

ah, I see

The problem is, PNFSprite is not assignable to Type[SceneObjectBound], because SceneObjectBound could be resolved to a different type. This is what the function looks like to the type checker:

class Scene:
    def create_object(
        self,
        object_class: t.Type[SceneObjectBound] = ...,
    ) -> SceneObjectBound:
        ...
``` so this is legal: ```py
foo: FooObject = scene.create_object()
``` but at runtime it would return `PNFSprite`, not `FooObject`.

This is what you want maybe? ```py
SceneObjectBound = TypeVar("SceneObjectBound", bound=SceneObject)

class Scene:
@overload
def create_object(self, object_class: type[SceneObjectBound]) -> SceneObjectBound:
...

@overload
def create_object(self) -> PNFSprite:
    ...

def create_object(self, object_class: type[SceneObject] = PNFSprite) -> SceneObject:
    # actual implementation

[soft matrix]

this will be fixed when typevar defaults happen

[trim tangle]

what's that about?

[soft matrix]

https://gist.github.com/Gobot1234/8c9bfe8eb88f5ad42bf69b6f118033a7#function-defaults

[opaque root]

Thank you for the quick answer, although to be honest I don't get how SceneObjectBound could be resolved to a different type incompatible with PNFSprite or why it's being resolved to something at all in that context
i'm probably just too stupid to understand something obvious
I tried getting overload to work, but unfortunately also left most parts out of the argument list to save 4 lines of channel space. In truth, the function looks like this, which is making it a lot more painful.

    def create_object(
        self,
        layer: t.Optional[str] = None,
        cameras: t.Optional[t.Union[str, t.Iterable[str]]] = None,
        object_class: t.Type[SceneObjectBound] = PNFSprite,
        *args,
        **kwargs,
    ) -> SceneObjectBound:

[trim tangle]

I don't get how SceneObjectBound could be resolved to a different type incompatible with PNFSprite or why it's being resolved to something at all in that context
When you call a generic function:

def f(xs: list[T]) -> tuple[T, T]:
    ...

x = f([1, 2, 3])
y = f(["a", "b"])
``` the type checker "solves" the type variable, i.e. assigns some value to it. Here in the first call `T` is resolved to `int`, and in the second call `T` is resolved to `str`.

To the type checker, your function looks like this: ```py
def create_object(self, object_class: type[SceneObjectBound] = ...) -> SceneObjectBound: ...
``` i.e. the default value doesn't change the type of the function. So when you call it with no arguments, it can resolve `SceneObjectBound` to be anything at all. ```py
x: FooObject = create_object()
x: BarObject = create_object()
``` The type checker doesn't understand that you want `SceneObjectBound` to be resolved to something specific, it doesn't look at the default value.

[opaque root]

Ah, that makes sense, thank you!

[trim tangle]

I guess you could make two functions, where one calls the other

[blazing nest]

Use overloads. One where object_class is passed and it returns the same type, and another one where object_class is missing and it returns the default

[opaque root]

I managed to write these three overloads that seem to work for pretty much all the invocations of create_object: https://paste.pythondiscord.com/ojubowuzif
When removing the first one, this breaks:

self.create_object(
    "bg",
    object_class = PNFText,
    x = CNST.GAME_WIDTH * 0.7,
    y = 10, # ...
)

And when removing the 2nd one, this breaks:

self.create_object("stage", "main", Boyfriend, scene=self, x=770, y=450)

So I guess that is about as compact as it gets, but at last it actually works

[trim tangle]

t.Optional[t.Union[str, t.Iterable[str]]] is actually the same as t.Optional[t.Iterable[str]]

also, what Python versions are you targeting?

[opaque root]

Still kinda hanging behind on 3.8, jumped ship from win7 to linux and haven't gotten around to "up"grading the abandoned partition to win10. I know typing looks a lot cleaner in 3.9+ since types are subscriptable there and you can drop Optional in favor of | None.
With the string iterable one you got me though, that one is definitely ugly.
Ultimately, that argument ends up here: https://github.com/Square789/PydayNightFunkin/blob/main/pyday_night_funkin/core/scene.py#L282, i don't think you can negate types though, so something like str | t.Not[t.Iterable[str], str] is not possible, is it?

[trim tangle]

yep, that's not possible

I would use tuple[str, ...] maybe?

also, why is "cameras" an iterable of strings? 🤔

[opaque root]

I guess tuple is realistically fine, but it would violate this "be as specific as possible with return types and as open as possible with parameter types" motto i picked up somewhere
And i guess about the cameras being this way it's just how this ended up being. The point of this method is to face the game code directly, and i found it more convenient to do create_object("foreground", "main") instead of create_object(self.foreground_layer, self.main_camera)

just the scene managing everything in a string-to-camera dict, but now i am seriously getting off-topic

Thank you again for mentioning overload, so glad it's finally getting the types right 😄

[gusty hornet]

does type hinting has any influence on the performance?

[trim tangle]

No, they exist purely for documentation, editors and for type checkers like mypy/pyright to find errors

[soft matrix]

It might make your program marginally slower

But it's unlikely to be noticeable

[gusty hornet]

ok ty 🙂

[trim tangle]

Cython or numba or other tools can inspect them to do their thing

[oblique urchin]

though mostly during importing

[gusty hornet]

ah so I can use them "for real" with those imports?

[oblique urchin]

runtime performance isn't affected except in the most indirect way

like maybe slightly worse cache locality

[tranquil turtle]

https://github.com/python/typeshed/blob/master/stdlib/builtins.pyi#L805

[rough sluice]

stdlib/builtins.pyi line 805

def __alloc__(self) -> int: ...```

[tranquil turtle]

what is this?

[brisk hedge]

"B.alloc() -> int\n
\n
Return the number of bytes actually allocated.")

[wispy spindle]

Anyone know if the technique used to give dataclasses dynamic __init__ signatures (for ide hints and mypy checking is usable) outside of data classes themselves? I am able to construct a function via string, exec() it and set it to cls.__init__ within __init_subclass__, but my IDE isn’t picking it up (haven’t tested with mypy yet)

Not sure if I’m missing some tricky work setting the signature (just can’t figure out where it is in the data class source code) or if this functionality just isn’t available outside of dataclasses

Oh WOAH I just stumbled across dataclass_transforms and that might have solved my problem

Well I guess not until 3.11… what’s the correct workaround until then?

[rare scarab]

Maybe there's a backport?

[hallow flint]

yeah, you can from typing_extensions import dataclass_transform . not all ides and type checkers support dataclass_transform yet though

[rare scarab]

Pyright probably does

[wispy spindle]

Awesome, I will try that out. Thanks

[trim tangle]

yep, pyright's maintainer was the one who originally proposed it

pyright had its own __dataclass_transform__ for months I think?

[oblique urchin]

correct

[trim tangle]

I've actually used it in... "Personal stubs"? For some internal libraries

Very useful

I sometimes make my own local stubs for some libraries with crap types

They let me live in my own fantasy typing world

While my coworkers who use the libraries suffer from severe Anytosis

[rare scarab]

Did some messing around to test out the issubclass(x, NoneType) issue.

import types
import typing

NoneType: type[None] = type(None)


def foo(x: int | str | None):
    pass


x_type = typing.get_type_hints(foo)["x"]
x_types = typing.get_args(x_type)

for typ in x_types:
    issubclass(typ, types.NoneType)  # works, but only in python 3.10+
    issubclass(typ, NoneType)  # pyright type error here
    issubclass(typ, type(None))  # pyright type error here

mypy actually fails if NoneType isn't annotated here.

[hallow flint]

if your class is similar enough to dataclass, this hack should work on all existing IDEs and type checkers:

if typing.TYPE_CHECKING:
    from dataclasses import dataclass as my_dataclass_transform
else:
    def my_dataclass_transform(cls): ...

[wispy spindle]

Interesting idea, that’s a solid workaround

Could probably come up with something pretty well covered by wrapping it into decorators

[brisk heart]

any neat way to simplify this? I'm trying to maintain some minimal level of compatibility.

if sys.version_info >= (3, 10):
    Self = typing.Self
elif typing.TYPE_CHECKING:
    from typing_extensions import Self  # noqa
else:
    try:
        from typing_extensions import Self
    except ImportError:
        Self = typing.Any

[blazing nest]

Do you not install typing_extensions?

Do you do it using a Python version check?

[brisk heart]

it's only an optional dependency

might make it required but I hoped to at least make my lib not require any dependencies at all

ig barely anyone uses 3.9 anyways so might as well

[soft matrix]

It's not in 3.9

You can also only use it from typing extensions at type checking time to avoid the dependency

[rustic gull]

I'm pretty sure you can use typing_extensions in versions before 3.9

[brisk heart]

The issue is minimizing dependencies

While also making the code look normal

typing compatibility is a pain tbh

[brisk hedge]

typing_extensions doesn't have to be imported at runtime, and can exist as a dev dependency

so that's at least a little better

if only python had sane package management

[rare scarab]

from __future__ import annotations

import typing as t

if t.TYPE_CHECKING:
  import typing_extensions as te

SomeAlias: te.TypeAlias = "str | int"


def function(lst: list[SomeAlias]) -> str:
  return " ".join(str(item) for item in lst)

You'll only have issues if you need to use typing.get_type_hints()

and with the future import and string type alias, this code is compatible with python 3.7

[brisk heart]

I actually do funnily enough. It's a library that heavily utilizes signature and annotation parsing.

[shut badge]

hmm interesting

makes sense

[trim tangle]

I think it's okay to have typing-extensions as a dependency. It's not heavy (the wheel is only 24KB), and there's a good chance someone already has it: it's the 7th most popular package on PyPI

especially if your users are already "hooked" on typing

[brisk heart]

noooo but my 0 dependencies

[trim tangle]

I don't think it's that important

tbh

What are you making, if that's not confidential?

[brisk heart]

a model lib for fucked rest APIs

kinda like pydantic but if it was more focused on deserialization

I had problems with APIs that have engrish names, terribly formatted values, horrible localizations and stuff like that

[trim tangle]

Well, consider this: your users will install aiohttp which depends on typing-extensions

[brisk heart]

wait it does?

[trim tangle]

it does

[brisk heart]

damn

ok it's fine then

[trim tangle]

https://github.com/aio-libs/aiohttp/blob/master/setup.cfg#L55

[rough sluice]

setup.cfg line 55

typing_extensions >= 3.7.4```

[brisk heart]

I'll just do it without pinning the version and hope it won't mess anything up

[trim tangle]

I think that's fine

although with other libraries, I'd be very cautious with not setting the top version limit

If you do A>=1.0.0 and A's author rewrites A completely in 2.0.0, you're in trouble

[brisk heart]

idk I rarely set it because it's a pain to manage

should get into that

[rare scarab]

Always bundle the latest version of typing extensions so you can use the latest typing

Try to import from typing extensions first (sometimes it will have bug fixes), then typing

Ex: there was a bug in TypedDict in 3.8 that caused it to not retain certain annotations

[void panther]

extend modifies the existing list and returns None

you can use + to merge them, or do it before the run call

you could also unpack in the literal

[hearty shell]

[..., *ETLs], I sometimes also use it when combining two exiting lists

!e

a = [1, 2, 3]
b = [5, 6, 7]

print(["a", "b", *a])
print([*a, *b])

[rough sluice]

@hearty shell :white_check_mark: Your eval job has completed with return code 0.

001 | ['a', 'b', 1, 2, 3]
002 | [1, 2, 3, 5, 6, 7]

[hearty shell]

I think it is a little slower but I think it is negligible. You are already creating a new list when you use the [] anyway

In [12]: a = list(range(1000000))

In [13]: b = list(range(2000000, 3000000))

In [14]: %timeit [*a, *b]
31.1 ms ± 519 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [15]: %timeit a + b
28.9 ms ± 870 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

[hearty shell]

How does one introspect variance?

[trim tangle]

str(T)[0]

Or look at T.__covariant__ and T.__contravariant__

That's some cursed design... why make two linked boolean flags when you could just have a __variance__ property?

[hearty shell]

I meant from the a class, but I was also being dumb xD I get to see what kind of T is it from Class.__parameters__ but of course builtins dont have that

[trim tangle]

ah

[hearty shell]

I wonder what runtime checkers do about that, when a library only has stubs

[trim tangle]

When would a runtime type checker consider variance?

[hearty shell]

It is in preparation for the bivariant property :P

[hearty shell]

Does beartype not consider variance?

[trim tangle]

I don't really know how it works tbh

[soft matrix]

Variance might be getting removed at runtime

[hearty shell]

Oh yeah I heard about that, where instead it would be inferred by typecheckers

I would miss the boilerplate

[brisk heart]

where are my dynamically defined typevar variables at

I don't want to have typevar definitions in all my files anymore 😭

[blazing nest]

I am looking to take advantage of generic typed dicts, any clue when those will release?

ie. a typing_extensions 4.3.0 release 💯

[hearty shell]

what was wrong with 4.2?

.

[grave fjord]

Like in mypy or pyright?

[blazing nest]

Only worked on Python 3.9

[blazing nest]

I am using Pyright - and it works - but it still fails at runtime for me

[blazing nest]

typing_extensions has already fixed it, but it is currently unreleased: https://github.com/python/typing_extensions/commit/1baf0a58b2b4c5327871d06801187cba47aa6975

[brazen jolt]

you could probably get away with this even without runtime support: ```py
from typing import Generic, TYPE_CHECKING

if TYPE_CHECKING:
from typing import TypedDict
else:
TypedDict = dict

class Foo(TypedDict, Generic[...]):
...

[grave fjord]

There's no tags :( https://github.com/python/typing_extensions/tags

[tranquil turtle]

>>> from typing_extensions import IntVar
>>> print(IntVar)
<function IntVar at 0x000002A634076440>

why is it needed?

[hallow flint]

it's a relic from an experimental feature

i think added by the pyre people

they were playing around with ways of doing type level arithmetic (useful for fixed size arrays, or e.g. tensor typing)

[tranquil turtle]

i want to add some overloads to int.__new__ in typeshed:

class int:
    # new
    @overload
    def __new__(cls: type[Self], __x: Literal[False]) -> Literal[0]: ...
    @overload
    def __new__(cls: type[Self], __x: Literal[True]) -> Literal[1]: ...
    @overload
    def __new__(cls: type[Self], __x: bool) -> Literal[0, 1]: ...
    # old:
    @overload
    def __new__(cls: type[Self], __x: str | ReadableBuffer | SupportsInt | SupportsIndex | SupportsTrunc = ...) -> Self: ...
    @overload
    def __new__(cls: type[Self], __x: str | bytes | bytearray, base: SupportsIndex) -> Self: ...

mypy complains about all three overloads:

  196:5   error    mypy  :misc               "__new__" must return a class instance (got "Literal[0]")
  196:5   error    mypy  :misc               Overloaded function signatures 1 and 4 overlap with incompatible return types
  198:5   error    mypy  :misc               "__new__" must return a class instance (got "Literal[1]")
  198:5   error    mypy  :misc               Overloaded function signatures 2 and 4 overlap with incompatible return types
  200:5   error    mypy  :misc               "__new__" must return a class instance (got "Literal[0, 1]")
  200:5   error    mypy  :misc               Overloaded function signatures 3 and 4 overlap with incompatible return types
``` is there a way to return literal from `int.__new__` ?

changing Self to int in cls: type[Self] doesnt help

[soft matrix]

Adding the annotation there shouldn't do anything

[tranquil turtle]

:(

[hearty shell]

Seems weird mypy doesnt accept that, pyright is fine with it

[solemn sapphire]

What exactly is SupportsRead[str] 🤔

I couldn't find anything nice on google

[tranquil turtle]

class SupportsRead(Protocol[_T_co]):
    def read(self, __length: int = ...) -> _T_co: ...

[solemn sapphire]

Honestly that looks cryptic

[tranquil turtle]

https://github.com/python/typeshed/blob/master/stdlib/_typeshed/__init__.pyi#L204-L205

[rough sluice]

stdlib/_typeshed/__init__.pyi lines 204 to 205

class SupportsRead(Protocol[_T_co]):
    def read(self, __length: int = ...) -> _T_co: ...```

[solemn sapphire]

How would you read this?

[acoustic thicket]

if x is an instance of a class which implements SupportsRead[str] it means that it has a read method which accepts an integer (for __length) and returns a str

[solemn sapphire]

I see, thanks 👍

[grave fjord]

The __length thing can go away soon (ish)

[solemn sapphire]

how?

[grave fjord]

You can use def read(self, length, /) -> ...

[solemn sapphire]

The reason this cannot happen now is?

[soft matrix]

Python 3.7 is still supported

[grave fjord]

can you use it freely in pyi's?

[soft matrix]

No, someone tried pr'ing to the decoupled ast module I'm forgetting the name of it but it was too much work to backport the feature

[solemn sapphire]

What feature would this be?

[grave fjord]

https://peps.python.org/pep-0570/

[hearty shell]

from typing import *

class A: ...
class B1(A): ...
class B2(A): ...

b1 = B1()
b2 = B2()

T1 = TypeVar('T1', A, B1)
T2 = TypeVar('T2', A, B1, B2)

def foo1(a: T1, b: T1) -> T1: ...
def foo2(a: T2, b: T2) -> T2: ...

reveal_type(foo1(b1, b2))  # OK
reveal_type(foo2(b1, b2))  # Error

Interesting this only errors on pyright

[grave fjord]

typed-ast?

[soft matrix]

Yeah that

[grave fjord]

https://github.com/python/typed_ast/pull/138#issuecomment-687966628

[trim tangle]

from typing import TypeVar, Generic
from typing_extensions import LiteralString

L = TypeVar("L", bound=LiteralString)


class Foo(Generic[L]):
    def __init__(self, value: L) -> None:
        self.value = value


foo = Foo("hmm")
reveal_type(foo)

Do you think it would be useful if type checkers inferred foo to be Foo[Literal["hmm"]]?

And is there any way to achieve that now?

(right now mypy infers Foo[str] and pyright gives an error)

[soft matrix]

What's the pyright error?

[trim tangle]

Actually it's from pylance, haven't tried pyright

I get:

Argument of type "Literal['hmm']" cannot be assigned to parameter "value" of type "L@Foo" in function "__init__"
  Type "str" cannot be assigned to type "LiteralString"
    "str" is incompatible with "LiteralString"
``` on the `foo = Foo("hmm")` line

[soft matrix]

Seems like a bug

But I'm also sort of interested as to where you see this being useful

[trim tangle]

const("optional") -> Parser[Literal["optional"]]
const("optional") | const("required") -> Parser[Literal["optional", "required"]]

[trim tangle]

yep, seems so https://github.com/microsoft/pyright/discussions/3589#discussioncomment-2980609

[grave fjord]

is this supposed to work in mypy?

from typing import *


if TYPE_CHECKING:
    from typing_extensions import ParamSpec

    P = ParamSpec("P")
    R = TypeVar("R")
    T = TypeVar("T")
    Coro = Coroutine[Any, Any, T]
    AsyncFn: TypeAlias = Callable[P, Coro[T]]
    
    
def start_soon(async_fn: AsyncFn[P, None], *args: P.args, **kwargs: P.kwargs) -> None:
    return None

eg a TypeAlias of a Param-Specced Callable?

obviously it works on pyright

yes it's supposed to and no it doesn't: https://github.com/python/mypy/issues/11855

[indigo locust]

How do I properly typehint a decorator class such that the static checker assumes its signature is the same as the signature of the callable passed in during construction?

from typing import Generic
from typing import ParamSpec
from typing import TypeVar

P = ParamSpec('P')
R = TypeVar('R')

class FunctionWrapper(Generic[P, R]):

  def __init__(self, wrapped: Callable[P, R]) -> None:
    ...

  def __call__(self, *args: P.args, **kwargs: P.kwargs) -> R:
    ...

Its something like this, I think...

Am I forgetting anything?

[hearty shell]

You cant inherit from Callable, just replace that with Generic

[indigo locust]

Right

At it fulfills the protocol Callable by virtue of having a __call__ method

[hearty shell]

Yup

Although I dont know if you call Callable a protocol

[indigo locust]

Potato tomato 😛

Another one

A little more generic, but I think this is the place to ask

isinstance(function, types.FunctionType) or inspect.isfunction(function)?

[void panther]

They should be equivalent, the inspect variant is probably clearer

mind that it may not be what you want unless you're accessing some attributes that only FunctionType has, as it'll exclude methods, builtins etc.

[indigo locust]

Well, I'm writing a custom property class, one which leans heavily on type annotations and signatures with specific shapes

And there's a wide gulf between a function and a generic callable in terms of how everything is laid out

So for the time being I'm limiting acceptable getters and setters and such to function objects only.

[indigo locust]

What's the technical term for a callable object?

A functor? Simply "a callable"?

[soft matrix]

a callable

[rare scarab]

!d callable

[rough sluice]

callable

callable(object)```
Return [`True`](https://docs.python.org/3/library/constants.html#True "True") if the *object* argument appears callable, [`False`](https://docs.python.org/3/library/constants.html#False "False") if not. If this returns `True`, it is still possible that a call fails, but if it is `False`, calling *object* will never succeed. Note that classes are callable (calling a class returns a new instance); instances are callable if their class has a `__call__()` method.

New in version 3.2: This function was first removed in Python 3.0 and then brought back in Python 3.2.

Similar names: scikit_learn.callable, aiohttp.callable

[trim tangle]

YO it's finally here!!
https://pyright-playground.decorator-factory.su/?gzip=H4sIAOfJsGIC_1WPwWrDMBBE7_qKwb1Y4OQDAukxvYheGnIpRajOqhFIWrOSTfP3ldOGkrksyzxmGC-cUK9TyF8IaWKpOF4nOjkZ8EKZJIzK_zOWvivlEjiXO25CJXHxrUrzlTLY3xP6znQDPnnO5_0DpZVSY3Sl4MDc_9W8mw-9U2g6k4e1IYdqbV8o-gGLizPtYDQ2z3jlTL_kqhXY3vzWfLst3TO3b03vLil1Wgkt5KJtK6hvpgaekHghjLMUFlxISP0AtecW4w0BAAA%3D

[acoustic thicket]

great success

[indigo locust]

What exactly are we celebrating?

[trim tangle]

in foo = Foo("hmm"), foo is inferred as Foo[Literal["hmm"]] instead of Foo[str], because of LiteralString

[indigo locust]

Yay...?

[trim tangle]

exactly

[sterile python]

Interesting feature, as I understand, the code

def execute(query: LiteralString): ...

execute(f"{ 1 + 1 }")

Would be incorrect for type checker?

[trim tangle]

PEP 675 says this about f-strings

...we also infer LiteralString in the following cases:

• String formatting: An f-string has type LiteralString if and only if its constituent expressions are literal strings. s.format(...) has type LiteralString if and only if s and the arguments have types compatible with LiteralString.
  so yes

[magic oak]

What type hint do you guys use for generator objects?

class my_generator() -> "generator?":
    ...

[trim tangle]

you can't typehint a class like that

Can you provide more details?

[magic oak]

I was trying with the typing.Generator one, just not sure if that's the standard practice

Sure, I can share the code.

from typing import Generator, Tuple

def get_locations() -> Generator[int, str, Tuple[float, float, float]]:
    cur.execute("SELECT id, name, x, y, z FROM minecraft.locations;")
    return (dict(
            ID=ID,
            name=name,
            coordinates=(x,y,z)
        ) for ID, name, x, y, z in cur.fetchall())

[trim tangle]

if you want to type-hint a generator function:

from collections.abc import Generator

def numbers() -> Generator[int, None, None]:
    yield 1
    yield 2

def things() -> Generator[int, str, list[str]]:
    foo = yield 1
    bar = yield 2
    return [foo, bar]
``` or you can use `Iterator` if your generator doesn't do any sending or returning ```py
from collections.abc import Iterator

def numbers() -> Iterator[int]:
    yield 1
    yield 2

[trim tangle]

Generator[dict[str, object], None, None] or Iterator[dict[str, object]]

if you want to have a more precise type, you will need to use a dataclass or a NamedTuple instead of a dict like that

[magic oak]

Oh yeah I messed up the type even. What's the difference on collections.abc vs typing?

[magic oak]

Dataclass is a good idea actually

[trim tangle]

In PEP 585, standard library types like tuple, list, dict, set, type and stuff from collections.abc are now generic, and their counterparts from typing are deprecated

!pep 585

[rough sluice]

**PEP 585 - Type Hinting Generics In Standard Collections**

Link

Status

Accepted

Python-Version

3.9

Created

03-Mar-2019

Type

Standards Track

[trim tangle]

Unrelated to typing, but it's probably better to just return a list. cur.fetchall() fetches everything into a list anyway.

[magic oak]

Yeah probably x) I'm a bit rusty on sql atm

Thanks for the help!

[sullen yacht]

hey guys,
I am using pydantic in my project.
I have the following class:

class Attribute(BaseModel):
    idx: str
    name: str
    value: str

how can I make all the class properties private and create getters and setters for each one ?

[unique mirage]

You might find something useful here: https://github.com/samuelcolvin/pydantic/issues/655

[trim tangle]

but why?

maybe you can provide some more details?

[rare scarab]

By private, do you mean non-serialized (transient) fields

[brisk hedge]

!pep 675

[rough sluice]

**PEP 675 - Arbitrary Literal String Type**

Link

Status

Accepted

Python-Version

3.11

Created

30-Nov-2021

Type

Standards Track

[brisk hedge]

Oh how neat

def string_launder(s: str) -> LiteralString:
    data = "\0\1...ABCD...\xfe\xff\u0100\u0101...\ufffe\uffff\U00010000\U00010001...\U001ffffe\U001fffff".split("") # split required
    out = ""
    for c in s:
        out += data[ord(c)]
    return out # look ma no type ignore

oh how neat

[oblique urchin]

we could probably make the .split unnecessary by adding an overload for str.__getitem__ where Self is a LiteralString: https://github.com/python/typeshed/blob/67f377898b870b6bcf80fc87667264a10beb0aba/stdlib/builtins.pyi#L552

[rough sluice]

stdlib/builtins.pyi line 552

def __getitem__(self, __i: SupportsIndex | slice) -> str: ...```

[brisk hedge]

very true

also I just realized this needs another 100kb in the string literal since split("") isn't valid so it'd need a separator

[uneven ginkgo]

how do i typehint a function return
where it can either return a string representation of a datetime
and in other cases it can return a datetime.datetime object

currently it is type hinted like this:
Union[str, datetime.datetime]
but my linter is complaining

[wooden solstice]

If return type can be determined by input type, I would use overloads

[uneven ginkgo]

i have a parameter that is "return_datetime" and its a boolean

[wooden solstice]

@overload
def func(*arg, return_datetime: Literal[False]) -> str: ...
@overload
def func(*arg, return_datetime: Literal[True]) -> datetime: ...
def func(*arg, return_datetime):
    ...

Never tried but I think it'll work like this?

[uneven ginkgo]

never used overload decorator and i don't know if it is neccesary

is there really not a way to type hint a return value that can either be a string or a datetime based on what happens in the function?

[soft matrix]

Overload here is the correct approach

[uneven ginkgo]

ah okey i will try to make it work. thanks!

[trim tangle]

The correct approach is to not have such behaviour IMO

Just return a datetime

[uneven ginkgo]

that might be a better way to fix the problem, yes

[signal sentinel]

What does the overload decorator do?

[pastel egret]

You use it to define multiple alternative type definitions for a function, like how it's done here. The final undecorated one is the actual implementation which is used at runtime. When you call the function, the type checker tries each definition in order, and uses the first that is valid.

[signal sentinel]

Would you know a good article on it?

[pastel egret]

https://mypy.readthedocs.io/en/stable/more_types.html#function-overloading

[grave fjord]

someone just pointed out to me this issue: https://github.com/python/mypy/issues/1362

but it seems that mypy is claiming that it doesn't support properties on decorated functions, but it does:
https://mypy-play.net/?mypy=latest&python=3.10&gist=2a3a774b43c011934a7b158f814b8849&flags=strict

[blazing nest]

How are you even expecting this to work at runtime? The order of the functions should be changed right?

[grave fjord]

ah it's no good someone else already had a pop at this https://github.com/python/mypy/pull/11719

[dull lance]

How would I go about writing a protocol for a class with read-only properties? For example:

from typing import Protocol

class MyClass:
    def __init__(self, x: int) -> None:
        self._x = x

    @property
    def x(self) -> int:
        return self._x

class MyClassProtocol(Protocol):
    def __init__(self, x: int) -> None: ...

    # How to type hint x?

    # What I tried:
    x: int    # property is incompatible with int

def make_instance(x: int) -> MyClassProtocol:
    obj = MyClass(x)
    return obj

[acoustic thicket]

class MyClassProtocol(Protocol):
    def __init__(self, x: int) -> None: ...

    @property
    def x(self) -> int: ...

[dull lance]

oh... how did I not think of that?

[rustic gull]

class BaseType:  
    pass

class ClassOne(BaseType):
    pass

class ClassTwo(BaseType):
    pass


my_type: BaseType = ClassOne()

Is this valid?

-- Does the ABC type hint apply to subclasses as well

[dull lance]

thanks

[rustic gull]

Or would I have to make something such as BaseTypes: typing.TypeAlias = BaseType | ClassOne | ClassTwo

[acoustic thicket]

its valid

[trim tangle]

x: int means that it's both readable and writable, I think

[wooden solstice]

from typing_extensions import Self

class A:
    def test(self) -> Self:
        return self
# error: Variable "typing_extensions.Self" is not valid as a type

https://github.com/python/typing_extensions/issues/26
It's fixed but not fixed?
using python 3.7, typing-extensions 4.2.0, mypy 0.961

[dull lance]

yeah, was wondering how to mark it as read only

[soft matrix]

That's not the issue

Mypy doesn't work with Self yet

[acoustic thicket]

classic mypy

[wooden solstice]

Thank you

[soft matrix]

My pr for it mostly works I just hadn't had time to work on it until 2 days ago

https://github.com/python/mypy/pull/11666

If you want to try using it

[trim tangle]

you could say it has

low Self-esteem

[rustic gull]

what’s type hinting?

[trim tangle]

check out the pins 🙂

[blazing nest]

How would I typehint something returning the dict_keys, dict_values iterators?

Is there any point? I already know that my dictionary is Dict[str, Any] and won't be mutated

[trim tangle]

They actually are not iterators, they're more like sets

they are iterable though

think about functions as what their caller would expect them to return. If it's something iterable, return Iterable

Everything else is an implementation detail. You might have to filter some of the keys tomorrow, or something like that.

[unique mirage]

It seems that you can use KeysView and ValuesView for the typing in this case. Take a look at the docs:

https://docs.python.org/3/library/typing.html#typing.KeysView
https://docs.python.org/3/library/typing.html#typing.ValuesView

[blazing nest]

Ah, I knew that it existed somewhere! Thanks

[blazing nest]

Right yeah

[trim tangle]

Why do you want to return the key view specifically?

[blazing nest]

I am wrapping a dictionary (well a MappingProxyType specifically, which I cannot subclass unfortunately as it is marked final)

[trim tangle]

ah

[blazing nest]

I want to forward all util methods it has

[trim tangle]

Then yes, it is probably a good idea

(You can always make your own wrapper that implements KeysView if you want to)

[trim tangle]

What are some use cases where Any is really the best solution?

[trim tangle]

I want to add an article about Any to my typing-tips.
Here's a draft:
https://gist.github.com/decorator-factory/4f1bfe96dfd2d065172c4f16c8187a78
I would be happy to hear what you think 🙂

(some of the links don't work, but pretend that they do)

[blazing nest]

Extremely large unions where you cannot accurately represent relationships or expected return types in Python's typing system. JSON essentially haha

[trim tangle]

hmm

well, you can type JSON in Pyright 🙂

But perhaps a better strategy would be to convert dynamic values to structured, typed values as soon as possible

and validate them while doing so

[blazing nest]

Right but if you have a deserializing method you still need to use Any

[trim tangle]

why?

it can accept object

or do you mean that the stdlib returns Any from json.loads?

[blazing nest]

Yeah

[rose root]

When accepting **kwargs? Probably could be typehinted without Any but it would be a lot simpler, depends on the use case though but I do think in the case of accepting a bunch of kwargs Any to be best case

[trim tangle]

Depends on why you're accepting **kwargs, I suppose

Often Any is the best thing for kwargs because of the limitations of the type system, and it's currently being addressed with ParamSpec (already available) and with some experiments in pyright (https://github.com/microsoft/pyright/issues/3002#issuecomment-1046100462)

[rose root]

Oh yes the unpack looks very nice so far, and completely fills the whole reason why I had to use any previously

Usually I’ll just take a bunch of arbitrary kwargs then later look for specific ones which then I can assign a type too

Although strictly speaking not required, it does lessen line length

The unpack looks like a very nice addition looking forward to using it more with generics too

[trim tangle]

**kwargs are not very fun for the user of a library. I think matplotlib is notorious for that

it can send you on a journey

I often go on this journey with aiohttp when making a request

[rose root]

I find the docs kind of tedious I’ll usually just go to the repository for aiohttp

[trim tangle]

I just press F12 and my editor teleports me to the implementation

the docs are formatted weirdly, yeah

I know it's kinda rude to make fun of nice free software people maintain on their free time that I just get to use, but

p
a
r
a
m
e
t
e
r
s?

[rose root]

Waiting for furo theme to be adopted more

[brisk heart]

in 3.8 issubclass(typing.Sequence, typing._GenericAlias) == True
in 3.9 issubclass(typing.Sequence, typing._GenericAlias) == False

how come?

I'm guessing it's to do with that the origin of typing.Sequence is collections.abc.Sequence but that makes no sense aaa

[soft matrix]

why are you running this?

[brisk heart]

annotation parsing

reflection stuff for a model lib

[soft matrix]

just check origin

dont run this

[oblique urchin]

we like to change the internals of typing with every release to keep you on your toes

[brisk heart]

I want to check any kind of collection

[oblique urchin]

(not really but the effect is similar)

[brisk heart]

yeah I noticed...

gotta keep my backwards compatibility

[rare scarab]

You know what typing needs? A way to infer a type from a value. Typescript has typeof value. Similar to reveal_type()

[soft matrix]

theres been an issue open on python/typing about this for a while

it would be cool

[brisk heart]

ok but how to do that in a backwards-compatible way?

[soft matrix]

would also mean that the idea ambv had for functions as callable annotations would be more generalised

oh wait you want to allow unsubscripted Sequence?

[rare scarab]

If only... ```py
def foo(a: str, b: str) -> str:
return a + b

FooType = reveal_type(foo)

[brisk heart]

that too

I needed to check it because I had this

def lenient_issubclass(obj: object, tp: typing.Type[T]) -> typing_extensions.TypeGuard[typing.Type[T]]:
    """More lenient issubclass."""
    if isinstance(tp, GenericAlias):
        return obj == tp

    return isinstance(obj, type) and issubclass(obj, tp)

an origin is already passed in

[soft matrix]

maybe take a look at typing-inspect

!pypi typing-inspect

[rough sluice]

typing-inspect v0.7.1

Runtime inspection utilities for typing module.

[soft matrix]

cause that'll be more helpful than me

[brisk heart]

all it checks is if the tp is smth like typing.Union[Foo, Bar] because that'd be a one-off special situation that should compare directly

since I have it as some typealias

it's funky but reflection in python is overall funky

[brisk heart]

going for no deps ideally but I've seen the source code

literally how can a typing lib not be typed though 😭

[oblique urchin]

the most recent additions to typing.py are typed 🙂

[brisk heart]

yeah that's nice

I'll just do this (also damnit pyright, how is that a Never)

def lenient_issubclass(obj: object, tp: typing.Type[T]) -> typing_extensions.TypeGuard[typing.Type[T]]:
    """More lenient issubclass."""
    if isinstance(tp, GenericAlias):
        if obj == tp:
            return True

        tp = tp.__origin__  # type: ignore  # apparently Never

    return isinstance(obj, type) and issubclass(obj, tp)

hopefully that fixes it 🤞

[soft matrix]

typing inside of typing.py seems a little cursed

also when i last was trying to do something pyright really didnt like it anyway

[brisk heart]

whenever I wanna see the stuff in typing I gotta use pyright's stubs to make it make sense

especially because abstracts in typing like Sequence don't show the methods it has

I give up (pytest)

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tp = typing_extensions.Annotated[ForwardRef('typing.TypedDict'), typing.Dict[str, int]]

    def normalize_annotation(tp: object) -> object:
        """Normalize an annotation and remove aliases."""
        if tp is type(None):  # noqa: E721
            tp = None

        if isinstance(tp, tutils.AnnotatedAlias):
>           tp = typing.get_args(tp)[1]
E           IndexError: tuple index out of range

tp         = typing_extensions.Annotated[ForwardRef('typing.TypedDict'), typing.Dict[str, int]]

literally how is that possible

3.8 again

why is it not using __metadata__ 😭

if it weren't for the majority of users I'd drop this version asap

[fleet vale]

Sorry if this is a super basic question that I missed the answer to in the docs, but:
Does Python have support in their type hints feature for checking an implemented method instead of the type?

[oblique urchin]

do you mean a type hint that means that a particular method exists?

that can be done with Protocol

[fleet vale]

Yes

Ah, thanks! Protocol is definitely what I needed

[trim tangle]

hi @oblique urchin what do you think of this piece? (approx 5 minutes)

[oblique urchin]

nit: ++mystery_object is needlessly confusing to people who don't know the language well

[trim tangle]

hmm yeah I probably went over the top in that snippet

what about this?

def do_something(mystery_object: Any) -> None
    print(mystery_object)  # ok
    mystery_object.quack()  # ok
    mystery_object += 1  # ok
    carefully_typed_value: int = mystery_object  # ok
    mystery_object(mystery_object)  # ok!

[oblique urchin]

yes that's better

[trim tangle]

(I also assume people who'll note launchMissiles already hate Any)

[oblique urchin]

actually print(mystery_object) also isn't a great example because that works with object too

[trim tangle]

that's a good point

[oblique urchin]

on "when to use Any", I'd generalize it into "when the type system isn't powerful enough to express the type you need". What you say is kind of a special case of that

also there's a typo reqeuests

[trim tangle]

Well, some things are expressible in the type system. But you might need to refactor quite a bit of code to make it work.

[oblique urchin]

that's true

[trim tangle]

Or maybe the annotation is just too complicated.

Consider asyncio.gather: it uses an overload ladder and lies in its return type to achieve better typing in 99.9% of the cases