type-hinting

[solid mesa]

what was the point of that

[hearty shell]

do you have annotations __future__ on?

[solid mesa]

nope

[hearty shell]

Then if you try running your module that will raise an exception

It is just not a supported thing for ints

[solid mesa]

wow you're right

appreciate it

tried in eval

guessing i should have tried that first

[hearty shell]

In general it is not possible to give constraints on types in this manner

You might be with runtime checkers but that doesnt affect your linter

[solid mesa]

i wanted to use mypy but it gives me sql orm errors

and i researched and flask_sqlalchemy hasn't found ways to support static stubs

[hearty shell]

Does anyone know what is wrong with these?

P = ParamSpec('P')
T = TypeVar('T')
T_co = TypeVar('T_co', covariant=True)

class Initializable(Protocol[P]):
    def __init__(self, *args: P.args, **kwargs: P.kwargs) -> None: ...

def init_args_foo(
    method: Callable[[Initializable[P], Any], T]
) -> Callable[Concatenate[Any, P], T]:
    ...
    
class MethodToPaste(Protocol[P, T_co]):
    def __call__(real_self, self: Initializable[P], *args: Any, **kwargs: Any) -> T_co: ...

def init_args_bar(
    method: MethodToPaste[P, T]
) -> Callable[Concatenate[Any, P], T]:
    ...

class User:
    def __init__(self, a: int, b: str = "boo") -> None: ...

    @init_args_foo  # Error
    def method_foo(self, *args, **kwargs) -> Self: ...

    @init_args_bar  # Error
    def method_bar(self, *args, **kwargs) -> Self: ...

Argument of type "(self: Self@User, *args: Unknown, **kwargs: Unknown) -> Self@User" cannot be assigned to parameter "method" of type "(Initializable[P@init_args_foo], Any) -> T@init_args_foo" in function "init_args_foo"
  Type "(self: Self@User, *args: Unknown, **kwargs: Unknown) -> Self@User" cannot be assigned to type "(Initializable[P@init_args_foo], Any) -> T@init_args_foo"
    Parameter 1: type "Initializable[P@init_args_foo]" cannot be assigned to type "Self@User"
      "Initializable[P@init_args_foo]" is incompatible with "User"

Argument of type "(self: Self@User, *args: Unknown, **kwargs: Unknown) -> Self@User" cannot be assigned to parameter "method" of type "MethodToPaste[P@init_args_bar, T@init_args_bar]" in function "init_args_bar"
  Type "(self: Self@User, *args: Unknown, **kwargs: Unknown) -> Self@User" cannot be assigned to type "(self: Initializable[P@init_args_bar], *args: Any, **kwargs: Any) -> T@init_args_bar"
    Parameter 1: type "Initializable[P@init_args_bar]" cannot be assigned to type "Self@User"
      "Initializable[P@init_args_bar]" is incompatible with "User"

[trim tangle]

I assume configparser.ConfigParser, yes

if you're using Pylance, you need to go to settings, find "Type Checking" and turn it from "off" to "basic" if you haven't done that already

You can also hover over config in your first screenshot and you'll see what type Pylance inferred for it

[hearty shell]

Ah, it was contravariance I think, trying to match Callable[[ProtoFoo], None] with Callable[[Foo], None]

F

Another instance where covariance on the first argument of Methods would be useful

[solar sphinx]

hey all, I'm following fastapi's tutorial and I'm getting a mypy error I'm unable to debug, can anyone spot why? @app.put("/items/{item_id}") async def create_item(item_id: int, item: Item, q: str | None = None): result = {"item_id": item_id, **item.dict()} if q: result.update({"q": q}) return result The error is highlighted on the result.update({"q": q}) -> Dict entry 0 has incompatible type "str": "str"; expected "str": "int"mypy(error)

[hearty shell]

result = {"item_id": item_id, **item.dict()} mypy is inferring result: dict[str, int] here

Just do result: dict[str, int | str] = {"item_id": item_id, **item.dict()}

[solar sphinx]

amazing, that fixed it 😄

now I need some time to digest it haha

ty

[trim tangle]

@solar sphinx I would just do result: dict[str, object]

since it really just has dynamic data

or you could return a pydantic model to add validation and stuff

[solar sphinx]

ok thanks. I'll play a bit with that idea

[thin terrace]

hey, how can I reference a class which is defined in lines later than when I need it?

[acoustic thicket]

put it in double quotes, typecheckers can recognize that

[thin terrace]

basically, I have two classes and they are reliant on each other, so I get errors when i try to typehint my function parameters.

[acoustic thicket]

or, put from __future__ import annotations at the top of your code

[spice locust]

when writing a package, is it good practice to add typing.Optional to all optional parameters or is leaving that out considered acceptable? in my case specifically there are quite a lot of these optionals and it makes the code harder to read

[oblique urchin]

Optional doesn't mean the parameter is optional, it means that it's allowed to be None. You should use it if you want to provide types for your users and if the parameter accepts None as a valid value.

[hearty shell]

Optional strikes again?

[brazen jolt]

if this package is an end product, not something like a library to be used, you can simply configure pyright to automatically handle this for you, however if it's going to be imported by others, and you want other people to be able to correctly see your types, you should mark them as Optional, as that is the preferred/standard way of doing it

(maybe there's also a way to configure mypy like this, but I'm not sure about that, I don't use mypy that much)

you could also keep not using optional and instead auto-generate stub files, which will then contain the Optional and that's what will be read by the type-checkers instead, there are some tools to help you do this

but generally, just use Optional, it's not that bad and you'll soon get used to it

[spice locust]

in my code the default None means a few different things. in some places it means "if you dont pass this parameter, it will use a default value defined somewhere else", and in other places its used when there are 2 mutually exclusive parameters, so the user must pass one but not both

[brazen jolt]

for cases like "must pass one or other, not both" use overloads

[spice locust]

ive heard of those before... can you give an example?

[trim tangle]

https://decorator-factory.github.io/typing-tips/pitfalls/optional-is-not-optional/

wait, maybe I misread

or did I?

[brazen jolt]

from typing import overload

@overload
def foo(*, x: int) -> None:
    ...

@overload
def foo(*, y: int) -> None:
    ...

def foo(*, x: Optional[int] = None, y: Optional[int] = None) -> None:
    # Implementation

[spice locust]

im aware that Optional[<type>] is the same as Union[<type>, None], so does that mean Optional indicates the user may pass a literal None as the argument value?

[brazen jolt]

so does that mean Optional indicates the user may pass a literal None as the argument value?
exactly, yes

def foo(x: Optional[int]):
    ...

foo(None)  # Valid

you can use optional even without default value

[trim tangle]

...but foo() is invalid

[spice locust]

in which case the behavior does not change at all, since the None will be replaced by something else down the road or ignored. however it might give the user the wrong idea

[brazen jolt]

for this code, user can do: ```py
foo(x=5)
foo(y=5)

But not

foo(x=5, y=3)

[brisk hedge]

What's best practice for marking potentially missing arguments in cases where None could be a meaningful value?

class _missing:...
missing = _missing()

def foo(x: int | _missing = missing) -> int: ...

??

[brazen jolt]

yeah, I think there was even a PEP about introducing sentinels directly

but for now, just doing object() is pretty common

[trim tangle]

well, type checkers don't understand the is missing check

[brisk hedge]

T | object is object, right?

[brazen jolt]

some people use custom class for type-hints, some just go with typing.NewType, etc.

[trim tangle]

how would you use NewType?

[hearty shell]

They do if you use an Enum right?

[oblique urchin]

yes, an Enum with a single member is a common workaround

[oblique urchin]

!pep 660

[rough sluice]

**PEP 660 - Editable installs for pyproject.toml based builds (wheel based)**

Link

Status

Accepted

Created

30-Mar-2021

Type

Standards Track

[brisk hedge]

there seems to be a lot of disparate ways to handle sentinels so I am hoping the pep is able to address that then

[oblique urchin]

oops no, 661?

!pep 661

[rough sluice]

**PEP 661 - Sentinel Values**

Link

Status

Draft

Created

06-Jun-2021

Type

Standards Track

[brazen jolt]

I've seen people define newtype for just object and casting sentinel values

it wasn't a pretty code though

[spice locust]

my code looks a little like this

def func(a: str = None, b: int = None):
    if a and b:
        raise Exception("Both a and b provided")
    if not a and b:
        raise Exception("Neither a nor b provided")
    # rest of code

will it work the same way if i use overloads?

[trim tangle]

Why not just have two functions?

[spice locust]

there are other parameters, a and b are just supplementary

[brazen jolt]

the fact that this allows to pass these params to be passed positionally makes it difficult

[trim tangle]

yeah, * might be a good idea

[spice locust]

the example was a bit oversimplified. in the actual code they are keyword only

[brazen jolt]

you could define an overload for just a as positional and then kw only overloads

but that's pretty weird

if they are kw only, it would be very similar to my example above

[spice locust]

ok, what about if the user could pass neither a nor b, either a or b, but not a and b?

[brazen jolt]

@overload
def foo(other_param: str, *, a: str):
    ..

@overload
def foo(other_param: str, *, b: int):
    ...

@overload
def foo(other_param: str):
    ...

def foo(other_param: str, *, a: Optional[str] = None, b: Optional[int] = None):
    # impl

[spice locust]

thank you. in the actual implementation would i still need to check if the user provided both and raise an exception?

[brazen jolt]

well, it should

unless it's only something used internally and you know that won't happen

but even then, it probably should

[trim tangle]

not all users use type checkers

and, well, sometimes types lie

[soft matrix]

This was removed from pep 484 I'm surprised pyright hasn't removed it

[spice locust]

the code isnt for internal use, hence all the type hinting and safeguards :)

[trim tangle]

yeah, internal code doesn't need any safety, YOLO 🎉

[spice locust]

if "internal" means the only person using it is the person who wrote it, that is

[brazen jolt]

they kept it as bool setting value which defaults to not allowing that, but since for a long time, pyright had the default behavior of allowing things like x: int = None and inferring that it's optional, they didn't want to just break backwards compatibility without a way to go back

it may get removed in the future, but I don't think that will happen any time soon

[soft matrix]

Pycharm used to have this but it's a bad feature imo

[trim tangle]

(I'm just kidding, I understand what you mean, open source code probably needs better documentation and fool-proofness because you don't interview people before letting them touch it)

[hearty shell]

And also not you in 4 months xD

[soft matrix]

It also use to still have type issues

[spice locust]

to recap, do i or do i not use Optional when an argument is actually optional? (meaning the user can omit it and it will get filled in by some other means)

[brazen jolt]

if those other means is None, then yes

[spiral fjord]

I like doing type | None

[brazen jolt]

if the default value is of the same type, then no

[spice locust]

thats Union[type, None] in python 3.10, right?

[brazen jolt]

def foo(x: Optional[str] = "a"):
    ...

foo(x=None)  # valid

[trim tangle]

Argument being optional is unrelated to whether its type is Optional, yeah

[brazen jolt]

only use optional if you want to allow the value to be None

[trim tangle]

the naming is a bit confusing

[brazen jolt]

whether that None comes from a default value, or from the user, that's irrelevant

[spice locust]

so since the default is None, that means i should use optional

[hearty shell]

Yeah

[spice locust]

got it. thanks

[brazen jolt]

yes, because that parameter can be None

[spice locust]

when using overloads, is it ok to omit the type hints in the actual function definition?

[hearty shell]

Omitting it wont impact your function specification but it is good for internal correctness

So that you can properly handle all cases inside of the implementation

[spiral fjord]

@overload
def utf8(value: None) -> None:
    pass
@overload
def utf8(value: bytes) -> bytes:
    pass
@overload
def utf8(value: unicode) -> bytes:
    pass
def utf8(value):
    <actual implementation>```
In this example from the pep they omit them

[brazen jolt]

omiting isn't an issue, but within the actual implementation, it will mean that those parameters will have a type of typing.Any, and will therefore allow you to do basically anything

that would mean that for example with the above, if you were to do value.xyz type-checker wouldn't detect an error there, even though it's not something that's actually accessible

[spice locust]

now that ive played around with overloading a bit, i have another question

couldnt i also indicate a parameter is optional by doing this?

@overload
def func(a: str): ...

@overload
def func(a: str, b: str): ...

def func(a, b = None):
    # impl

[brazen jolt]

you could, but why would you?

there's no reason to prohibit the user from doing func("hi", None)

it is what you'll get in the implementation anyway from the default value

[spice locust]

alright

[blazing nest]

Yeah, I used to like this a lot but I've since realized its downside: being able to re-export a function.

def func(a: atr = None): ...
def wrapper(a: str = None):
    return func(a)  # Can't pass 'Optional[str]' to 'str'

[fossil nest]

how would you type hint a dictionary (specifying key and value types)

it looks like you can do:

dictionary: dict[str, str]
# and
dictionary: dict[str: str]
```which is better?

[void panther]

Well, only the first is valid

[fossil nest]

really? vscode isnt protesting to the second one for me

with the key being tuple[str, str] and the value being typing.Callable

[void panther]

Then you're probably not type checking, both pyright and mypy don't like it

[fossil nest]

I'm not type checking, What do you mean? is this vscodes auto type hinting tool thingy messing up?

[trim tangle]

Are you using pylance?

Go to settings -> find "type checking" -> set to Basic

[acoustic thicket]

how is that valid syntax

oh wait

slicing

thats funny

[fossil nest]

oo fun, i never knew about this. time to put it on strict :)

wow that is pain having to write that all out

is there any way to type hint a method a will block indefinitely?

[brazen jolt]

typing.NoReturn

[fossil nest]

ah thank you

[brazen jolt]

though it will be renamed to Never in 3.11

but NoReturn will still work, at lest for a while

if you won't need compatibility with 3.10 once 3.11 is released, I'd suggest using typing.Never though, since it's a bit cleaner in what it represents

people often confuse NoReturn with no return statement, hence returning None, which is not what this annotation is for

[brazen jolt]

you can use typing_extensions.Never , which is a backport

[fossil nest]

ah thats fine, ill probably stick with 3.10 for a while so NoReturn works fine 😄

[trim tangle]

1. You probably need Iterator and not Generator, unless you are doing something strange
2. Make a type alias for the Generator thing since it's repeating a lot
3. Using qualified import of typing makes it more of a pain. Just import names from typing
4. Since 3.9, Generator and Iterator and other stuff needs to be imported from collections.abc (see PEP 585, and I also made a thing: https://github.com/decorator-factory/flake8-pep585)

[fossil nest]

im using a generator to yield (infinite) values tho, is this considered as an iterator?

[brazen jolt]

nice! I was looking for something like flake8-pep585 a while ago and couldn't find anything, good to know!

[trim tangle]

Every generator is also an iterator

But maybe you need throw, I don't know

[trim tangle]

Sound like I finally made something useful 🙂

[fossil nest]

i dont need throw. for context its a method that yields a tuple when called with next() and it will never cause a stop iteration error (while True: yield...), so this should be an iterator instead?

[brazen jolt]

generators can do a bit more than just iterators, you can even send them values, but in vast majority of cases, you don't need to do that, and so using iterator as the type-hint is sufficient

[trim tangle]

You can't really type an infinite iterator/generator. But yes, if you only need next, you need an iterator

[brazen jolt]

with the Generator annotation, it takes in YieldType, SendType, ReturnType parameters

if you're only going to be yielding, that's exactly what Iterator is for

[fossil nest]

so this looks a bit better? ill put in an alias as fix error suggested as well to make it a bit cleaner

[brazen jolt]

it's a lot shorter than the original, so I'd say yes, but again, you should consider doing from typing import Iterator, instead of typing.Iterator, and as fix error said, it should actually be from collections.abc import Iterator as of 3.9

and you may want to make a type-alias for that entire type if you're going to be repeating it that much

from typing import TypeAlias
from collections.abc import Iterator

MyIterator: TypeAlias = Iterator[str, socket]

_tasks: list[MyIterator] = list()
_rdict = dict[socket, MyIterator] = dict()
_wdict: dict[socket, MyIterator] = dict()

[spice locust]

do i understand correctly that when type hinting a list-like parameter i should use typing.Sequence, a dict-like parameter should be typing.Mapping, and a dict-like parameter that gets modified should be typing.MutableMapping?

[brazen jolt]

well, sometimes you do want to use a list explicitly

you should only use sequence if you know you won't need anything list-specific

[buoyant swift]

there aren't really that many, though. the general advice is "accept as broad as you can"

[spice locust]

if the function just iterates over it with a for loop, Sequence is correct?

[brisk hedge]

Iterable would be broader

[brazen jolt]

a sequence is an abstract base class, defining that a class of that type will implement __len__ and __getitem__

while if you were to for use a list type explicitly, you know it will also support append function for example, etc.

similarly with Mapping, with a dict, you know there will also be methods like setdefault or get, while a simple Mapping type only tells you that you will be able to use setitem and getitem with something hashable as a key to obtain some value

[brisk hedge]

i think that part is clear already

[brisk hedge]

in a nutshell yes, you should accept types with as little functionality as you use

[brazen jolt]

yeah, unless you have some specific reason to use an exact type, don't do it

[spice locust]

.items() is only available for a dict right?

[brazen jolt]

yes

however a Mapping would allow you to do (k, some_mapping[k] for k in some_mapping)

which would be effectively the same

[brisk hedge]

no

[brazen jolt]

oh interesting

[brisk hedge]

https://docs.python.org/3/library/collections.abc.html

[brazen jolt]

didn't think pure mapping had items, figured it was only values and keys

good to know

[buoyant swift]

well if you have values and keys can't you just make items

[brisk hedge]

hence why it's a mixin method

[spice locust]

i see some standard lib functions with annotations like

(function) func: (a: str, b: str | None = ..., c: int = ...) -> None

is the ... any different from None?

[brisk hedge]

It's some omitted default value

for b it's probably None but you can't really know unless you look closer at the signature

[spice locust]

when i look at the source, the default for b is None and the default for c is an int like 0

why is the default value omitted?

[hearty shell]

To the typecheker that is irrelevant so there is no point adding that to a stub

[spice locust]

i see

[brisk hedge]

i guess it doesn't matter for the signature as long as it's the same type as annotated

[spice locust]

but wouldnt the stub be more useful if the user knew the default value

[brisk hedge]

it could be a pretty complex value

and anyways that's more for the documentation

[hearty shell]

What does vscode display? 🤔

[spice locust]

this

[hearty shell]

Yeah I guess it is what Olivia said then

[cedar sundial]

How do I tell mypy that I know what a type will be? E.g

def func(arg: dict[str, list[int]] | None = None) -> None:
    if isinstance(arg, type(None)):
        arg = {"test": [1, 2, 3]}
    var = arg.get("test")

Item "None" of "Optional[Dict[str, List[int]]]" has no attribute "get"

[hearty shell]

if you do if arg is None then it works

[hearty shell]

Has anyone floated around supporting a special cased type hinting for the builtin configparser and the upcoming tomlib libraries for ini and toml files? Or is this kind of stuff outside the scope of typehinting in the near to mid future?

[soft matrix]

Maybe take a look at try_cast on pypi

[hearty shell]

Humm interesting, still it is a bit hacky imo that you have to keep a python version of a config file in sync with a standardised format

[summer berry]

Thing = list[Union[int, "Thing"]] works but Thing = list[int | "Thing"] results in a TypeError cause a "str" operand is not supported for the pipe. I tried removing the quotes and using from __future__ import annotations but that doesn't help. So is Union the way to go here?

[oblique urchin]

depends on your Python version, but it sounds like yes

[summer berry]

3.10

[oblique urchin]

The future import doesn't help because type alases aren't syntactically in annotations

If you're on 3.10 then list[int | "Thing"] works

[summer berry]

It doesn't for me.

Well, my full alias is ```
List = list[int | float | bool | str | "List"]
E TypeError: unsupported operand type(s) for |: 'types.UnionType' and 'str'

[oblique urchin]

oh interesting, indeed it doesn't for me either. That's odd, I thought we made type.__or__ accept anything

[brisk hedge]

Another ugly solution is to use an if TYPE_CHECKING: block

[oblique urchin]

or list["int | Thing"]

[summer berry]

Yeah but if I have to import TYPE_CHECKING I might as well just import Union

[hearty shell]

If you do foo: TypeAlias = "" then any string inside would be a valid type hint

[summer berry]

Undecided if I prefer this over Union but I will keep it in mind, thanks

[tranquil turtle]

Thing: TypeAlias = "list[int | Thing]"

[tranquil turtle]

how to assign callable to instance attribute?
this dont work in runtime and mypy complains:

from typing import Callable

class X:
    factory: Callable[[int], int]

    def __init__(self, factory: Callable[[int], int]) -> None:
        self.factory = factory
        # error  mypy:assignment      Cannot assign to a method
        # error  mypy:misc            Invalid self argument "X" to attribute function "factory" with type "Callable[[int], int]"
        # error  mypy:assignment      Incompatible types in assignment (expression has type "Callable[[int], int]", variable has type "Callable[[], int]")

i can use Cells or change __getattribute__, but i want to do that in the most efficient way

[hearty shell]

I think if you wrap the callable in ClassVar it works but I guess that might suggest it is available even prior the initialization

I mean

I think ClassVar is what makes sense here no?

[tranquil turtle]

i can just comment typehint in class body!

from typing import Callable

class X:
    # factory: Callable[[int], int]

    def __init__(self, factory: Callable[[int], int]) -> None:
        self.factory = factory

[tranquil turtle]

no, i want to use different factories for different instances

[brazen jolt]

huh, pyright doesn't mind this, it's just mypy being weirdly strict about allowing you to assign to parameters of a class, which are of callable type

[oblique urchin]

yes, it's a longstanding mypy bug

https://github.com/python/mypy/issues/708

[brazen jolt]

I suppose by defining it above, it thinks factory is a function of that class, and it doesn't like you reassigning it

[hearty shell]

It is not that, it is how it handles the self param I think

[oblique urchin]

right

[brazen jolt]

ah yeah, interesting

glad I don't use mypy, lol

[hearty shell]

Though I remember seeing it somewhere that it was intentional? Or maybe it was Eric saying that if you really wanted that to be available at the class (and thus have it be bound) you need to wrap it around ClassVar

[oblique urchin]

I think were was an attempt to fix it, but we had to revert it because of some bad side effects

[tranquil turtle]

from __future__ import annotations
from typing import Callable, TypeVar

KT = TypeVar('KT')
VT = TypeVar('VT')

class computed_dict(dict[KT, VT]):
    __slots__ = ('factory',)

    def __init__(self, factory: Callable[[KT], VT]) -> None:
        self.factory = factory

    def __missing__(self, key: KT) -> None:
        self[key] = self.factory(key)

d = computed_dict[int, int](lambda i: i**2)
print(d) # {}
d[2]
d[4]
d[5]
print(d) # {2: 4, 4: 16, 5: 25}

im implementing dict subclass which computes values if key is missing
maybe it's already implemented somewhere?

its very similar to defaultdict

[oblique urchin]

how is it different from defaultdict at all?

[buoyant swift]

defaultdict doesn't compute based on the key

[tranquil turtle]

defaultdict calls factory without arguments, my dict calls factory with one argument: key

[buoyant swift]

this is what defaultdict should have been, tbh

[brazen jolt]

why not just override __missing__ of defaultdict then?

hm, though you may want to define the correct type on __init__'s factory as Callable that takes in the key

[hearty shell]

Would you not expect missing to actually return the value though?

[tranquil turtle]

def __missing__(self, key: KT) -> VT:
    res = self[key] = self.factory(key)
    return res

``` fixed

[cunning raven]

How can i type hint the class itself in return without actually doing -> "TheClass"

[summer berry]

You can get rid of the quotes I believe if you use from __future__ import annotations

Otherwise you have to wait until 3.11 for https://peps.python.org/pep-0673/

[cunning raven]

class Example:
  def some_func(self) -> "Example":
    ...

ohhh

really?

[hearty shell]

If you are using pyright you can use the typing-extension package to do that

[oblique urchin]

you don't have to wait, typing_extensions.Self already works

(though not on mypy)

[hearty shell]

from typing_extensions import Self

class Example:
  def some_func(self) -> Self: ...

[cunning raven]

thx all

[hearty shell]

test1: Callable[[], Literal["Foo"]] = lambda: "Foo"
def test2() -> Literal["Foo"]: ...

foo = functools.cache(test1)()
bar = functools.cache(test2)()

reveal_type(foo)  # Type of "foo" is "str"
reveal_type(bar)  # Type of "bar" is "Literal['Foo']"

Special cases ftw

[trim tangle]

Any good examples about why object should be preferred over Any?

The ones I've got:

1. httpx.Response.json returning Any and a stupid programmer (me) calling await response.json(). This is probably not the worst use case for Any though
2. Using Mapping[str, Any] for application settings allows you to be lazy with validation and stuff. But this is already so problematic that Any is not the big issue here

[oblique urchin]

more broadly, Any is a code smell, so it's a worthy goal to try to get rid of it in your code

[brisk heart]

I wish I could avoid using Any in my reflection stuff but it's practically damn impossible to properly do some meta-type-hints

[trim tangle]

Yeah I agree, I'm just writing an article for my github pages thing and I'm looking for examples of "Any == bad"

and perhaps some valid use cases of Any

[brazen jolt]

I had to go with any for wrapper function that was using struct.pack along with some special pre-processing and returning the output, which is a tuple of values dependant on the input

this could've been handled by overloads, but sturct.pack supports formats like fff for three floats, so overloads would soon get very messy there

[brisk heart]

I use dict[str, Any] for some of my pydantic validators because nobody would actually interact with that

[brazen jolt]

Any was the best decision there imo, since I really don't want to write that many overloads and if I were to use object, I'd have to use casts everywhere when getting the value out

[trim tangle]

couldn't you have used dict[str, object]?

I think object vs Any is mostly for the code working with the value. Like, so that you don't accidentally do foo["bar"].launch_missiles()

[brisk heart]

Well the structure is predictable but I just didn't feel like writing out typeddicts that would be almost identical to the defined model except for one change

Like if I wanna do PROPER_VALUES[data["foo"]] and know in advance that the key is going to be an integer I do not think extra asserts really help anything

I try to be as precise as possible for anything the user interacts with but a one liner really doesn't deserve an entire typed dict imo

[trim tangle]

Maybe I misunderstand what a pydantic validator is. Does it accept already validated values? Or does it receive unvalidated input?

[brisk heart]

It's more like a converter honestly

It can be used to validate but the most common use is parsing

What's I'm trying to say is that sometimes the extra leeway that Any gives is much better than having several lines of type validation

[trim tangle]

I agree, that's probably also the case with returning Any vs object or big union from Response.json().

On the other hand, with Response.json(), it's probably not a good idea to not validate the response of an external service.

[brisk heart]

A big union is probably even worse than object honestly

[brazen jolt]

I always use a type alias for json outputs

[brisk heart]

Sometimes pyright goes out of its way to ignore an explicit Any annotation because it supposedly knows what the type is

[brazen jolt]

that alias is a big union, but at least it's clear on what it is

then again, there's always an isinstance check afterwards to narrow that type, or raise exc otherwise

[trim tangle]

@brazen jolt What if you want to provide a custom JSONEncoder that parses {"__datetime__": <UNIX timestamp>} into a datetime object?

not sure if that's a good idea, but that's generally in the realm of possibility

well, to be fair, if you're doing that, it's probably a good idea to parse the data into proper objects with a defined set of keys

[brazen jolt]

ig use Union[JsonType, something]

[trim tangle]

I would just return object and then use some standard mechanism for parsing into a proper object

(unless you're doing some generic transforms over JSON)

[brazen jolt]

yeah, I mean there's usually not much shared between the union types there anyway, so object would probably actually be a better solution anyway

[grave fjord]

@trim tangle I think it's a historical artifact of when the json.loads API was first annotated. There's was more of a focus on avoiding false negatives and gradual typing

[trim tangle]

I don't think I understand you

[oblique urchin]

I think he's saying that's why json.loads is annotated as returning Any

[terse sky]

Probably just lack of support for recursive types

You could have an overload for custom validators that returned Any if you wanted, based on how the API is structured, iirc

[oblique urchin]

Recursive types don't help all that much there, you still get the standard issues with Union return types

[grave fjord]

It's also got all those fancy hooks

[little hare]

how can i make a function define that it returns a class that is a mix of the decorated class and another class?

tldr trying to typehint mocks made with unittest mock :^)

[hearty shell]

https://github.com/python/typing/issues/213

Not possible atm

[little hare]

@oblique urchin do you have any updates on quora/pyanalyze#181 and the above issue? I see you were looking to possibly implement it 👀

[mighty linden]

GitHub

Add an Intersection type

[oblique urchin]

yeah I haven't gotten around to it though it still seems interesting. I heard rumors that people are working on a PEP for intersection, but nothing has come of it

Support for arbitrary intersection types seems quite hard though. I would find it most useful for intersecting Protocols

[little hare]

true

[acoustic thicket]

what makes it hard?

[spiral garnet]

Hello there, is there any way to shorten a Union by creating a type that contains multiple possible types. Like

Union[str, int, bool]

Is long to type and i wanted to do something like

NewType["NewType", str, int, bool]

But well not working as expected
Ty 😄

[hearty shell]

You can use type aliases

StrIntBool: TypeAlias = Union[str, int, bool]

[spiral garnet]

Oh wait, yes typealias could do this

[hearty shell]

the : TypeAlias is not necessary here but it helps with self documentation

[spiral garnet]

Yep, okay thanks 😉

I read the doc and was like "oh type alias could be cool" and did not tilt that was a solution ><

give me coffee

[pastel egret]

Also you can use type variables to parameterise.

from typing import TypeVar, Tuple, Union
T = TypeVar('T')
PairOrSingle = Union[T, Tuple[T, T]]
PairOrSingle[int] # = Union[int, Tuple[int, int]]

[trim tangle]

All problems in computer science can be solved with a new level of indirection 🙂

[tranquil turtle]

in case I want to use the Descriptor only in class X, how do I correctly specify the types of the descriptor methods?
are these the correct type-hints?

class Descriptor:
    def __get__(self, inst: X | None, owner: type[X]) -> Any:
        ...

    def __set__(self, inst: X, value: Any) -> None:
        ...

    def __delete__(self, inst: X) -> None:
        ...


class X:
    d = Descriptor()

general case:

class Descriptor:
    def __get__(self, inst: object | None, owner: type) -> Any:
        ...

    def __set__(self, inst: object, value: Any) -> None:
        ...

    def __delete__(self, inst: object) -> None:
        ...

corresponding docs: https://docs.python.org/3.11/reference/datamodel.html#object.__get__

[grave fjord]

Can you make a mypy play?

[tranquil turtle]

wtf

from __future__ import annotations
from typing import Generic, TypeVar
T = TypeVar('T')

class X:
    def __init_subclass__(cls) -> None: ...

class Y(X, Generic[T]): ...
class Z(Y[int]): ...

# Traceback (most recent call last):
#   File "D:\thisfile.py", line 9, in <module>
#     class Z(Y[int]): ...
#   File "D:\Programs\Python\310\lib\typing.py", line 309, in inner
#     return cached(*args, **kwds)
#   File "D:\Programs\Python\310\lib\typing.py", line 1341, in __class_getitem__
#     if any(isinstance(t, ParamSpec) for t in cls.__parameters__):
# AttributeError: type object 'Y' has no attribute '__parameters__'

if i comment __init_subclass__ it works fine: ```py
class X:
# def init_subclass(cls) -> None: ...
...

[oblique urchin]

maybe call super().__init_subclass__()

[tranquil turtle]

🤔

which class in typing also implements __init_subclass__?

Generic?

[oblique urchin]

presumably

[tranquil turtle]

https://github.com/python/cpython/blob/main/Lib/typing.py#L1837

[rough sluice]

Lib/typing.py line 1837

def __init_subclass__(cls, *args, **kwargs):```

[tranquil turtle]

yeah

ok, this works:```py
class X:
def init_subclass(cls) -> None:
super().init_subclass()
# my implementation

[soft plume]

is the correct way to cast a list of dicts coming in externally (either file or api)?

T = TypeVar("T")

def get_prompt(path: str, cls: Type[T]) -> List[T]:
    with open(f"data/prompts/{path}", encoding="utf-8") as file:
        return [cls(**p) for p in yaml.safe_load(file)]

[grave fjord]

That's not a cast

And there's no one correct way to restructure from yaml

there's loads: pydantic, cattrs, marshmallow etc

f-strings for paths is usually catastrophically wrong

And it's usually better to send yaml.safe_loads a bytes file - as yaml documents use a BOM to support various encodings

[grave fjord]

But other than that it's fine

[trim tangle]

well

it doesn't ensure that the values from the YAML have the correct type

which will lead to issues downstream

[soft plume]

this is how it be called

from pydantic import BaseModel, EmailStr, Field



class TypeEnum(str, Enum):
    CONFIRM = "confirm"


class PromptModel(BaseModel):
    message: str
    name: str
    type: TypeEnum

initial_prompt = get_prompt("personal/initial.yml", PromptModel)

@grave fjord @trim tangle

so that still wont ensure the right valus?

[grave fjord]

pydantic already has fancy restructure functions

[soft plume]

I feel like when I call cls(**p) it will error out through pydantic

[grave fjord]

You don't need to make your own

[soft plume]

I googled "pydantic restructure functions" and couldn't find what you are referring to

are you referring to this?
https://pypi.org/project/pydantic_loader/

@grave fjord

[grave fjord]

This get_prompt abstraction isn't useful imho

[trim tangle]

@soft plume I think what graingert suggests is having something like py class Prompts(BaseModel): prompts: list[Prompt] and then doing Prompts(prompts=yaml_data)

[grave fjord]

Yeah currently your get_prompts is sensitive to which class gets passed and the current working directory

[soft plume]

this is good feeback. thanks!

[tranquil turtle]

@functools.cache spoils function signature, so i use this: @(lambda x: x) if TYPE_CHECKING else functools.cache
🤯

[trim tangle]

finally some decent use for the new 3.9 decorator syntax /s😛

[hearty shell]

Off topic for this channel, but it is part of the python format spec, {:,.0f} just means "format as a float, use comma as a separator for the thounds and zero precision"

https://docs.python.org/3/library/string.html#format-string-syntax

[slow chasm]

Thanks!

deleted the msg

[rare scarab]

Is there any way to return a typed dict with keys based on arguments? In typescript, it would be this.

function doSomething<T extends string>(keys: T[]) => Record<T, string> {
  // ...
}

// doSomething<"foo" | "bar">(["foo", "bar"]) => {foo: string, bar: string}
const {foo, bar} = doSomething(["foo", "bar"]);

[oblique urchin]

def foo(x: Sequence[T]) -> dict[T, str]: should work, though it generally won't infer a TypedDict

[rare scarab]

So it won't infer the type a dict[Literal["foo", "bar"], str]?

[oblique urchin]

depends on the type checker, but generally not I expect

[rare scarab]

I was hoping to have a function that allows specifying of fields for an api

Worst case: have the caller provide their own type.

Though casting to dict[Literal["foo", "bar"], str] works

[blazing nest]

I don't remember, did we ever reach a good solution for generic class attributes? That is, in my case, two Optional[] attributes which has to either be defined together or both None

[tranquil turtle]

@overload
class X:
    x: None
    y: None

@overload
class X:
    x: int
    y: str

class X:
    x: int | None
    y: str | None
    ...

[hearty shell]

You can probably use __new__ to achieve something like this

Lot of boilerplate though x)

from __future__ import annotations
from typing import *

T1 = TypeVar('T1')
T2 = TypeVar('T2')

class Foo(Generic[T1, T2]):
    x: T1
    y: T2

    @overload
    def __new__(cls) -> Foo[None, None]: ...
    @overload
    def __new__(cls, x: T1, y: T2) -> Foo[T1, T2]: ...

    def __new__(cls, x: Any = ..., y: Any = ...) -> Foo: ...

reveal_type(Foo(3, "foo"))  # Type of "Foo(3, "foo")" is "Foo[int, str]"
reveal_type(Foo(3))  # No overloads for "__new__" match the provided arguments
reveal_type(Foo())  # Type of "Foo()" is "Foo[None, None]"

Humm... And no way to subclass this

[hearty shell]

Apparently this works too, but it looks like a bug

class Foo(Generic[T1, T2]):
    x: T1
    y: T2

    @overload
    def __init__(self: Foo[None, None]): ...
    @overload
    def __init__(self, x: T1, y: T2): ...
        
    def __init__(self, x: Any = ..., y: Any = ...): ...
    
reveal_type(Foo(3, "foo"))  # Type of "Foo(3, "foo")" is "Foo[int, str]"
reveal_type(Foo(3))  # No overloads for "__new__" match the provided arguments
reveal_type(Foo())  # Type of "Foo()" is "Foo[None, None]"

[oblique urchin]

what part seems buggy to you? It's an established mypy feature, we use it in typeshed

[hearty shell]

I say bug because the initializer is not really provided "evidence" that it is of type Foo[None, None] when it is called

[oblique urchin]

overloads are like that

[hearty shell]

Humm I get that the overload matches since it is not known what type the foo is, I guess it is just the fact that the class assumes that as its type post matching

Like this

class Bar(Generic[T1]):
    @overload
    def bar(self: Bar[int]) -> int: ...
    @overload
    def bar(self: Bar[str]) -> str: ...

    def bar(self: Any) -> Any: ...

a = Bar()
reveal_type(a.bar())  # Type of "a.bar()" is "int"
reveal_type(a)        # Type of "a" is "Bar[Unknown]"

[oblique urchin]

That seems wrong

Is that pyright picking the first overload even in the presence of Any?

[hearty shell]

Yes? I am confused now

[hearty shell]

What did you mean by this?

[oblique urchin]

Overloads aren't safe in general. You trust the author of the overloads that the return types are right

The type checker doesn't check

[hearty shell]

The part that I said seemed like a bug was having the type checker match the first overload and then at the same time infer its own type when based on the general match

I think this is something exclusive to matching an __init__

[oblique urchin]

Not sure what you mean by that

[hearty shell]

I just mean that when matching the overloads, it is picking the first case where the types matches, which is what is happening on Bar, since it is initialised with an annotation it assumes Bar[Any] and then when matching on the overload Bar[int] is compatible. But on Foo, not only does it match the Foo[None, None] but that also has the side effect of telling the type checker that it is indeed a Foo[None, None]

It matches a specific case from a general Foo[Any, Any] and then takes that match as evidence of being Foo[None, None]

[oblique urchin]

But in that example it's a simple match, it's just based on parameter counts

Pyright's behavior of matching against Any in overloads is a bit problematic, mypy and pyanalyze do it differently. https://pyanalyze.readthedocs.io/en/latest/reference/signature.html#pyanalyze.signature.OverloadedSignature.check_call

But your Foo example doesn't rely on that

[hearty shell]

It seems to have the same behaviour in mypy

https://mypy-play.net/?mypy=latest&python=3.10&gist=c386205e6433f4b23037a1eef7f01fd3

[oblique urchin]

pyanalyze produces ```Revealed type is 'Any[multiple_overload_matches]' (code: inference_failure)
In /Users/jelle/py/tmp/baroverl.py at line 21
18:
19:
20: a = Bar()
21: reveal_type(a.bar())
^
22: reveal_type(a)

[trim tangle]

change py class Foo: a: int b: str | None c: float | None to ```py
class Bar:
b: str
c: float

class Foo:
a: int
bc: Bar | None

Maybe there's some deep relation between this and database normalization, but I'm not that deep into it 🙂

(get it? relation? anyway...)

[brisk heart]

I don't really get it but I love the vibe

[spare mauve]

can someone help me interpret this

  Type "(self: Self@Setup, ctx: Context[Unknown]) -> Coroutine[Any, Any, None]" cannot be assigned to type "((CogT@Command, ContextT@__init__, **P@Command) -> Coro[T@Command]) | ((ContextT@__init__, **P@Command) -> Coro[T@Command])"
    Type "(self: Self@Setup, ctx: Context[Unknown]) -> Coroutine[Any, Any, None]" cannot be assigned to type "(CogT@Command, ContextT@__init__, **P@Command) -> Coro[T@Command]"
      Parameter 1: type "CogT@Command" cannot be assigned to type "Self@Setup"
        "Cog*" is incompatible with "Setup"
      Parameter 2: type "ContextT@__init__" cannot be assigned to type "Context[Unknown]"
        "Context[Unknown]*" is incompatible with "Context[Unknown]"
    Type "(self: Self@Setup, ctx: Context[Unknown]) -> Coroutine[Any, Any, None]" cannot be assigned to type "(ContextT@__init__, **P@Command) -> Coro[T@Command]"
      Parameter 1: type "ContextT@__init__" cannot be assigned to type "Self@Setup"```

this is the code where the warning appears

  @Command
  async def ping(self, ctx: commands.Context):
    await ctx.send(f"the bot ping is currently: {round(bot.latency * 1000)}ms")

it points to the @Command

[hearty shell]

What is Command?

[soft matrix]

Command shouldn't be used as a decorator

Use command

[oblique shadow]

How do you typehint a list containing itself?

[trim tangle]

What does that mean?

Like this?

xs = []
xs.append(xs)

With pyright you can do

Foo = list["Foo"]

xs: Foo = []
xs.append(xs)

But this kind of list is pretty useless

[oblique shadow]

I'm not asking for the practical use, just how you'd hint it in vanilla Python

[terse sky]

That's not really a type

A list that contains lists, sure

Mypy doesn't do well with recursive types but iirc one of the other type checkers can do it

[trim tangle]

What do you mean by "vanilla Python"?

Without an external tool, type annotations don't mean anything. Mypy doesn't support recursive types so with mypy you can't do it. With Pyright you can use a type alias as I've shown above

[oblique shadow]

I don't really know. I'm somewhat new to typehinting, so I'm curious about various ways to achieve this

So there's no way to make stuff like

type_of_s=List[type_of_s]

without aliasing?

[trim tangle]

No

[brisk hedge]

well, Thing = list['Thing'] might be accepted by pyright

but mypy still has influence over the typing of the language

[rare scarab]

I like to use Thing: TypeAlias = "list[Thing]"

maybe even Thing: TypeAlias = "str | int | float | bool | dict[str, Thing] | list[Thing] | None" <- json values

[trim tangle]

Nobody expects the Spanish inquisition!
But did you expect permalinks in Pyright Playgound?
I hope you did, because I just added them.

1. Go to https://pyright-playground.decorator-factory.su/
2. If you don't see a big "Generate permalink" button, purge the cache and reload
3. Type in some code
4. Press the "Generate permalink" button
5. Now your address bar has a permalink, such as https://pyright-playground.decorator-factory.su/?gzip=H4sIAFERkWIC_0tJTVNIT81LLUosSY0vSC3KTczJzMvW0LTiUgCCotSS0qI8BaWMkpKCYit9_fLycr3K_NKS0qRUveT8XP3yxJLkDPsy25TAcpNyy_D0iORAJS4A7dJJyFMAAAA%3D

Unlike with mypy-play.net, the link is completely stateless. I stole the idea from the TypeScript playground

[buoyant swift]

interesting function you got there 👀

[trim tangle]

Now we just need for Discord to add inline links 🥴

even telegram has those

[rustic gull]

yeah instead of adding stupid shit like party mode

[ruby sigil]

any guide on how to begin with type hinting?

[tranquil turtle]

https://mypy.readthedocs.io/en/stable/getting_started.html
it is tutorial for mypy, it also describe how to use typehints

[ruby sigil]

thanks

[tranquil turtle]

https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html
https://mypy.readthedocs.io/en/stable/cheat_sheet.html

[gritty crater]

In the event that you need either the file_id, or the file_name, followed by a bunch of kwargs, and want to make the id / name an arg, should you use typing.overload? (and then ensure None not in (file_id, file_name))

[trim tangle]

why do you accept either a file ID or a file name?

can you show the code maybe?

[gritty crater]

Oh I see, this is an unfinished example: py async def download_file( *, file_id: Optional[str]=None, file_name: Optional[str]=None, authorisation: Optional[B2ConnectionInfo]=None, byte_range: Optional[str]=None, content_disposition: Optional[str]=None, content_language: Optional[str]=None, expires: Optional[str]=None, cache_control: Optional[str]=None, content_encoding: Optional[str]=None, server_side_encryption: Optional[str]=None ) -> aiohttp.ClientSession: if file_id and not file_name: # do request elif file_name and not file_id: # do request else: # raise errorI'd like to specify that you must provide either the file_id, or the file_name. The rest are optional. (This is because there are separate API routes for each, download_file_by_id, and download_file_by_id)

[trim tangle]

What are byte_range, content_disposition, content_language, expires, cache_control, content_encoding and server_side_encryption?

sounds like HTTP-level details

also why does it return a ClientSession?

[gritty crater]

ClientSession is returned because this is still in testing, I'm going to make it the response a class eventually, once I get this portion out of the way.

Those parameters are required, according to this: https://www.backblaze.com/b2/docs/b2_download_file_by_name.html

[trim tangle]

I think those are the response headers

Cache-Control : computed from the download request, otherwise the value provided when the file was uploaded, otherwise the value specified on the bucket.

yeah, this definitely sounds like a response header

[soft matrix]

i think the question is why isnt it returning a ClientResponse object

[gritty crater]

Oh I see

On related note, I will need those values for my get_download_authorization request: ```py
async def _get_download_authorisation(
connection_info: B2ConnectionInfo,
bucket_id: str,
session: aiohttp.ClientSession,
file_name_prefix: Optional[str] = '',
valid_duration_in_seconds: int = 604800,
content_disposition: Optional[str]=None,
content_language: Optional[str]=None,
expires: Optional[str]=None,
cache_control: Optional[str]=None,
content_encoding: Optional[str]=None,
content_type: Optional[str]=None
):
account = await _authorise_account(connection_info, session)

await _http(
    f'{account.api_url}/b2api/v2/b2_get_download_authorization',
    session=session,
    method='GET',
    params={
        'bucketId': bucket_id,
        'fileNamePrefix': file_name_prefix,
        'validDurationInSeconds': valid_duration_in_seconds,
        'b2ContentDisposition': content_disposition,
        'b2ContentLanguage': content_language,
        'b2Expires': expires,
        'b2CacheControl': cache_control,
        'b2ContentEncoding': content_encoding,
        'b2ContentType': content_type,
    },
    headers=account.authorisation_token
)```In the event that these are `None`, will the request 'work'?

[trim tangle]

I don't know, you could try 🙂

are you sure those are not response headers as well?

what is bucket_id for example?

[gritty crater]

A bucket is like a storage container for a bunch of files, and every one of em' has an ID. You can only get the download authorisation to a single bucket at a time.

A bit like an AWS S3 bucket

[gritty crater]

Ah, it seems like you are right, they are request headers. It seems like there's a pretty good amount of inconsistency in these docs :(

[trim tangle]

you mean response headers?

yeah it would help if they had explicit "request headers" and "response headers" sections

[gritty crater]

https://static.themilkyway.tech/file/the-void/screenshots/firefox_bBqbnkyZ8f.png

Err - request

The response section is located further down

Oh god, finally got it working: ```py
async def _get_download_authorisation(
connection_info: B2ConnectionInfo,
bucket_id: str,
session: aiohttp.ClientSession,
file_name_prefix: str,
valid_duration_in_seconds: str = '604800',
content_disposition: Optional[str] = None,
content_language: Optional[str] = None,
expires: Optional[str] = None,
cache_control: Optional[str] = None,
content_encoding: Optional[str] = None,
content_type: Optional[str] = None
) -> DownloadAuthorisation:
account = await _authorise_account(connection_info, session)

params = {
    'bucketId': bucket_id,
    'fileNamePrefix': file_name_prefix,
    'validDurationInSeconds': valid_duration_in_seconds,
    'b2ContentDisposition': content_disposition,
    'b2ContentLanguage': content_language,
    'b2Expires': expires,
    'b2CacheControl': cache_control,
    'b2ContentEncoding': content_encoding,
    'b2ContentType': content_type
}
params = {key: value for key, value in params.items() if value is not None}

data = await _http(
    f'{account.api_url}/b2api/v2/b2_get_download_authorization',
    session=session,
    method='GET',
    headers={'Authorization': account.authorisation_token},
    params=params
)

return DownloadAuthorisation.from_response(data)```

Now comes the download_file portion where I need the either the file_name or file_id

[trim tangle]

I think an @overload is fine. But consider if you can make it into two functions

[gritty crater]

Yeah, they'll end up being pretty similar so I'm going to try leaving it as one

@overload
async def download_file(
        *,
        file_id: Optional[str] = None,
        bucket_id: str,
        file_name_prefix: str,
        session: aiohttp.ClientSession,
        connection_info: Optional[B2ConnectionInfo] = None,
        byte_range: Optional[str] = None,
        valid_duration_in_seconds: Optional[str] = '604800',
        content_disposition: Optional[str] = None,
        content_language: Optional[str] = None,
        expires: Optional[str] = None,
        cache_control: Optional[str] = None,
        content_encoding: Optional[str] = None,
        server_side_encryption: Optional[str] = None
) -> DownloadedFile: ...


@overload
async def download_file(
        *,
        file_name: Optional[str] = None,
        bucket_id: str,
        file_name_prefix: str,
        session: aiohttp.ClientSession,
        connection_info: Optional[B2ConnectionInfo] = None,
        byte_range: Optional[str] = None,
        valid_duration_in_seconds: Optional[str] = '604800',
        content_disposition: Optional[str] = None,
        content_language: Optional[str] = None,
        expires: Optional[str] = None,
        cache_control: Optional[str] = None,
        content_encoding: Optional[str] = None,
        server_side_encryption: Optional[str] = None
) -> DownloadedFile: ...


async def download_file(
        *,
        file_id: Optional[str] = None,
        file_name: Optional[str] = None,
        bucket_id: str,
        file_name_prefix: str,
        session: aiohttp.ClientSession,
        connection_info: Optional[B2ConnectionInfo] = None,
        byte_range: Optional[str] = None,
        valid_duration_in_seconds: Optional[str] = '604800',
        content_disposition: Optional[str] = None,
        content_language: Optional[str] = None,
        expires: Optional[str] = None,
        cache_control: Optional[str] = None,
        content_encoding: Optional[str] = None,
        server_side_encryption: Optional[str] = None
) -> DownloadedFile:```

:(

[fervent sierra]

As a word of warning: be careful with overloads. Overloading in python isn't as safe as in compiled languages where the correct overload is picked in compile time: your code has to decide on the correct implementation in runtime. The function that implements the actual logic is prone to typing errors; you'll see that there is no way to guaratee that the implementation is returning the correct type if the overloads return different types; the best your type checker can do for you is verify that you are returning a union of the return types of the overloads, but not the correct one

Also, @gritty crater , if your API requires the file_id or file_name, then make those parameters of type str instead of Optional[str], so that the functions can't be called without them

[solar sphinx]

hi. I'm currently testing pydantic for the first time, I'm wondering, is there a way to access a class' schema either as a dict or a list, without calling object_instance.dict() on an instance of the object? for reference this is the class I'm working with at the moment: class Job(BaseModel): job_id: int title: str departments: str location: str internal_id: int updated_at: datetime requisition_id: int | None url: str desc: str Ideally I'd like to be able to do something like Job.schema() and get back something like ["job_id", ... , "desc"] (without having to instantiate at all)

[solar sphinx]

update: well it wasn't as straightforward as I hoped, but I got what I wanted via: >>> json.loads(Job.schema_json())["properties"].keys() dict_keys(['job_id', 'title', 'departments', 'location', 'internal_id', 'updated_at', 'requisition_id', 'url', 'desc'])

[spice locust]

whats the difference between typing.Sequence and typing.Iterable?

[trim tangle]

Sequences support indexing (like foo[0]) and length (len), and are also iterable.

And iterables are things that support iter

[spice locust]

alrighty thank you very much

[trim tangle]

Also, you should import these from collections.abc because the ones from typing are deprecated

!pep 585

[rough sluice]

**PEP 585 - Type Hinting Generics In Standard Collections**

Link

Status

Accepted

Python-Version

3.9

Created

03-Mar-2019

Type

Standards Track

[trim tangle]

Also see https://docs.python.org/3.10/library/collections.abc.html#collections-abstract-base-classes

[spice locust]

im targeting python 3.6+ does that matter?

[trim tangle]

Ah, that only applies to 3.9+. So you can ignore that

3.6 is already at end of life, why do you support it?

[spice locust]

compatibility i guess

[trim tangle]

Why not support python 2 then? 🙂

What are you developing?

[spice locust]

the code in question is a package that already supports 3.6+, so i'd rather not drop support for a python version

[tranquil turtle]

is there a way to typehint a specific module?

from types import ModuleType

import sys

def f() -> ModuleType:
    import sys
    return sys

sys2 = f()
reveal_type(sys) # types.ModuleType
reveal_type(sys2) # types.ModuleType
reveal_type(sys.argv) # list[str]
reveal_type(sys2.argv) # Any

i want sys2.argv to be of type list[str]

import sys
class X:
    s = sys

x = X()

reveal_type(x.s.argv)
# misc - Member "s" is not assignable
# Revealed type is "Any"
``` (i use mypy)

from typing import ClassVar
import sys

class X:
    s: ClassVar = sys

x = X()

reveal_type(x.s.argv)
# Revealed type is "Any"

pyright has module aliases

[trim tangle]

You could make a protocol

Do you actually require that it's a module?

[soft matrix]

There was something on the issue tracker about this

But this does look like a bug with mypy

[tranquil turtle]

I want provide reference to some module in instance attribute
I know which module it is, so i can use Protocol instead of module in typehints, but its not good way, because i should duplicate all functions of module into this Protocol class

[trim tangle]

Why do you need this module as a class variable?

[tranquil turtle]

just for convenience so you don't have to import different modules explicitly

[trim tangle]

That sounds less convenient to understand than import sys

[livid cypress]

Has anyone here used django-types package?
I cant seem to set it up

[fervent sierra]

Does anyone know if it's at all possible to express a type like "An ordered mapping where the values are a Sequence[int]"?

Mapping[SomeKey, Sequence[int]] does not imply order as far as I know;
Dict[SomeKey, Sequence[int]] expresses most of what I want, but since the value type is invariant, i can't fill it in with anything, since I can't instantiate the exact type Sequence
Dict[SomeKey, Tuple[int, ...]] works OK, but it's super verbose and I have to convert all values to tuples (so I can't use my lists directly)

[tranquil turtle]

maybe OrderedDict[SomeKey, Sequence[int]] ?

[fervent sierra]

But the value type for OrderedDict is also invariant, isn't it? Since it is writable just like vanilla Dict

[indigo locust]

Anything new and exciting happening in the world of type hinting?

[trim tangle]

How would a type checker use the information that a mapping is ordered?

As far as it's concerned, there is no difference between an "ordered" and "unordered" mapping

[trim tangle]

Dict[SomeKey, Sequence[int]] expresses most of what I want, but since the value type is invariant, i can't fill it in with anything, since I can't instantiate the exact type Sequence
That's not quite how invariance works.
Dict being invariant means that if you have a x: Dict[str, list[int]], you can't assign it to a variable v of type Dict[str, Sequence[int]]. (because you could do v["foo"] = (1, 2, 3) and break the original x dict).

However, it doesn't mean that values of Dict[str, A]can't be subtypes of A. After all, this is perfectly fine, right?: ```py
class A: pass
class B(A): pass

some_a: A = B()

foo: dict[str, A] = {}
foo["bar"] = some_a
``` so you can skip the step and just do foo["bar"] = B(). This is fine because you're "upcasting" a value of type B to a value of type A, but not dict[str, A] to dict[str, B] or vice versa. (you could say that T is covariant in T)

So this is perfectly legal: ```py
from collections.abc import Sequence

foo: dict[str, Sequence[int]] = {}
foo["bar"] = [1, 2, 3, 4, 5]
foo["baz"] = (1, 2, 3, 4, 5)

[fervent sierra]

Regarding the ordering of the dict, indeed, the type system doesn't know about it, but I want to signal to the users of the API that the order is relevant. And since a Dict has a deterministic order but Mapping doesn't, I was dissuaded from using Mapping. I'm aware, though, that the type checker couldn't possibly say anything about any particular order of the Dict keys; it'll just validate that it's a Dict and therefore, implicitly ordered in some sense.

And regarding the variance, you're absolutely right. Every time I think I've got variance down, I stumble on something like this. It's interesting that this is fine

class A: pass
class B(A): pass

x: List[A] = [B()]

but this isn't:

class A: pass
class B(A): pass

list_of_b = [B()]
x: List[A] = list_of_b

since list_of_b in the second example is inferred to be a List[B]

[pastel egret]

Well that still does make sense, since you can next do x.append(A()), but that'd cause list_of_b to be invalid.

[fervent sierra]

Yeah, makes total sense if you're aware that list_of_b will have it's type inferred to List[B], based solely on the line where it's declared, not on its subsequent usage in the assignment. But it definitely caught me off guard -.-

[trim tangle]

Maybe accept a Iterable[tuple[str, int]]?

I think order being indeterministic on a Mapping just means that you can't control it, or it might change if you update it. But I don't think it means that different invocations of items or keys or values will produce different results

[pastel egret]

It's not really something the type system's designed to specify.

[trim tangle]

yep

[trim tangle]

If order is important, why do you accept a mapping? Maybe you could tell more about your problem?

[hasty hull]

Has there been any discussion in pyright for treating a TypedDict as Mapping[str, UnionKeyTypes] instead of just Mapping[str, object]?

[trim tangle]

you mean, when accessing with arbitrary string as key?

hmm that would make sense, but only with a @final TypedDict.

https://pyright-playground.decorator-factory.su/?gzip=H4sIAD4MlmIC_42QzUrEMBSF93mKQ92kUAr-rAKCiPgEuipF0jQZw3SSkqTOtOq7e1Mdf2AWdnWac-85l88Ev0OaR-s2sLvRh4SHedT9nVWpgrFODoypQcaIe-_5t1cKBvoOAjGFVc4C1iXGem1g-FbPq0UZ3ou8-7UR9IuWwxNVak5WQ4NtCZzh0W2d37sK0vVQ0jmf0GlwCsFbji7pV8kp6rqujzfdysB_sheBwcbU0HDLWCeDyAO4xmtxKAQKWVQoZlJXFyQWEs15BdKX7TsznF4q0NZ6DpV4g_RsI2M3nxyOlcs_MGS5otj8QtHJJV-0nEBB1gkU3ZSw99PQZxLKh6BVQvLUYHSAjH_ofABCpVUEywEAAA%3D%3D

[hasty hull]

Ah yeah I always forget about @final

[trim tangle]

well, mypy doesn't support @final on a TypedDict, does it?

[hasty hull]

I don't think so

[trim tangle]

so not sure about the proposal

[soft matrix]

I thought it was going to add support for it

[hasty hull]

I hope so

Would be really nice if this could pass type checks:

from __future__ import annotations
from typing import Mapping, TypedDict, final

FileTypes = bytes

# this has to be key agnostic
RequestFiles = Mapping[str, FileTypes]

def post(files: RequestFiles | None = None):
    ...

@final
class FileParams(TypedDict):
    file: FileTypes

def foo(files: FileParams):
    # error: Argument of type "FileParams" cannot be assigned to parameter "files" of type "RequestFiles | None" in function "post"
    post(files)

[rare scarab]

What if you do class FileParams(TypedDict, RequestFiles)

Nope. ```py
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

[blazing nest]

I think the order should be the other way, but yeah that's a good idea

[rare scarab]

Reversing the order makes it worse.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/.../.pyenv/versions/3.10.2/lib/python3.10/typing.py", line 1095, in __mro_entries__
    return super().__mro_entries__(bases)
  File "/home/.../.pyenv/versions/3.10.2/lib/python3.10/typing.py", line 956, in __mro_entries__
    if isinstance(b, _BaseGenericAlias) or issubclass(b, Generic):
TypeError: issubclass() arg 1 must be a class

[soft matrix]

Just do it in an if TYPE_CHECKING block

[hasty hull]

Pyright reports an error for that as well :/

All base classes for TypedDict classes must also be TypedDict classes

[soft matrix]

I'm pretty sure you can just type ignore though

And stuff will work as expected

[hasty hull]

Yeah, my current solution is to just cast(Any, files) as that's the easiest solution to use with my codegen system

https://github.com/microsoft/pyright/issues/3526

[humble ice]

would you guys say there are any technical or semantic differences between

var: str | None = None 

and

var: Optional[str] = None

i know using pipes is new to 3.10(?) so i haven't come across many or any code bases using it yet
do you think that will come with time?

using Optional kind of gives you some meaning right away just because of the word, but i do like the cleaner look of a | over []

but then again with Optional[] i only need to write None once

[hearty shell]

The technical difference is that at runtime they are completely different things so if a library relies on that for runtime magic then they have to re adapt for that. As for semantics, given | is pythons "or" operator, It has very clear meaning imo

[oblique urchin]

They're equivalent to a type checker. str | None is preferred for 3.10+ code, because the meaning of Optional is confusing, especially in function arguments.

[humble ice]

oh interesting, Optional made intuitive sense to me but i think i've heard from others that they misunderstood it at first

[oblique urchin]

I've seen a lot of people write def f(x: Optional[int] = 0), because x is an optional parameter

when really they just need def f(x: int = 0)

[humble ice]

how often do you see Optional type hints where the default isn't None?

[oblique urchin]

legitimate ones, almost never

but our codebase has >200 places matching 'Optional.[a-zA-Z]+. = [^Nf]'. I suspect most of those annotations are wrong.

[humble ice]

when you say "our" you mean for work? black? cpython?

side question: are there efforts being made to add type hinting to modules in the standard lib?

[oblique urchin]

work codebase

There's no concerted effort to add type hints directly in the stdlib. A few modules do have annotations

[lime wyvern]

how can i type defaultdict(set)? i tried DefaultDict[set] for mypyc but it fails with py : error: "defaultdict" expects 2 type arguments, but 1 given

if i try with py DefaultDict[set, str] = defaultdict(set) i get ```py
domainsorter.py:36: error: "str" has no attribute "add"
domainsorter.py:36: error: Invalid index type "str" for "defaultdict[Set[Any], str]"; expected type "Set[Any]"
domainsorter.py:70: error: Argument 1 to "defaultdict" has incompatible type "Type[Set[Any]]"; expected "Optional[Callable[[], str]]"

[acoustic thicket]

it should look like name: defaultdict[str, set] = defaultdict(set), the type of the key goes first

it'd also be better to be more specific about the set, like defaultdict[str, set[int]]

[lime wyvern]

thank you, that worked, and i typed the set[str] as well, thank you 🙂

[acoustic thicket]

pyright has 0 open issues currently, damn

[trim tangle]

I think that just has to do with how it deals with issues.

Namely, Eric seems to be very eager to close the issues.

[little hare]

microsoft/pyright#1

[mighty linden]

GitHub

Fix typos

[trim tangle]

Which I think is a good thing, at least theoretically. You don't get obsolete problems cropping up. If something is out of scope or not really fixable, close it.

[little hare]

zamn

[hearty shell]

Yeah he is very pragmatic, he will not hesitate to close an issue x)

[little hare]

microsoft/pyright#3514 4 minutes from creation to triage

[mighty linden]

GitHub

Question about the command pyright-langserver

[trim tangle]

What's with the name as designed btw?

When I see this label on a project I always want to respond "well, fix your design then!"

[hearty shell]

Either, but that warning is not related to type hints

If you want any though I would use list[Any]

Humm not from either 3.8 or 3.9 onwords

hold on

3.9 on-words you can just use list[Any], prior to that yes you would need to import from typing

[trim tangle]

Why do you want list[Any] though?

[soft matrix]

is a video id always some random object

id guess that its a str or an integer

you could use list[str | int]

[trim tangle]

Why do you have a mix of strings and integers?

re.findall in this case returns a list of strings

[cedar sundial]

This might be obvious but when typing a nested dictionary, should you annotate each layer?

[blazing nest]

Unless it is infinite, I would recommend you do

[trim tangle]

Why do you have a nested dictionary?

[spiral fjord]

Can you self reference a type alias in a type alias?

[cedar sundial]

It's for multiple results that we get from processing

[trim tangle]

inb4

def y():
    return defaultdict(y)

[blazing nest]

Yeah, if you use Pyright 😅

[trim tangle]

Yes, in Pyright

Tree = Union[str, dict[str, "Tree"]]

[spiral fjord]

problem fixed

[cedar sundial]

Is this something that would be ideal or is there a better way to type a dict: dict[str, dict[str, dict[str | int, dict[str, list[str] | int]]]]

[spiral fjord]

What is a dict[str, list[str], int]?

[cedar sundial]

Oops, meant to be a union

[trim tangle]

If this is some kind of opaque response from a remote API, I would keep it as dict[str, object] or object. But if it has some specific keys and types and you want to work with them, validate it and convert to a proper object (maybe a dataclass).

[spiral fjord]

Are the keys variable?

[cedar sundial]

constant

[spiral fjord]

TypedDicts

[trim tangle]

I think using dicts for structured data is an anti-pattern. You can't add logic or behaviour to them, and you can't express the domain rules in types. A dict is for mapping arbitrary keys of some domain to some values of another domain

Can you perhaps show an example of such dict?

[cedar sundial]

result = {
    "key1": {
        "inner_key1": {
            0: {
                "constant_key1": ["a", "b", "c"],
                "constant_key2": 1
            },
            1: {
                "constant_key1": ["a", "b", "c"],
                "constant_key2": 1
            }
        }
    }
}

Where the inner_key1 keys have a range of 200

[soft matrix]

ranges cant be encoded into types atm

unless you mean it has 1-200 as a suffix

in which case have fun typing all those keys out

[cedar sundial]

The former :p Think I could leave that open as int

[hearty shell]

Does anyone know of a way to disable pyright on vscode ipynb other then turning it off completely?

[vapid quarry]

if we are storing a tuple as a value and the values are of type float, then does the below look correct?

map_pricing_dictionary:dict[str, tuple(float, float)] = {}

or should it be the following instead:

map_pricing_dictionary:dict[str, tuple[float, float]] = {}

[soft matrix]

the latter

[vapid quarry]

ok thank you (:

[soft matrix]

using function calls in a type hinting setting is generally incorrect

[vapid quarry]

What do you mean?
Are you addressing me?

[soft matrix]

yes

[vapid quarry]

where's the function call?

I am just storing tuples in a dictionary

[soft matrix]

tuple(float, float)

(im pretending that the dict[...] isnt a function call)

[trim tangle]

actually it will fail

!e

print(tuple(float, float))

[rough sluice]

@trim tangle :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | TypeError: tuple expected at most 1 argument, got 2

[vapid quarry]

Oh yes!!! I am putting floats in parenthesis(). And generally, functions use parenthesis.
But we are not calling a function here, we are just type hinting so we would use square brackets[] instead.

[soft matrix]

you arent incorrect in this case but if you actually tried this at runtime youd get an error

!e ```py
from typing import TypeAlias
x: TypeAlias = dict[str, tuple(float, float)]

[rough sluice]

@soft matrix :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 2, in <module>
003 | TypeError: tuple expected at most 1 argument, got 2

[rare scarab]

use tuple[float, float] instead of tuple(float, float)

[vapid quarry]

Yes using tuple[float, float] lol

[oblique urchin]

yes, Protocol

[trim tangle]

well, everything has a __repr__ method 🙂

so for that you could use object

</nitpick>

class Writable(Protocol):
    def write(self, chunk: str, /) -> int:
        ...
```or if you don't care about the return type: ```py
class Writable(Protocol):
    def write(self, chunk: str, /) -> object:
        ...

also, if you only need write, maybe accept a callback instead of the full object?

Instead of py def write_lyrics(fp): fp.write("We're no strangers to love") fp.write("You know the rules, and so do I") ... you can do ```py
def write_lyrics(write):
write("We're no strangers to love")
write("You know the rules, and so do I")
...

If someone happens to have an object that supports write, they can just pass the_object.write (a bound method)

But if they don't, they don't need to construct a custom class for that

hm?

oh, you can totally type hint a callable

from collections.abc import Callable

def write_lyrics(write: Callable[[str], object]) -> None:
    write("We're no strangers to love")
    write("You know the rules, and so do I")
    ...

Congratulations @buoyant swift, you were rickrolled by a code block

[vapid quarry]

I'm working on an algo, but cant do the following math 🤦‍♂️
I need to find the price
price + 75 = price * 0.13

Below is the original equation:

profit = price - ((price * 0.13) + cost + shipping)

We have everything except for price

[vapid quarry]

So I guess the final equation would look like:

price = (profit + cost + shipping) / 0.87

[trim tangle]

What does a - (b + c) simplify to?

[vapid quarry]

a - b - c

[trim tangle]

p + 75 = p * 0.13
75 = p * 0.13 - p
75 = p*(-0.87)
p = -75/0.87

Also this is #type-hinting . Do you need this at the type level? You probably want to use Agda then

[trim tangle]

This seems correct though

[vapid quarry]

profit = price - ((price * 0.13) + cost + shipping)
profit = price - (0.13price + cost + shipping)
profit = price - 0.13price - cost - shipping
profit = 0.87price - cost - shipping
# switch sides
0.87price - cost - shipping = profit
0.87price = profit + cost + shipping
price = (profit + cost +shipping) / 0.87

LORD!!!!!

[vapid quarry]

Honestly, had to use an online math solver to come up with the equation.
But when you said this, it helped me actually break it down step by step.

[pale otter]

I'm going to guess price is of type float

[vapid quarry]

Thanks a lot!!

[trim tangle]

In any case... This is probably the wrongest channel on this server

[vapid quarry]

Yeah, sorry, I should posted this in a help channel.

[trim tangle]

Well actually, representing money with floats is a bad idea, because they are imprecise and don't support decimal arithemtic well

[pale otter]

okay you got me, perhaps Decimal

[vapid quarry]

Math server actually

[trim tangle]

But for a microeconomics model it's probably fine, since it's not exact anyway

[vapid quarry]

Yeah price is float.

[pale otter]

All is but an approximation

Could I ask my question now Tony?

[vapid quarry]

Yes, I'm done.

[pale otter]

When refactoring codes from previous versions of Python (e.g. 3.8.X) to 3.10 that support hype hinting, what's the best practice in terms of replacing stuff from __annotations__ and typing?

[trim tangle]

What do you mean by stuff from __annotations__?

The main changes are:

1. Use stuff from the standard library (like list[int] instead of typing.List[int]). This is from PEP585
2. Use | instead of Union and Optional
3. If you were using some stuff from typing-extensions, you might be able to import it from typing now
   Some of these changes can be automated with https://github.com/asottile/pyupgrade or https://github.com/asottile/reorder_python_imports but they don't handle (1) well. So I have a flake8 plugin for this purpose: https://pypi.org/project/flake8-pep585

[pale otter]

oooo cheers for that, there's a flake8 plugin for everything

[tranquil turtle]

(my english is bad, sorry for that)

proposal:
make types.ModuleType generic
it accepts one string literal as parameter

x: types.ModuleType['os'] - means x is os module, and you can use x as alias of import os; os

it will allow us to use specific modules in function arguments, instance attributes, ...

now if we use ModuleType in typehint, it doesnt contain information about which one module it is
but if we add one generic argument, type-checkers will be able type-check all operations with this module

from types import ModuleType
from typing import TypeVar

T = TypeVar('T', bound=ModuleType)

def f(x: T) -> T: return x

import os
os2 = f(os)

reveal_type(os) # types.ModuleType
reveal_type(os2) # types.ModuleType
reveal_type(os.system) # <some signature>
reveal_type(os2.system) # Any

[trim tangle]

What's the use case?

I guess conditionally importing one module or the other 🤔

How should this handle relative modules? ModuleType['.foo'] in one module might be different from ModuleType['.foo'] from another

[soft matrix]

is there a way to fix this?

do i need to use cast?

[trim tangle]

Sounds like a bug in %software_name%

[soft matrix]

cast seems to work but this seems like an unfortunate case for bidirectional inference

[trim tangle]

Why doesn't pyright respect variable annotations by the way?

it's annoying! the annotation is there for a reason

[soft matrix]

¯_(ツ)_/¯

ive ran into this a lot

[tranquil turtle]

pyright is smarter than you (no)

[trim tangle]

I think it's a popular pattern in Java and C#, when you have a concrete implementation Foo of IBar, to do ```java
IBar myFoo = new Foo(...)

[soft matrix]

i feel like this should work```py
TYPE_TRANSFORM_MAP: Final[Mapping[str, str]] = {
"Dlc": "DLC",
}

but pyright forces

TYPE_TRANSFORM_MAP: Final = cast(Mapping[str, str], {
"Dlc": "DLC",
})

cause otherwise TYPE_TRANSFORM_MAP is a dict[str, str]

[trim tangle]

Well, this is unsafe

[soft matrix]

am i not upcasting?

[trim tangle]

Nope, cast just changes the type and trusts you

https://pyright-playground.decorator-factory.su/?gzip=H4sIAGk_mWIC_0srys9VKKksyMxLV8jMLcgvKlFITiwu4aqwUsjMK1GwVTAx4iovSQMyQMIaxSVFOgoVmlxFqWWpiTnxQI2pGkBpTS4AOmUfQUgAAAA%3D

[soft matrix]

yeah but isnt casting a dict to a mapping safe?

[trim tangle]

Ah, yes, that part is safe

I meant that if you accidentally do Mapping[int, list[tuple[Lock, ...]]] the type checker will not help you

(easy blunder to make, I know /s)

[blazing nest]

It's always ironic when I see a cast in a traceback because of some bug

[rustic gull]

why in type hints you call an object with [] and not ()?

[oblique urchin]

a new operator was needed because () on types already means instantiation

[rustic gull]

so that operator means?

a reference of a class without making an instance?

[oblique urchin]

[] in type hints is parameterization. list[int] is a list that contains ints

[rustic gull]

yeah i know, so its like i said like sorta of a reference?

[buoyant swift]

sorta? list[int] does give you something that's like list back, but it doesn't really make sense to think about it in terms of runtime behavior

[rustic gull]

yeah i think its like a reference because as jelle said that () makes an instance so i understood it like it makes a reference of that type without creating and instance of it

is their like a pep for it cuz i would very much would like to read it

[buoyant swift]

https://peps.python.org/pep-0585/

[rustic gull]

thank you!

[somber mortar]

I'd like to type a tuple of 2 items, str and one of these specific strings, I guess I could use a namedtuple, but are there other ways to specify that it must be a tuple of 2 items and then type them?

ah Tuple[str, Union[specific, strings]] seems like it

[brisk hedge]

or in 3.10+, tuple[str, first_thing | second_thing | third_thing]

[somber mortar]

a bit too early for me to depend on 3.10 but great to know it gets nicer

[hearty shell]

Are you talking about literal string s?

[somber mortar]

yes

[hearty shell]

Ah, then you can use Literal directly

It takes the union of them

[somber mortar]

ah right

TIndexDirection = Literal[pymongo.ASCENDING, pymongo.DESCENDING, pymongo.GEO2D, pymongo.GEOHAYSTACK, pymongo.GEOSPHERE, pymongo.HASHED, pymongo.TEXT]
TIndexKey = Tuple[str, TIndexDirection]

basically is what I ended up with for now

just needed some typing around pymongo to make my own utility functions nicer

[hearty shell]

Humm that shouldn't* work I think though

[somber mortar]

no?

'

ok, Union it is

[hearty shell]

What are pymongo.ASCENDING?

are they stirngs or enums

[somber mortar]

ah, int

I thought they were str

[brisk hedge]

Are they constants or enum members?

[somber mortar]

IDE having a bad time indexing the dependencies so didn't bother checking on python repl

well, due to the IDE problems I don't have a quick way to jump to the definition to see

[hearty shell]

The problem is that you cant use variables as literals, the only "exception" are enums

[somber mortar]

well the Union should work well enough

[brisk hedge]

It unfortunately won't, since they are not types

[somber mortar]

oh right

lemme see if I can find the source for the file

[brisk hedge]

Oh, these aren't even all ints

[somber mortar]

https://github.com/mongodb/mongo-python-driver/blob/0fe4ba7f0ba6389d9083b56bc43d57d4ee095f32/pymongo/__init__.py#L17-L68

lul

yeah

ints, or strings

oh

I was just looking into this because it seemed pymongo<4 didn't have the types in it, and it looked like motor didn't support pymongo>=4, but it seems it does now

this project is early enough so I can just update motor to 3 + pymongo to 4 and it had some types for these things already

looks much more reasonable

[cunning raven]

uhh one question, how do i type hint this

from enum import Enum

class EnumTest(Enum):

  a = 1
  b = 2
  c = 3

def test(num: int) -> ??:
  return EnumTest(num)```

Still wondering if it's

-> EnumTest or typing.Type[EnumTest]

i think the second

[buoyant swift]

EnumTest, right? if you were just returning EnumTest and not EnumTest(num), then you would use typing.Type[EnumTest]

[cunning raven]

oh no, the first

[buoyant swift]

typing.Type is the class itself

[cunning raven]

i think it's the opposite

[buoyant swift]

it's not the opposite

[cunning raven]

Oh wait no no no

sorry sorry sorry

yeh you're right sometimes i mistake them, sorry, and thanks!

[lapis adder]

prolly a google-able question, but is there any typed difference to using typing.List vs list for annotation besides consistency?

[hearty shell]

[lapis adder]

appreciate it

[hearty shell]

As an aside, there is also the fact that this erros

!e

from typing import List

List[3]

[rough sluice]

@hearty shell :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 3, in <module>
003 |   File "/usr/local/lib/python3.10/typing.py", line 312, in inner
004 |     return func(*args, **kwds)
005 |   File "/usr/local/lib/python3.10/typing.py", line 1142, in __getitem__
006 |     params = tuple(_type_check(p, msg) for p in params)
007 |   File "/usr/local/lib/python3.10/typing.py", line 1142, in <genexpr>
008 |     params = tuple(_type_check(p, msg) for p in params)
009 |   File "/usr/local/lib/python3.10/typing.py", line 176, in _type_check
010 |     raise TypeError(f"{msg} Got {arg!r:.100}.")
011 | TypeError: Parameters to generic types must be types. Got 3.

[hearty shell]

It is not really relevant for "normal" use though

[oblique urchin]

not in 3.11

[drifting cloud]

how can i exclude the __pypackages__ directory and everything in it from mypy. The directory is located in the same folder as the pyproject.toml file. Here is my current mypy config which isnt working

[tool.mypy]
mypy_path = "__pypackages__/3.10/lib/"
exclude = ['__pypackages__']

[covert dagger]

exclude is a regex not folder name, according to the docs, https://mypy.readthedocs.io/en/stable/config_file.html#confval-exclude
so it is matching __pypackages__ exactly, not any files inside that folder
maybe exclude = "__pypackages__/.*" will work

[red atlas]

is there a way to annotate "iterable of strings except str itself"?

[oblique urchin]

no

though I believe pytype has a special case so it doesn't accept str for either Sequence or Iterable or both, forgot the details

[red atlas]

thanks ill check that out

unfortunately pytype doesnt seem to have much ide integration but it is what it is

[blazing nest]

Now that my thread has finally been accepted. If anyone wants to discuss here just @me:
https://mail.python.org/archives/list/[e-mail]/thread/FZW5CASB4DSMVH45USZBCQB6NR4O6UNO/

[drifting cloud]

sadly that doesnt work, ig ill try to configure pyright instead then

[soft matrix]

TypedMapping is something I'd really like

It would also make TypedDict a lot less special cased

[soft matrix]

I also don't think there's much point having the new method return a mapping proxy as if you are using typed dicts you are using typed python which should already warn you about writing to the dict

[blazing nest]

True, I wouldn't want or use this behavior but it somewhat follows the precedent of the type checks currently done

[soft matrix]

Oh and probably a MutableTypedMapping would be good if you went with this idea

[covert dagger]

how would a MutableTypedMapping be different from a TypedDict?

[cunning raven]

Sorry if this isn't the proper channel for this question, but i don't know what else i can do.

[soft matrix]

TypedDict actually has dict in its bases TypedMutableMapping wouldn't

[cunning raven]

sorry what?

[soft matrix]

That's not in response to you

You need to use future.annotations or wrap your annotations in strings

[cunning raven]

it runs fine

but when print make_application_image says undefined

[blazing nest]

It wouldn't have the .get() methods, etc.

[soft matrix]

!e
import collections.abc
print(set(dict.dict.items()) - set(collections.abc.MutableMapping.dict.items()))

[rough sluice]

@soft matrix :white_check_mark: Your eval job has completed with return code 0.

{('popitem', <method 'popitem' of 'dict' objects>), ('fromkeys', <method 'fromkeys' of 'dict' objects>), ('setdefault', <method 'setdefault' of 'dict' objects>), ('__repr__', <slot wrapper '__repr__' of 'dict' objects>), ('__hash__', None), ('__doc__', "dict() -> new empty dictionary\ndict(mapping) -> new dictionary initialized from a mapping object's\n    (key, value) pairs\ndict(iterable) -> new dictionary initialized as if via:\n    d = {}\n    for k, v in iterable:\n        d[k] = v\ndict(**kwargs) -> new dictionary initialized with the name=value pairs\n    in the keyword argument list.  For example:  dict(one=1, two=2)"), ('__ior__', <slot wrapper '__ior__' of 'dict' objects>), ('__delitem__', <slot wrapper '__delitem__' of 'dict' objects>), ('copy', <method 'copy' of 'dict' objects>), ('__or__', <slot wrapper '__or__' of 'dict' objects>), ('__eq__', <slot wrapper '__eq__' of 'dict' objects>), ('__setitem__', <slot wrapper '__setitem__' of 'dict' objects>), ('pop', <method 'pop' 
... (truncated - too long)

Full output: https://paste.pythondiscord.com/tixibumijo.txt?noredirect

[covert dagger]

seems pretty niche, I don't see many people using it, if I am going through the trouble of making a custom class, I'll just make the "keys" be attributes of the object to support heterogeneous types

but on the other hand no harm in adding it, if TypedMapping is being added as well, just for symmetry with collections.abc

[soft matrix]

I'd like to use it to support typing my MultiDicts

In a way that would work with mypy

[blazing nest]

As in multidict from PyPI with case incensitive keys?

[soft matrix]

Yes

https://github.com/Gobot1234/steam-ext-tf2/blob/main/steam/ext/tf2/schema.pyi is what I have currently but it's not supported without type ignores

[blazing nest]

Hmm, you wouldn't be able to tell the type checker about the case insensitiveness but I assume you will always use the same casing locally but the user (I am imagining HTTP Headers) could do it differently?

[soft matrix]

I don't care about case insensitive

It's the duplicate key support I'm after

Because valve's propriety key value mapping vdf supports duplicate keys for some reason

[blazing nest]

Ah yeah, but no typing for the getall() methods by just adding a way to make a mapping readonly - unless you can tell the type checker to grab a key with that value (value being a type var). I've been thinking of that, because I saw how it was like possible in TypeScript but haven't had much of a use yet

[terse sky]

I kind of feel like putting a lot of time into things TypedDict is a bit pointless because the code in question can just be upgraded to use a dataclass or something similar

Unless there's actually some kind of reason to use it beyond slapping types on existing code that uses dicts, which is less effort than refactoring to use dataclasses

If there is I'd be really curious to see what it is

[soft matrix]

But I'd like it to be typed all the way down

I also have similar stuff for where I actually use it like a TypedDict to create an object

[trim tangle]

Well, if it's a response of some API, you can't really prove that it's what it returns. (and maybe tomorrow it will be something different, or maybe you misunderstood the docs, or the docs are outdated) So you have to do runtime checks anyway

[soft matrix]

I have faith 😎

[terse sky]

I want it typed all the way down too. So I turn it into a dataclass the moment I encounter it.

Aside from some performance overhead that's not going to matter I can't see why I'd want to leave it as a dict

[lapis adder]

@runtime_checkable
class Middleware(Protocol):
    @classmethod
    def from_file(cls: type['Middleware'], path: PathType)                     \
            -> type['Middleware']:
        ...

    def to_file(self, path: PathType) -> None:
        ...
    
    def __getitem__(self, key: str) -> Any:
        ...
    

@runtime_checkable
class SupportsSchema(Protocol):
    def verify_schema(self, trunc_numerics: bool=...) -> bool:
        ...

is this valid usage of typing.Protocol or is it overspecifying?

[soft matrix]

But everything is Any?

[clear heath]

Is this the place to ask for help with pyright in my editor? It refuses to honor my pythonPath settings
Sublime Text 4, LSP-Pyright, pyright v1.1.252

[feral wharf]

hey, got a kinda weird thing with pylance

ContextMenuCogCallback = Union[
    Callable[[Cog, "Interaction", Member], Coro[Any]],
    Callable[[Cog, "Interaction", User], Coro[Any]],
    Callable[[Cog, "Interaction", Message], Coro[Any]],
    Callable[[Cog, "Interaction", Union[Member, User]], Coro[Any]],
]

# does not complaing when doing this:
# ContextMenuCogCallback = Callable[[Cog, "Interaction", Union[Member, User, Message, Union[User, Member]]], Coro[Any]]
# over the current
def context_menu_decorator_wrapper(..., func: ContextMenuCogCallback) -> ...:
    def handle_calling(..., argument: Union[Member, User, Message, Union[User, Member]]) -> ...:
        ...
        return func(..., argument) # <- this one

on argument :

(parameter) argument: Member | User | Message
Argument of type "Member | User | Message" cannot be assigned to parameter of type "Member"
  Type "Member | User | Message" cannot be assigned to type "Member"
    "User" is incompatible with "Member" [PylancereportGeneralTypeIssues]

Argument of type "Member | User | Message" cannot be assigned to parameter of type "User"
  Type "Member | User | Message" cannot be assigned to type "User"
    "Member" is incompatible with "User" [PylancereportGeneralTypeIssues]

Argument of type "Member | User | Message" cannot be assigned to parameter of type "Message"
  Type "Member | User | Message" cannot be assigned to type "Message"
    "Member" is incompatible with "Message" [PylancereportGeneralTypeIssues]

Argument of type "Member | User | Message" cannot be assigned to parameter of type "Member | User"
  Type "Member | User | Message" cannot be assigned to type "Member | User"
    Type "Message" cannot be assigned to type "Member | User"
      "Message" is incompatible with "Member"
      "Message" is incompatible with "User" [PylancereportGeneralTypeIssues]

but it's an Union?

[blazing nest]

Yeah but Pylance is more-or-less correct here. If you want it to work you need to change it to Callable[[Cog, "Interaction", Union[Member, User, Message]], Coro[Any]]

[feral wharf]

Ah

[blazing nest]

Technically, Pylance could merge this for you but what if the other arguments weren't the same? Think of it as ContextMenuCogCallback being an "overloaded function" and if only some parameters being possible when argument was a specific type then merging like this for you wouldn't work

[hearty shell]

You could use a typevar for this

[oblique urchin]

I don't think it could, the error is unavoidably correct

[hearty shell]

class A: ...
class B: ...
class C: ...

T = TypeVar('T', A, B, C, Union[B, C])

CallableFoo = Callable[[T], Any]

def deco(func: CallableFoo[T]) -> ...:
    def handle_calling(argument: T) -> ...:
        return func(argument)

It will match your argument, but this can get finiqui if you introduce keyword argumetns and such

T = TypeVar("T", User, Member, Message, Union[Member, User])

ContextMenuCogCallback = Callable[[Cog, "Interaction", T], Coro[Any]]

def context_menu_decorator_wrapper(..., func: ContextMenuCogCallback[T]) -> ...:
    def handle_calling(..., argument: T) -> ...:
        return func(..., argument)

[trim tangle]

Union means "any of the types", so you need to handle each of the possibilities. For example, if you have an x: Union[int, str], you can't do x + "foo" and you can't do x + 42. What you can do is:

1. do an isinstance check to narrow the type:

if isinstance(x, int):
    print("answer:", x + 42)
else:
    print("answer: " + x)

2. do an operation that's supported on both types:

y = x * 2

[grave fjord]

hash is probably a bad example there...

[hearty shell]

x: Union[str, int]
y = x * 2

[trim tangle]

good point

[hybrid bridge]

l

[rustic gull]

https://pyright-playground.decorator-factory.su/?gzip=H4sIAD63nmIC_2WPsQqDMBRF93zF21QIfoDg0KVdWugQukgRa6MN2BiSOPj31bzGxNZBvNd7wkmnxzfYWQnZg3irUVs4yJnCWRhL4cQl16KlwGbFb42m0HNbL3Nev4S0hpDr0EjLoPSLNMEmofAYJ_ksMScZOepJ7JfYhKXLy5KQdmiMAUemX4cK1_esILA83ZpM4Tz9rw10eQNRyINqTQWezVyDr0sj-5FpzneKoQ2aW7eo4vc_8bOOLrXRqVOoMHu7PM8jp4hI3Z2q4BMTH4YXvALFAQAA
Any way I can get this to work?

[acoustic thicket]

i think Fruit shouldnt be generic that way

https://pyright-playground.decorator-factory.su/?gzip=H4sIAGnknmIC_1WQwWrDMBBE7_sVe7MMwh9gyKGX9NJAIKaXEIyarFyBvTKSUuK_T6yNE1cHwezMjh6ywQ-YptFxh24YfUj4wZPGLxeTxk9iCu6ssZlG-jZBY0epfcSp_XWcItj3eku3RByd57g0Hai3ANtwdanBzVKiCpkUGn_8lS8b0UUJAOfexIj73nBSz8ePkj6VNeDj2FnFOgMu1msxa8mNc0ctVccZ5JTHsDPc-X80MnnTZL2iET8QKSkTvfBUVSXF-VptqAyzSsGFLFo11OI_nUB_ZPr8p2qoMnV5B0SNhpiVAQAA

[grave fjord]

@trim tangle I can't zoom or scroll horizontally the pyright playground

So I can't see what the bounds are

[trim tangle]

yeah that sucks

it doesn't work well on mobile

PRs are welcome 👀

[finite horizon]

Considering the following code:

from typing import TypeVar, Generic
import attr

BaseFactoryType = TypeVar("BaseFactoryType", covariant=True)
FactoryType = TypeVar("FactoryType", covariant=True)
SubprocessType = TypeVar("SubprocessType", covariant=True)
ScriptSubprocessType = TypeVar("ScriptSubprocessType", covariant=True)
DaemonType = TypeVar("DaemonType", covariant=True)


@attr.s(kw_only=True)
class BaseFactory(Generic[BaseFactoryType]): ...

@attr.s(kw_only=True)
class Factory(BaseFactory[FactoryType]): ...

@attr.s(kw_only=True)
class Subprocess(Factory[SubprocessType]): ...

@attr.s(kw_only=True)
class SubprocessImpl:

    factory: Subprocess[SubprocessType] = attr.ib()

Why am I getting these errors and how do I solve them?

❯ mypy test.py
test.py:35: error: Type variable "test.SubprocessType" is unbound  [valid-type]
test.py:35: note: (Hint: Use "Generic[SubprocessType]" or "Protocol[SubprocessType]" base class to bind "SubprocessType" inside a class)
test.py:35: note: (Hint: Use "SubprocessType" in function signature to bind "SubprocessType" inside a function)
Found 1 error in 1 file (checked 1 source file

Also, is this the right way to define subclasses with their (sub)types?

[soft matrix]

The errors tell you how to fix the issue

[finite horizon]

@soft matrix I can't use factory: Generic[SubprocessType]

test.py:23: error: Variable "typing.Generic" is not valid as a type  [valid-type]
test.py:23: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
test.py:23: error: Type variable "test.SubprocessType" is unbound  [valid-type]
test.py:23: note: (Hint: Use "Generic[SubprocessType]" or "Protocol[SubprocessType]" base class to bind "SubprocessType" inside a class)
test.py:23: note: (Hint: Use "SubprocessType" in function signature to bind "SubprocessType" inside a function)
Found 2 errors in 1 file (checked 1 source file)

[trim tangle]

Well, SubprocessImpl is not generic

Did you mean factory: Subprocess[Any]?

Or to make SubprocessImpl generic in SubprocessType

[trim tangle]

For example, this is invalid because T has no meaning inside the class:

T = TypeVar("T")

class Foo:
    bar: T

Or here:

baz: list[T] = []

[finite horizon]

so, SubprocessImpl should only handle Subprocess or a subclass of it

at least, that's what I'm trying to achieve

[trim tangle]

So you want the factory to be Subprocess[Any] or Subprocess[object]?

If you want to specify a factory of what the SubprocessImpl is holding, you'll need to make SubprocessImpl generic as well

[finite horizon]

either Subprocess or a concrete implementation/subclass of Subprocess

[trim tangle]

A already means "A or subclass of A"

[finite horizon]

ok, so, I could have factory: Subprocess?

but then,

test.py:23: error: Missing type parameters for generic type "Subprocess"  [type-arg]
Found 1 error in 1 file (checked 1 source file)

[trim tangle]

Why is Subprocess generic?

[finite horizon]

I'd like to be able to reference it's type

[trim tangle]

Hm?

[finite horizon]

k, I should explain

because I could be totally wrong here

class Base: ...

class SubBase(Base): ...

I'd like to differentiate the types of both of these

[trim tangle]

What do you mean by differentiate?

[upper horizon]

hi

im in pesian

and you?

[finite horizon]

so that, in a function, I could say, def foo(one: Base, two: SubBase)

[trim tangle]

You can already say that

[finite horizon]

class Base: ...
class SubBase1(Base): ...
class SubBase2(Base): ...
class SubBase3(Base): ...

def foo(sub: Base): ...

will mypy accept Base, SubBase1, SubBase2, etc as a valid sub?

[trim tangle]

Yep

[finite horizon]

Won't I have to declare def foo(sub: Union[SubBase1, SubBase2]) ...

[upper horizon]

i like discord bot whit python
Can you tell me about the Discord robot with Python? I want to join Bot in voice Join

[finite horizon]

or better, the function should accept a subclass, but not the parent class

[trim tangle]

That's the point of subclassing: you can use a subclass wherever you need a parent class. That's called "polymorphism" (or "subtype polymorphism")

[finite horizon]

yeah, but with typing involved(I'm pretty new to it), it seems to complicate these definitions

[dapper yacht]

Running mypy I get the following error

Argument 1 to "get" of "dict" has incompatible type "Type[Resource]"; expected "Type[ResourceType]" 

ResourceType is defined like so:

ResourceType = TypeVar("ResourceType", contravariant=True, bound=Resource)

Problematic method:

    def _get_batch_size(item_type: Type[ResourceType]) -> Optional[int]:
        res_map: Dict[Type[ResourceType], Optional[int]] = {Resource: 1}
        item_type_batch_size = batch_sizes.get(Resource, None)   # <--
        return item_type_batch_size

What am I missing here?

[upper horizon]

pls

[stray summit]

anyone aware of a way to concatenate a appended optional keyword argument to a parameterset?

[dapper yacht]

Sorry for asking that question here, Help was more appropriate I guess!
Changing

res_map: Dict[Type[Resource], Optional[int]] = {Resource: 1}

to

res_map: Dict[Type[ResourceType], Optional[int]] = {Resource: 1}

Fixed it. Can someone explain why?

[stray summit]

Type is for stuff like

def make(klass: Type[T]) -> T:
   return ...

when you want a dict of subclasses, then you likely want a typevar instead

[dapper yacht]

Here, ResourceType is a TypeVar. Or am I maybe misunderstanding you?

[surreal quiver]

Hello, how do I validate a TINYTEXT or TINYINT in Pydantic?

[stray summit]

without seeing the definition, i have no idea

[dapper yacht]

@stray summit See here for the def

[stray summit]

oh, its above, so yes it is a typevar

also you have a bug in your code, you likely wanted batch_sizes.get(item_type, None)

[dapper yacht]

Yep, I cut out portions of it to focus on the relevant part

I'm just curious why the typehint for the dict cannot handle 'Type[TypeVar]' here?

But it is still required to have a return value or parameter use a TypeVar

Where's the logic in this OR why is this seemingly (to me) ugly workaround necessary?

[stray summit]

@dapper yacht i beleie its becasue you want a covariant, but declared a contravariant

@dapper yacht bascially a inexact simplification is you want subclasses in the collection, then the relationship is covariant and if you want to allow superclasses in the collection its contravariant, and the definitions of those terms is tricky to get without understanding the underpinning math ideas

[dapper yacht]

I see

But should covariant or contravariant not matter in this case? As I am trying to typehint the base class?

[stray summit]

you see, that basicially means you want subtypes in the collection -> covariant

[dapper yacht]

But if the subclasses override certain methods it should be contravariant, no?

(which is the case for my subclasses)

As they are not interchangeable

[stray summit]

for the case of your paricular mapping that maps types to integer, the usage you demonstrate is covariant

for other use-cases your usage may very well be contra-variant, but the specific listing you have there looks very co-variant

[dapper yacht]

You are talking about the dictionary now?

I'm confused, how can a mapping be co-variant? I thought that was a relationship between classes (and their sub/superclasses)

[stray summit]

the mapping is not covariant

but the key has to be

so you can actually look them up in a compatible manner

if it was instances instead of types it would be more clear

[dapper yacht]

class Resource:
  def go():
    return 1
class SpecificResourceContra(Resource):
  def go():
    return 2
class SpecificResourceCo(Resource):
  def go_other():
    return 2

ResourceType_contra = TypeVar("ResourceType_contra", contravariant=True, bound=Resource)
ResourceType_co = TypeVar("ResourceType_co", contravariant=True, bound=Resource)

So List[Type[ResourceType_contra]] would be allowed to contain "Resource" and "SpecificResourceContra"
While List[Type[ResourceType_co]] would be allowed to contain "Resource" and "SpecificResourceCo"
While List[Type[Resource]] can contain all 3?

And this is why mypy is complaining about Type[Resources] not fitting into Type[ResourceType_contra], as its not a subset?

(for reference ```py
Argument 1 to "get" of "dict" has incompatible type "Type[Resource]"; expected "Type[ResourceType]"

If this is the case I dont understand why mypy does not manage to infer, that in this case the class Resource, while of Type Resource, is also Type of ResourceType_Contra

[stray summit]

@dapper yacht my very rough understanding is, that the bound typevar allows to be bound to any subclass, and then wen you resolve it, the type Resource cannot match if a subclass thats contravariant is used as for example if item_type was SpecificResourceCo, then Ressource would bee disallowed

[dapper yacht]

I see.. I think I am starting to get a grasp of it but there's definitely some extra research to be done

Thanks for your time!

[stray summit]

@dapper yacht i strongly recommend that, once you have a final verdict, please share, since the topic is indeed tricky

i foudn a answer to my querstion - mypy_extensions has DefaultNamedArg

[stray summit]

the final solution was a generic callable protocol that adds the parameters as defined, its still a bit clunky but definitively better than DefaultNamedArg

[blazing nest]

Ah, as in __call__(self, *args: P.args, added: bool = True, **kwargs: P.kwargs)?

[tranquil turtle]

[offtop]
why you name class-variable klass? isnt cls or class_ better?
why exactly k instead of c?

[brisk hedge]

cls can imply "current class", besides that just taste

[surreal quiver]

Nobody knows this ?

[brazen trellis]

does anyone know if there is a way to ignore missing values with pydantic?

[rare scarab]

klass reeks of java.

[rare scarab]

I believe you can define a Meta class with allow_missing = True

[brazen trellis]

i see.. i've never work w meta classes before, got a link?

[rare scarab]

https://pydantic-docs.helpmanual.io/usage/model_config/

class MyModel(BaseModel):
  x: str

  class Config:
    config = value

[oblique urchin]

interestingly it's a common naming convention in the core (the name occurs 267 times in the CPython repo)

[rare scarab]

Although, if you want to ignore missing values, why use pydantic in the first place? (fastapi?)

There's some options for getting around it.

First, you could declare a default value, like None.

class MyModel(BaseModel):
  x: str
  y: str = None

You could also use a subclass. ```py
class MyModel(BaseModel):
x: str

class MyModelY(MyModel):
y: str

MyModelTypes: TypeAlias = MyModel | MyModelY

def foo(model: MyModelTypes):
if isinstance(model, MyModelY):
do_something_with_model_y(model)

If you're using TypedDict, you can annotate the field with NotRequired

[brazen trellis]

my use case has changed from wanting to verify that all params are specified to setting default params downstream of typechecking. yeah, i should look for another option instead pydantic now

[brazen trellis]

but for now will just do this, thanks

[stray summit]

i use cls in classmethods, and i prefer klass over class_ unfortunately using type shadows the global

[blazing nest]

Depending on situation I may use initializer or subclass

[indigo locust]

If I want to type hint an iterable of types

Like, for example, a list of bases or subclasses

Would it be Iterable[type] or Iterable[typing.Type]

I would lean towards the former I think, since there's no parameterization happening. But that's just a swing in the dark

[summer berry]

Type[Any] is equivalent to Type which in turn is equivalent to type, which is the root of Python’s metaclass hierarchy.
https://docs.python.org/3/library/typing.html#typing.Type

So they are equivalent.

Given that typing.Type is deprecated since 3.9, I'd say go with type.

[acoustic thicket]

parameterizing it seems a bit tricky

actually

i'm not sure how the parameterization would work

even if you passed in unrelated classes they would all ultimately be subclasses of the same class (object)

[summer berry]

Parameterisation of the Iterable or of type?

[acoustic thicket]

of type

[summer berry]

Not sure what you mean.

x: Iterable[type[A]] = [A] is fine and x: Iterable[type[A]] = [object] fails.

Because the typevar is covariant I believe

[acoustic thicket]

oh wait
i was thinking of parameterizing with a typevar

yeah makes sense for concrete types

[deft merlin]

👀 how do u type hint a type hint

like if a did t = typing.Literal[...] what should i type hint t?

[acoustic thicket]

typing.TypeAlias but its not really necessary

[deft merlin]

ah ok, thanks 👍

and if i do something like some_type = typing.something[something], its really a constant so is it better to use scream snake or just normal snake ??

[acoustic thicket]

name it like you name classes

SomeType

[deft merlin]

ok, thanks :)

[stray tide]

Why can't I?

[wooden solstice]

from typing import Dict, cast
from typing_extensions import TypedDict

class D(TypedDict):
    name: str

d: D = {"name": ""}  # Line 7
d2: D = dict({"name": ""})  # Line 8

Using mypy, Line 8 gets reported while Line 7 doesn't.

Related post: https://github.com/python/mypy/issues/12081

I think I got that "The issue is that type checkers don't know what constructors do"
But isn't it just the same when using keyword arguments?

Why it IS working on Line 7?

[covert dagger]

on line 7 it is literal dict syntax and mypy matches it with the typeddict specification you gave as the annotation, on line 8 you made the dict but passed it to the dict function, whose return type is decided by the typeshed stubs, and it is just a simple dict[T1, T2]

[wooden solstice]

Thank you very much for answering.
Actually, what I tried to mean was

d3: D = dict(name="")  # Line 9

I just found that I put a wrong line

This line also calls dict() but with keyword argument and it passes type check and I wonder how/why

Is it mypy that has a sort of dedicated logic for dict(**kwargs), but not for other overloads such as dict(another_dict, **kwargs)?

[desert delta]

mypy is unhappy with this annotation, can anyone explain why?

T = TypeVar('T')

@overload
def ensure_tuple(v: Tuple[T, ...]) -> Tuple[T, ...]: ...

@overload
def ensure_tuple(v: T) -> Tuple[T]: ...

def ensure_tuple(v: T) -> Union[Tuple[T], Tuple[T, ...]]:
    if not isinstance(v, tuple):
        out = (v,)
    return out

Overloaded function signatures 1 and 2 overlap with incompatible return types

(and apologies if this is the wrong channel for this question .... I'm new 🙂 )

[trim tangle]

@desert delta generally this function is not well-typeable. Because to trigger the second overload, the type checker needs to prove that the argument is not a tuple, which is not always possible:

def f(t: T) -> None:
    x = ensure_tuple(t)

We don't know what type x has, because T could be a tuple.

I think there was an issue about this in Pyright's repo but I can't quite find it

Why do you want such a function though?

[desert delta]

I'm adding typing to an existing codebase (zarr-python)

it would be nice if mypy could express "everything except a tuple"

then I could split the overload definitions along that axis

[covert dagger]

yes mypy special cases some patterns

[wooden solstice]

Yep. Thank you so much!

[hasty hull]

Is it possible to have pyright respect an if MYPY block? e.g.

MYPY = False
if MYPY:
    Data = Union[PrimitiveData, "Mapping[str, Any]"]
else:
    Data = Union[PrimitiveData, "Mapping[str, Data]"]

Pyright currently reports this on the last line ```
error: Illegal type annotation: variable not allowed unless it is a type alias (reportGeneralTypeIssues

[soft matrix]

Probably just assign it as a TypeAlias

[hasty hull]

No luck unfortunately :/

if MYPY:
    Data = Union[str, "Mapping[str, Any]"]
else:
    Data: TypeAlias = Union[str, "Mapping[str, Data]"]

error: Illegal type annotation: variable not allowed unless it is a type alias
error: "Data" is declared as a TypeAlias and can be assigned only once

Same error if you declare both as TypeAlias

[brisk heart]

MYPY = False just makes it a bool and MYPY: Literal[False] = False is incorrect to mypy 😭

I have up on mypy because of stuff like this that makes it not only an absolute pain when it matters the most but also impossible to deal with

[rare scarab]

Try MYPY: Final = False

Though you could just use typing.TYPE_CHECKING

[hasty hull]

Yeah I feel that pain, I wish I could give up on mypy but this is library code so it needs to be compatible with mypy :/

typing.TYPE_CHECKING doesn't work because Pyright also sees that and I don't want to just use

Data = Union[str, "Mapping[str, Any]"]

When I could use:

Data = Union[str, "Mapping[str, Data]"]

[soft matrix]

What's stopping you?

[brisk heart]

damn lib users

[hasty hull]

Good point, I figured mypy would report an error if a user calls the function that the Data type is used for as it''s a recursive type but I haven't actually tested it

[rare scarab]

does mypy really not support recursive type aliases?

[wicked scarab]

Can we somehow say that for example: def foo(x: str | int, name): if x is a string, then the name parameter exists, otherwise, it doesn't (decided by the type-checker)

[tranquil turtle]

from typing import overload

@overload
def foo(x: int) -> None: ...

@overload
def foo(x: str, name: str) -> None: ...

def foo(x: int | str, name: str = None) -> None:
    pass # implementation

[wicked scarab]

Oh, that works? I thought overloads can only control the types

[tranquil turtle]

!d typing.overload

[rough sluice]

typing.overload

@typing.overload```
The `@overload` decorator allows describing functions and methods that support multiple different combinations of argument types. A series of `@overload`-decorated definitions must be followed by exactly one non-`@overload`-decorated definition (for the same function/method). The `@overload`-decorated definitions are for the benefit of the type checker only, since they will be overwritten by the non-`@overload`-decorated definition, while the latter is used at runtime but should be ignored by a type checker. At runtime, calling a `@overload`-decorated function directly will raise [`NotImplementedError`](https://docs.python.org/3/library/exceptions.html#NotImplementedError "NotImplementedError"). An example of overload that gives a more precise type than can be expressed using a union or a type variable:

[wicked scarab]

The @overload decorator allows describing functions and methods that support multiple different combinations of argument types.
The docs doesn't say that it can control the existence of arguments

[tranquil turtle]

idk, but it works for mypy```py

from typing import overload

@overload
def foo(x: int) -> None: ...

@overload
def foo(x: str, name: str) -> None: ...

def foo(x: int | str, name: str = None) -> None:
pass # implementation

reveal_type(foo)

Overload(def (x: int), def (x: str, name: str))

foo(0) # ok
foo('') # error
foo(0, '') # error
foo('', '') # ok

[soft matrix]

Works for both major type checkers

[wicked scarab]

Thanks

[uneven ginkgo]

I am type hinting a Generator comprehension.
what three arguments are needed for the Generator[] typehint??

Currently it is

var: Generator[ClassName, None, None] = (bla for bla in blabla)

what can I place instead of None?

[hearty shell]

None is correct, the way generators are typed is Generator[yield type, send type, return type].

[uneven ginkgo]

Thanks Mathias!
I'm not 100% sure what send type is. haven't heard it before but I'll check it out for sure 🙂

[hearty shell]

The return type is just what gets returned when the iteration stops, it is wrapped in the "StopIteration" exception, and the send is for the gen.send method. It is more applicable to actual generator functions, the docs have an example

def echo_round() -> Generator[int, float, str]:
    sent = yield 0
    while sent >= 0:
        sent = yield round(sent)
    return 'Done'

[oblique urchin]

right, I don't think generator comprehensions can usefully have a non-None send or return type

[cedar sundial]

How do you get around the *unpack type in mypy? I keep getting an incompatible type [arg-type]. It wants me to put the type as Any even though I know what the type will be