type-hinting

[solid light]

Yeah, that makes sense

Isn't that the same as a typevar though?

[brisk hedge]

T = TypeVar("T")
def triple(value: T) -> list[T]:
  print("value is", value)
  return [value, value, value]

triple(1) # in this function call, T is `int`
triple(()) # in here it's `tuple[()]`

[solid light]

Ah, I see

So TypeVars are dynamic (change value), whereas TypeAliases don't

Yeah, that makes sense

Thanks

[buoyant swift]

a typevar is like specifying a parameter

[trim tangle]

https://decorator-factory.github.io/typing-tips/tips/typevar-typealias-newtype/

[trim tangle]

So... this is hack time. anyone knows if there's a way to use mypy/pyright to produce some kind of "map" telling the type of every expression in a module/function/etc.?

as data, that is. so I can access it later

[soft matrix]

there is for mypy, im currently working on a way to do it with pyright to not much avail

[trim tangle]

ooh

any pointers?

[soft matrix]

https://github.com/Gobot1234/steam.py/blob/main/docs/extensions/annotations.py its not fun

[trim tangle]

pure madman, making a whole metaprogramming system for a steam library

my utter respect

[soft matrix]

the important thing is https://github.com/Gobot1234/steam.py/blob/main/docs/extensions/annotations.py#L146-L147

[rough sluice]

docs/extensions/annotations.py lines 146 to 147

result = build([BuildSource(mod.__file__, mod.__name__)], options, stdout=stdout, stderr=stderr)
types = result.types```

[trim tangle]

why do you need this btw?

[soft matrix]

its for the docs so that everything gets cross linked even with future.annotations but its broken on the latest mypy for reasons that are beyond me

[trim tangle]

ah

[soft matrix]

sphinx kinda sucks at resolving complex things

[terse swallow]

@soft matrix what does fmt mean in your code(comment)?

fmt: off
...
fmt: on

[soft matrix]

its something to turn off black (see #black-formatter)

[terse swallow]

oh I see thanks

[lethal solar]

Can someone explain me why this is wrong ?
https://pyright-playground.decorator-factory.su/?gzip=H4sIAKyZ5mMC_y2NvQqAMAyE9zxFRgURXAVXRyc3kVJqCkJpxEQnH974c8txfNxdSF4ERxJtAU0LRXROSJ0rhFKscM2iPgeq8PTpoBbTKjqJ7ngZ09ls4Ezl13-02SRAeJd75g-oXWD3PhUlQGS2ZPQP9c-nZoYbK768CJQAAAA%3D

class Test:
    def __set__(self, instance, value: list[str | int] | None):
        pass

class Foo:
    test = Test()

foo = Foo()

foo.test = [1]

while this is good

def test(a: list[str | int]):
  pass

test([1])

[reef yacht]

I would guess pyright doesn't understand __set__. I had to look it up myself heh

[trim tangle]

This sounds like a bug in Pyright. Could you file an issue on github? https://github.com/microsoft/pyright/issues

pyright-playground is now on the latest pyright version (1.1.293), the behaviour is the same

[lethal solar]

done 👌

[jade viper]

Hey guys, what is the best way to type hint any kind of array with any kind of number in it in Python 3.8? My type hint currently looks really cluttered: array: Union[List[Union[float, int, Decimal, np.float64, np.int64]], np.ndarray]

[soft matrix]

there isnt really a number type (cursory python/mypy#3186) you probably need to define a protocol that defines the operations you wish to use on your "array"

[trim tangle]

Could you provide some context?

what do you mean by "number" and "array"? 🙂

[jade viper]

Sure! I'm currently creating some helper functions for company-wide usage. The current function I'm creating checks if every number in array is within a margin of each other. For example:

is_within_margin([1, 2, 3], margin=1) -> True
is_within_margin([1, 2, 3], margin=0.5) -> False

And I want it to be as robust as possible, so the array in question can be a regular Python list, a NumPy ndarray and etc. and the contents of these arrays have to be numbers, because they're checked for margins, so int, float, decimal.Decimal, NumPy numbers and etc.

Should I just type hint it as List[Union[int, float]]?

[oblique urchin]

you probably want something like Sequence or even Iterable instead of list

the number type is harder as @soft matrix said

[jade viper]

What's the difference between Sequence and Iterable in this case?

[oblique urchin]

Iterable only allows iterating, Sequence also allows len(), indexing, and a few other operations

[jade viper]

I see, thank you so much everyone!:)

[rustic gull]

Yes, you can type hint it as List[Union[int, float]] to handle different types of numbers within the list.

[trim tangle]

Can you show the implementation?

[trim tangle]

actually int is a "virtual subclass" of float from a type system point of view, so Iterable[int | float] is the same as Iterable[float]

[trim tangle]

You can define a protocol for anything supporting comparison and subtraction: ```py
from collections.abc import Sequence
from typing import TypeVar, Self, Protocol

class Magnitude(Protocol):
def le(self, other: Self) -> bool:
...

def __lt__(self, other: Self) -> bool:
    ...

def __sub__(self, other: Self) -> Self:
    ...

Mag = TypeVar("Mag", bound=Magnitude)

def is_within_margin(items: Sequence[Mag], margin: Mag) -> bool:
return max(items) - min(items) <= margin

though why would you extract such a simple function into a company-wide library

These uber-utility functions are usually more complicated than you really need. This function is probably harder to understand than if you have something like ```py
def is_within_margin(items: Sequence[float], margin: float) -> bool:
return max(items) - min(items) <= margin

also, the generic version of is_within_margin allows for some questionable uses, like... you can definitely use this function with sets

[oblique urchin]

I think their function is a little more complicated than that, it's that the margin between every two adjacent elements is less than x

[trim tangle]

oh, maybe

their description

The current function I'm creating checks if every number in array is within a margin of each other
seems to contradict the example 👀

[oblique urchin]

the example was is_within_margin([1, 2, 3], margin=1) -> True

[trim tangle]

Yeah

well, it's not that much more complicated

def is_within_margin(items: Iterable[Mag], margin: Mag) -> bool:
    return all(b - a <= margin for a, b in itertools.pairwise(items))

or does it need to go both ways? (not necessarily increasing)

[soft matrix]

arent you assuming the input is sorted :)?

[trim tangle]

wdym

[soft matrix]

idk why we are guessing what the implementation looks like anyway

[trim tangle]

yeah, i don't get what the function is supposed to do

[soft matrix]

uhh would is_within_margin([1, 3, 2] 1) return True 🤔

[trim tangle]

no, because 3 - 1 > 1

[soft matrix]

yeah but is that actually how this would work?

[trim tangle]

this?

[soft matrix]

this referring to is_within_margin

[trim tangle]

well, the examples are hinting that they want differences between adjacent elements

[soft matrix]

oh wait yeah ok i see that now

[trim tangle]

if you want to check all items, it's actually easier

[regal summit]

    @overload
    def embed_and_file(self, as_attachments: Literal[True] = ...) -> dict[str, Embed | list[discord.File]]:
        ...

    @overload
    def embed_and_file(self, as_attachments: Literal[False] = ...) -> dict[str, Embed | discord.File]:
        ...

    def embed_and_file(self, as_attachments: bool = False) -> dict[str, Embed | discord.File] | dict[str, Embed | list[discord.File]]:``` is the `...` correct for the default values for `as_attachments` in the overloads?

[soft matrix]

dare i say no default is required there

well for the Literal[True] one

[trim tangle]

    @overload
    def embed_and_file(self, as_attachments: Literal[True]) -> dict[str, Embed | list[discord.File]]:
        ...

    @overload
    def embed_and_file(self, as_attachments: Literal[False] = ...) -> dict[str, Embed | discord.File]:
        ...

    def embed_and_file(self, as_attachments: bool = False) -> dict[str, Embed | discord.File] | dict[str, Embed | list[discord.File]]:

That function signature looks sus. Could you show the implementation maybe?

[regal summit]

    @overload
    def embed_and_file(self, as_attachments: Literal[True] = ...) -> dict[str, Embed | list[discord.File]]:
        ...

    @overload
    def embed_and_file(self, as_attachments: Literal[False] = ...) -> dict[str, Embed | discord.File]:
        ...

    def embed_and_file(self, as_attachments: bool = False) -> dict[str, Embed | discord.File] | dict[str, Embed | list[discord.File]]:
        image = self.game.level.image()
        buffer = BytesIO()
        image.save(buffer, format='png')
        buffer.seek(0)
        embed = self.embed
        embed.set_image(url='attachment://level.png')
        file = discord.File(buffer, filename='level.png')
        if as_attachments:
            return {'embed': embed, 'attachments': [file]}
        else:
            return {'embed': embed, 'file': file}
``` it starts complaining if I dont give them a default value in the overloads

[trim tangle]

what's the complaint?

Also, what is that function supposed to do? How would it be useD?

[regal summit]

I use the default variable here

[regal summit]

well it returns a dict that has an embed and a file, if as_attachments is True, returns attachmetns which is a list of that file instead of just that file

[trim tangle]

How would I use its return value?

[regal summit]

file being a disocrd.File object

await ctx.send(view=view, **view.embed_and_file())

this is used more than once and you need to embed and file together for the file to appear on the embed so a function is the easiest

[trim tangle]

You need to have a default for False, but not or True. Because if you don't provide anything, it should resolve to the False variant.

[regal summit]

but sometimes you want is at file=File and sometimes you need it as attachments=[file]

so everything should have the default ... except for the actual default?

[trim tangle]

wdym?

[trim tangle]

It should be like this

[regal summit]

wait no only the actual default should have the default ...

[trim tangle]

yes

[regal summit]

so if id have like Literal[None] I would not give it a default either

[trim tangle]

wdym

[regal summit]

another overload

[soft matrix]

(Literal[None] == None fyi)

[regal summit]

yeah ok

bad example

Literal['something']

[trim tangle]

Yes. If all of them have a default and you call embed_and_file(), what variant should the type checker pick?

[trim tangle]

My point is: if you have a dynamic return value like this, the user will not benefit from a type like dict[str, Embed | discord.File]. If I had to use this design I'd probably do dict[str, Any] and not bother

[soft matrix]

or use a TypedDict :)

[trim tangle]

not sure if mypy will enjoy that

[regal summit]

eh aight

[soft matrix]

arent they using pyright?

[trim tangle]

oh ye

[regal summit]

are there big differences?

or just some opinion based things

[trim tangle]

the user will not benefit from a type like dict[str, Embed | discord.File]
...because if you do return_value["embed"], it is, well, either an embed or a file, according to the type checker. annoing!

[trim tangle]

yeah, a lot of different decisions

[regal summit]

yeah but my only use case will be in the mapping unpacking

[trim tangle]

in that case, do dict[str, Any]

[regal summit]

its not like anyone else will ever use this lol

aight

[regal summit]

is there a way to specify this btw? Can I subclass a dict and overload that with my keys just to typehint thing

not that im going to

[trim tangle]

You can use TypedDict

!d typing.TypedDict

[rough sluice]

typing.TypedDict

class typing.TypedDict(dict)```
Special construct to add type hints to a dictionary. At runtime it is a plain [`dict`](https://docs.python.org/3/library/stdtypes.html#dict "dict").

`TypedDict` declares a dictionary type that expects all of its instances to have a certain set of keys, where each key is associated with a value of a consistent type. This expectation is not checked at runtime but is only enforced by type checkers. Usage...

[regal summit]

oh ok

[hushed gorge]

I got a function which takes in a Enum and retuns the value of that enum. (more logic in it than that but thats input + output)

How do I type hint that correctly?

[pastel egret]

Do you mean taking in the enum class, or just a single enum member and then accessing .value? I'm not sure how well type checkers set the type of .value.

[hushed gorge]

enum member

[pastel egret]

You might need to make a protocol, perhaps that would work.

[heady flicker]

If you wish to get the literal type of value — you're out of luck. If not — just typehint it as str or int

However, if you're using mypy, there might be an extension that does what you want

[hushed gorge]

its for a library I maintain. its not that big a deal bc this is a helper function which is not really intended for public use anyway but its always better to type hint it all. (but I am not pulling in additional dependencies just for type hints)

[heady flicker]

Then just use my first suggestion :)

[hushed gorge]

got it, thanks. I thought I was missing some fancy Enum[T] -> T shenanigans

[pliant vapor]

how come TypeVar allows you to associate it with data types like

T = TypeVar('T', str, bytes)
T = TypeVar('T', bound=str)

but TypeVarTuple doesn't?
like i would've thought

Ts = TypeVarTuple('Ts', str, int)

would parameterize an arbitrary amount of strings or ints

but that's not allowed

[pastel egret]

Since it's such a complicated feature and they weren't sure if it'd be useful, those were deliberately left out of the PEP: https://peps.python.org/pep-0646/#variance-type-constraints-and-type-bounds-not-yet-supported

It's easy to allow them later, harder to deprecate and remove the feature if it turns out to be a problem.

[pliant vapor]

ah ok

[soft matrix]

idk if TypeVarTuple would ever even get constraints they just kinda are annoying to deal with

[tranquil turtle]

is there a PEP about type-level Map?

i need this: ```py
class X(Generic[tuple[A1, A2, A3, ...]]):
def init(self, a1: Y[A1], a2: Y[A2], a3: Y[A3], ...) -> None: ...

[soft matrix]

theres a draft for it somewhere on typing-sig

but for now no, youre kinda screwed

class X[*As]:
    def __init__(self, *args: Map[Y, As]) -> None: ...
```I think is the proposed syntax for the special form

[tranquil turtle]

Is something like this possible?:```py
TD = TypeVar('TD', bound=TypedDict)
class X(Generic[TD]):
def init(self, **kwargs: **TD) -> None: ...
def get(self) -> TD: ...

x = X(a=1, b='abc')
x.get()['a'] # int
x.get()['b'] # str

[tranquil turtle]

currently im using this code:```py
class X(Generic[tuple[Any, ...]]):
def init(self, *a: Y[Any]) -> None: ...

[soft matrix]

Generic[tuple[Any, ...]] is wrong

[tranquil turtle]

why? there is no restrictions to A's

[soft matrix]

its not got any type vars in it?

[brisk hedge]

that should have a typevartuple

[soft matrix]

ye

[tranquil turtle]

class Sequence(DataClass[tuple[Any, ...]]):
    __slots__ = ('dclss',)

    def __init__(self, *dclss: DataClass[Any]) -> None:
        self.dclss = dclss

    def read(self, buf: IBuffer, /) -> tuple[Any, ...]:
        result: list[Any] = []
        for i, dcls in enumerate(self.dclss):
            item = dcls.read(buf)
            result.append(item)
        return tuple[Any, ...](result)
``` it works for me

[soft matrix]

well thats different

but ok, if it works it works

[tranquil turtle]

i dont need Generic[*As], i need Generic[tuple[*As]]

[soft matrix]

but that doesnt make sense

unless youre referring to an abitiary Generic here and not typing.Generic

[tranquil turtle]

class DataClass(Generic[T]): # these objects can read some data from some stream and return value of type T
    def read(self, ...) -> T: ...

class Sequence(DataClass[tuple[Any, ...]]: # these objects are reading several pieces of data and returning them as tuple
    def __init__(self, *dclss: DataClass[Any]) -> None: ...
    def read(self, ...) -> tuple[Any, ...]: ...

# this is better, but there is no type-level Map now
class Sequence(DataClass[tuple[*As]]):
    def __init__(self, *args: Map[DataClass, As]) -> None: ...
    def read(self, ...) -> tuple[*As]: ...

if you are interested you can find imlementation here: https://github.com/denballakh/ranger-tools/blob/master/rangers/std/dataclass.py#L833
and example of usage here: https://github.com/denballakh/ranger-tools/blob/master/rangers/sav.py

[visual vigil]

Is it valid to use a class as a field specifier when using dataclass_transform? Pyright complains about this code:

from typing import dataclass_transform

class Field:
    ...

@dataclass_transform(field_specifiers=(Field, ))
class ModelBase: ...

class Model(ModelBase):
    foo: int = Field()  # Expression of type "Field" cannot be assigned to declared type "int"; "Field" is incompatible with "int"

When using a function as the field specifier you can just make it return typing.Any, but I can't think of an equivalent for classes.

[soft matrix]

Just subclass Any instead

Or if you want slots use class Field(Any if TYPE_CHECKING else object): ...

[visual vigil]

pyright seems to allow that, although mypy complains that it's an invalid base class. I guess it doesn't matter much since mypy doesn't have proper support for dataclass_transform yet anyway afaik, although I'd be interested to know if there's a "correct" way of doing this

[median tangle]

I want to create a typed dict that has some keys which I know will be present, but can also have any number of other keys, that I don't know the names of ahead of time. I know there's total=false kwarg, but that doesn't seem to be doing what I'd like: ```py
class MyDict(TypedDict, total=False):
x: int

d: MyDict
d["x"] # "x" is not a required key in "MyDict", so access may result in runtime exception
d["y"] # "y" is not a defined key in "MyDict"

what I'd like is instead:

d: MyDict
d["x"]  # Returns type int
d["y"]  # Returns type Unknown, but no error, just like with simple dict type

[oblique urchin]

this isn't supported in the standard type system. I added an extension for it in pyanalyze: https://github.com/quora/pyanalyze/blob/5e03ed89528139ce7067c32910988f8a5ce007f6/pyanalyze/extensions.py#L590

[rough sluice]

pyanalyze/extensions.py line 590

def has_extra_keys(value_type: object = Any) -> Callable[[_T], _T]:```

[trim tangle]

Anyone knows why the bottom two reveal_types here give tuple[Literal["baz"], Unknown] and tuple[Literal["quack"], Unknown]
(warning: cursed typing stuff)
https://pyright-playground.decorator-factory.su/?gzip=H4sIACzJ6GMC_4VTwWrjMBC96ysGn2zQZtnd7h4MWUhc6MWUpS57CUaojlxMHMkryyEp_fiOJDtRTNvVwbZm3pv3ZhjXWu2BsXowgxaMQbPvlDbApVSGm0bJntQWYk5dI5-ndN4YoXlbGI1BCo-nTvzlmsKdkEI3FYU_WhlVqZbCSp4IWcFyAsXRKkrIOgysMZCFgSyiUKkD1w2XZvmoB5GQPATkCHhSg9wuZ04qJY3m18wiZBbvMxPyIHoTAu39jI1ypXZDt8FmXEclvMK9kuIdn4RULe97yHB08TiPTY4sCrZkmaQE8GxFjWNvZGMYi3vR1hR24pQCIg-8HURqCRoJqaMl8OW3U_RseyxpwZCErvE5i7simHHvWU77Vu3rnLmYehZmZuirU19dpBeLBbkQJNZxjNQ1PTY7TqygsC5LCn0KxVQpTM2KjsO7bbSojMfF0y7ZwtnV_D6yms2q-qL36FNs_1v03M686mjbEWYCPoUzDY27QbhNCYR9EBu1Wi2GGJqZdMyedymEQCdshq4V_u51_aiOqOe2LKqVwk38RsfrE9d4jTjr3W5Hl_gLfm8Q953Cj3IK_xt4tcNEfEPhJ4VfCXWbluAhWhwEbxn-_SI-Xl8n86P6Mfkg7d18kn75LD2Zs4DgvAFjCkJ9twQAAA%3D%3D

[hearty shell]

That is some wacky behaviour

y = Cons("bar", "a_string", Cons("baz", "b_string", Cons("quack", """If any type that is generic is used""", None)))
x = Cons("bar", "a_string", Cons("baz", "b_string", Cons("quack", """Anything else""", None)))

reveal_type(look_up("bar", x))
reveal_type(look_up("baz", x)) # Error
reveal_type(look_up("quack", x))

reveal_type(look_up("bar", y))
reveal_type(look_up("baz", y)) 
reveal_type(look_up("quack", y)) # Error

Even if the generic type is fully saturated it behaves the same

[trim tangle]

💀

w a c k

[frigid jolt]

hi, I'm using attrs and I'm defining a class using @attrs.define(...), I have an attribute that looks like this

    date: str = attrs.field(converter=convert_to_date)

and a function

def convert_to_date(string: str) -> datetime: 
     return datetime.strptime(string, "%Y-%m-%d")

now, I've annotated it as str because from the Api I obviously get a string, I was wondering if there's a way to accept a string (from the Api) but to show to the user the attribute as datetime

[soft matrix]

why does date: datetime not work?

[frigid jolt]

because where I create the object I get a typing error, the type that I'm providing is str while the type that the class accepts is datetime

I know that I could type ignore it

or I could call the converter before the object creation

but I was wondering if it can be fixed as it is, without type ignoring or moving things

[soft matrix]

well ig since there isnt support for stdlib converters youre kinda screwed, youll have to type: ignore somewhere

or cast

or call the function yourself before passing it

[frigid jolt]

an example on how should I use cast?

[soft matrix]

typing.cast(datetime, string_date)

[mystic harness]

on line 12 i get a error from pyright that i dont understand fully why:

Expression of type "V@LoggedDict | _Missing" cannot be assigned to return type "V@LoggedDict | None"
  Type "object* | _Missing" cannot be assigned to type "V@LoggedDict | None"
    Type "_Missing" cannot be assigned to type "V@LoggedDict | None"
      Type "_Missing" cannot be assigned to type "V@LoggedDict"
      Type cannot be assigned to type "None"

i get that its saying the return type does not match the annotation, however, it will NEVER return MISSING. self._dict.get(key, MISSING) is never going to occur because it has a if to prevent this. MISSING is not annotated as the return type because that would be lying, the program never returns MISSING... only V | None. I think the only solution is ignoring the error

[frigid jolt]

thanks

[soft matrix]

why dont you just use default = None?

[mystic harness]

just learning about generics and sentinels, this is not a real application

[soft matrix]

thats the default for self._dict.get already i would imagine

[mystic harness]

i'll simplify things then

[trim tangle]

well, pyright can't prove that MISSING is the only instance of _Missing. Tomorrow you can do MISSING2 = _Missing()

you can do if isinstance(default, _Missing):

IIRC there was some pep about sentinels, not sure how it's going

[mystic harness]

yeah i read about it

i think its 664

one sec

[trim tangle]

!pep 666

[rough sluice]

**PEP 666 - Reject Foolish Indentation**

Link

Status

Rejected

Python-Version

2.2

Created

03-Dec-2001

Type

Standards Track

[trim tangle]

o

[mystic harness]

https://peps.python.org/pep-0661/

[trim tangle]

ah it's a draft

[mystic harness]

if not isinstance(default, _Missing):
  ...

works, but sentinels are meant to be used like

if obj is sentinel:
  ...

no?

[trim tangle]

well

define meant 🙂

[mystic harness]

were made to be used in such a way

i'm learning about them so sorry if i'm being extremely stupid

[trim tangle]

yes, it's common to use is with user-defined sentinels

but type checkers don't like that because they're mean

[mystic harness]

alright

thanks for your help

[mystic harness]

i did something i guess

from __future__ import annotations

import typing as t

class Sentinel:
    _registry: t.Dict[str, Sentinel] = {}

    if t.TYPE_CHECKING:
        __name__: str
        __repr__: t.Callable[[Sentinel], str]
        __bool__: t.Callable[[Sentinel], t.Literal[False]]
        __eq__: t.Callable[[Sentinel], t.Literal[False]]

    def __new__(cls: t.Type[Sentinel], name: str, repr: t.Optional[str] = None) -> Sentinel:
        repr = repr if repr is not None else f"<{name}>"

        sentinel = cls._registry.get(name, None)
        if sentinel is not None:
            return sentinel

        cls.__name__ = name
        cls.__repr__ = lambda _: repr
        cls.__bool__ = lambda *_: False
        cls.__eq__ = lambda *_: False

        sentinel = super().__new__(cls)
        return cls._registry.setdefault(name, sentinel)

MISSING = Sentinel("MISSING")

def func(x: Sentinel | int = MISSING) -> int:
    if not x:
        return 0
    return x + 1

print(func())
print(func(1))
>>> 0
>>> 2

this way pyright doesnt complain about x being a sentinel and not int

i got the basecode from the 661 pep because i'm lazy

[trim tangle]

func(0) 🙂

[mystic harness]

ill figure out something useful one day 💀

[jolly anchor]

does anyone know if there's a way to change types based on type checker that's running? I found a small bug in Mypy that's not present in pyright and I want to work around it

[trim tangle]

I don't think there's a consistent way 😦

if you find something, tell us

[jolly anchor]

I will, but I'm more tempted to fix the error in Mypy now 😄

not sure if is straightforward though

uh, I also have a plugin for project, so maybe I can do the swap there

[slender timber]

A new module would be a waste for it

Better to just stash it in typing_extensions

[pastel egret]

Well part of the idea is that it isn't typing-specific, it's solving a general need that happens to also help type checkers.

[wooden terrace]

from typing import cast
...
return self._dict.get(key, cast(V, default) # line 12

[trim tangle]

cast 💀

[wooden terrace]

Does nothing during runtime - it's just there to tell typecheckers that you're 100% certain something is a certain type

[trim tangle]

yeah, but it just "turns off" the type checker

so I would avoid it if there's another way

[wooden terrace]

lol what

It doesn't "turn off" typecheckers, it's asserting to the typechecker that specific value at that moment in time is that specific type. Typecheckers are still gonna throw errors if you try doing things the cast type isn't kosher with.

In text form: ```python
from typing import cast

var: str | dict = {}
var = cast(str, var)
var.upper() # will error at runtime, but mypy has no problems with it
var.get('key') # mypy no likey this though

[trim tangle]

It doesn't "turn off" typecheckers, it's asserting to the typechecker that specific value at that moment in time is that specific type
that's what I meant by turning off the type checker 🙂

it's like a # type: ignore (but smaller in scope)

[wooden terrace]

It's a lot more granular than # type: ignore - it doesn't take the whole line down with it

[wooden terrace]

This is exactly the situation the function is intended for: When the developer knows a value is 100% for sure a certain type but the typechecker doesn't.

[trim tangle]

Yeah, my point is - in this situation it's completely avoidable

[wooden terrace]

But I agree with the sentiment further up in the thread the original implementation should be kosher

mypy is aggressive as hell

[trim tangle]

well, it just doesn't understand this kind of guard 🙂

[wooden terrace]

Not yet 🤷‍♂️ . Speaking of did typing.TypeGuard get back-ported to any versions?

[trim tangle]

Yeah, via typing-extensions

!pypi typing-extensions

[rough sluice]

typing-extensions v4.4.0

Backported and Experimental Type Hints for Python 3.7+

[wooden terrace]

from typing import Any
from typing import TypeVar
from typing import cast


class SentinelMeta(type):
    def __call__(cls, type_=Any):
        return cast(type_, cls)

    def __getitem__(cls, type_):
        return cast(type_, cls)


class Sentinel(metaclass=SentinelMeta):
    ...


V = TypeVar("V")


def function(dict_: dict[str, V], key: str, default: V = Sentinel[V]) -> V | None:
    if default is Sentinel:
        return dict_.get(key)
    return dict_.get(key, default)

(because I hate giving up and this feels a lot more elegant for the "end-user")

[tranquil turtle]

I want to write something like this: ```py
def f(x: T, n: LiteralInt) -> tuple[T, ...]: ...

```py
f(T, 0) -> tuple[()]
f(T, 1) -> tuple[T,]
f(T, 2) -> tuple[T, T]
``` Is it possible to express somehow?
I think it is possible using mypy plugin, but i do not know how to write plugins for mypy. Is there any documentation/tutorial/guide about that?

[trim tangle]

Is it possible to express somehow?
You can write a pyramid of overloads, similar to asyncio.gather. But you can't write a general signature

I think it is possible using mypy plugin
Yes

i do not know how to write plugins for mypy
It's painful

Is there any documentation/tutorial/guide about that?
Kind of, it's very brief https://mypy.readthedocs.io/en/stable/extending_mypy.html

I have an example, though I don't claim it's well-documented, useful or uses the right approach
https://github.com/decorator-factory/mypy-plugin-attempt

See test.py for a demo

[tranquil turtle]

Is it possible to express somehow?
You can write a pyramid of overloads, similar to asyncio.gather. But you can't write a general signature
Right, i forgot about it

Is there any documentation/tutorial/guide about that?
Kind of, it's very brief https://mypy.readthedocs.io/en/stable/extending_mypy.html
I know about it, but is not useful at all because i have zero experience in writing mypy plugins. So im looking for any other more beginner-friendly resource

[trim tangle]

Well, you can look at my example. Or some other examples, like the SQLAlchemy plugin (https://github.com/dropbox/sqlalchemy-stubs -- seems to be this?.. idk, I haven't used sqlalchemy)

But yes, the documentation is very sparse. I was mostly just browsing through mypy code ||in mild despair||

[tranquil turtle]

Your example is good, thank you. I'll take a look

[trim tangle]

in mild despair... because it's kind of all coupled together, many objects expose public flags/knobs/toggles that anyone can mess with
mild because at least it has type annotations 😛 though ironically it doesn't work well with pyright

[tranquil turtle]

There are also builtin plugins, but seems like they behave a bit differently. For example, thay have no plugin function and mypy.plugin.Plugin subclass

[trim tangle]

¯_(ツ)_/¯

I think it would be more valuable to work on some "in-band" customizations, something like "typing macros"/"type-level functions", than on out-of-band stuff specific to a particular type checker

I don't know if it's really worth the complexity. But it would be good if many things currently hardcoded in type checkers were extracted like that

type annotations are already complicated as hell

at least once you go down the rabbit hole of details

mypy and pyright are rather complex pieces of software

[tranquil turtle]

# code:
if TYPE_CALCULATION:
  from thislib.typing import calc_type_of_f # is not imported at runtime but typechecher know from where import hooks
def f(*args: Any, **kwargs: Any) -> CalculatedType[calc_type_of_f]: ...


# thislib/typing.py
# every time function f is used typechecher calls this hook to determine return type
def calc_type_of_f(ctx: Context, *args: X) -> X:...

# X is just a regular python object (not typechecker-specific) which is acceptable in annotations. For example, classes (int, str), specialized generic classes (list[int], list[T]), typevars, unions, special forms (Any, Never), ...
# hook will parse these objects and return annotation-object for return type. After that typechecker can parse it into internal representation and continue working 

I think this would require adding typecheking-related stuff to stdlib

Because differentiating between Union[a,b], a|b and a|b can be painful. (a|b is not always a types.UnionType object, it can be of type typing._UnionType IIRC)

[tranquil turtle]

So it is basically like mypy plugins, but it is not using mypy-specific objects

This also require typecheckers to be able to run python code and pass/receive some objects to it.
It is easy for mypy, but is almost impossible for pyright

Why does Pyright exist at all? How did someone come up with the idea to write a python typechecker in js?

[trim tangle]

Why does Pyright exist at all?
Pyright moves much faster than mypy, has all the shiny new features and is faster.
It also provides editor integration (show type on hover, auto-imports, go to definition etc.)

How did someone come up with the idea to write a python typechecker in js?

1. TypeScript 🙂
2. It's faster. Perhaps even considering mypyc, but don't quote me on that
3. Microsoft probably wanted to integrate well with VSCode or browsers
4. What's wrong with that?

[hearty shell]

Last I checked mypy doesn't hold a candle to Pyright when it comes to the on the fly experience, they support partial AST and asynchronous calls to the type checker, meaning fast type information

[trim tangle]

CPython is written in C, PyCharm is written in Java

[tranquil turtle]

partial AST
You mean it can recover from syntax errors? Yes, it is a great feature

[tranquil turtle]

weird weird weird

[trim tangle]

I suppose the same reason hadolint is written in Haskell and not Bash 😄

though the Haskell compiler is mostly written in Haskell

[tranquil turtle]

I think it is a bit unnatural to write typechecker for language in different language.

[hearty shell]

Yup, mypy wasn't designed for that I think, it is more of a tool you integrate into your CI

[trim tangle]

PyCharm's built-in type checker is written in Java. Somehow people don't complain very often

though yeah, it would probably be easier to contribute to if it was written in Python

Rust would probably be a better choice though 🙂

[tranquil turtle]

And using also would be easier
To use pyright you need node.js be installed, but for mypy you need only python interpreter (and you likely to have it)

[trim tangle]

that is true

just like you need Java to run PyCharm 🙂

[tranquil turtle]

Mypy supports plugin written in python. It is not hard for you (as python developer) to understand it and write your own plugin
But to write plugin for pyright (if they even exists) you need to learn ts (because not every python developer knows ts)

[trim tangle]

It is not hard for you (as python developer) to understand it and write your own plugin
well, as you can probably see, it's not that easy

[tranquil turtle]

It is possible in reasonable time
You can't learn new language and write plugin in it in shorter time

[hearty shell]

Pyright doesn't support plugins because they disagree with the design, requiring 3rd party software to type check programs was not something Eric agreed with I think (security and whatnot )

[rare scarab]

plus, who would want to write a javascript plugin for type checking python scripts?

[brisk hedge]

I don't think that's the problem there

[cunning plover]

class User(pydantic.BaseModel):
    id: int

class EmailResult(pydantic.BaseModel):
    id: int

def email_send(input: User) -> EmailResult:
    return EmailResult(id=input.id)

def func_to_task_need(
    function: Callable[[pydantic.BaseModel], pydantic.BaseModel]
) -> task_need:
    return task_need(function.__name__)

func_to_task_need(email_send) yields mypy error

(function) email_send(input: User) -> EmailResult
Argument 1 to "func_to_task_need" has incompatible type "Callable[[User], EmailResult]"; expected "Callable[[BaseModel], BaseModel]"  [arg-type]mypy

error appears only when i replace Callable argument from any to BaseModel (accepted output fine)

how correctly to type this situation?

def func_to_task_need(function: Callable[[Any], pydantic.BaseModel]) -> task_need:
    return task_need(function.__name__)

is accepted without errors, but oh well, i don't want Any as type

[soft matrix]

you cant really

my best suggestion would be

import pydantic

from collections.abc import Callable
from typing import Any, TypeVar

class User(pydantic.BaseModel):
    id: int

class EmailResult(pydantic.BaseModel):
    id: int

def email_send(input: User) -> EmailResult:
    return EmailResult(id=input.id)


ParamT = TypeVar("ParamT", bound=pydantic.BaseModel)
RetT = TypeVar("RetT", bound=pydantic.BaseModel)

def func_to_task_need(function: Callable[[ParamT], RetT]) -> RetT:
    return task_need(function.__name__)```

but mypy might need a type ignore on the func_to_task_need line cause free type vars is a spooky concept in python

but that should at least make the other issues go away

[cunning plover]

huh. It works. without any type ignores

you are magician 🙂

[trim tangle]

To elaborate on why your version doesn't work -- you're declaring that function should be comfortable with accepting any kind of BaseModel, because inside of func_to_task_need it's legal to do e.g. func_to_task_need(EmailResult(id=42))

[cunning plover]

makes sense 🙂 probably

TaskFuncInput = TypeVar(
    "TaskFuncInput",
    Type[pydantic.BaseModel],
    List[Type[pydantic.BaseModel]],
)
TaskFuncOut = TypeVar(
    "TaskFuncOut",
    Type[pydantic.BaseModel],
    List[Type[pydantic.BaseModel]],
)
TaskFuncType = Callable[[TaskFuncInput], TaskFuncOut]

i thought any instance of EmailResult(id=42) satisfies being parent of pydantic.BaseModel

hmm... no, i don't get it

class typing.TypeVar
Bound type variables and constrained type variables have different semantics in several important ways. Using a bound type variable means that the TypeVar will be solved using the most specific type possible:

oh.

[trim tangle]

Yeah you need to use bound

Don't use the constrained thing, just forget it exists

👀

[cunning plover]

https://www.youtube.com/watch?v=MOuMZWarmxg

[oblique urchin]

the constrained thing here means your type needs to be solved to exactly type[BaseModel] or list[type[BaseModel]]

which is almost certainly not what you want

[cunning plover]

certainly

[cunning plover]

TaskFuncOut = TypeVar(
    "TaskFuncOut",
    Type[pydantic.BaseModel],
    List[Type[pydantic.BaseModel]],
)

so TypeVar magically makes it not constrained then 🤔 allowng BaseModel being parent to supplied type

[oblique urchin]

that's a "constrained" TypeVar to be clear

[cunning plover]

hmm

how to unconstrain?

[oblique urchin]

bound=Union[Type[pydantic.BaseModel], List[Type[pydantic.BaseModel]]]

[cunning plover]

(function) task_factory(
    function: TaskFuncType,
    input: TaskFuncInput@task_factory,
    output: TaskFuncOut@task_factory,
    needs: TaskNeededType,
    type: TaskQuantityType
) -> Task
Value of type variable "workflow.TaskFuncOut" of "task_factory" cannot be "Type[List[Any]]"  [type-var]mypy

hmm it yield error to

task_factory(
    function=query_db,
    type=workflow.TaskQuantityType.SINGLE,
    input=QueryDBParams,
    output=List[User],
    needs=workflow.REQUEST_INPUT,
),

Probably because i need refactoring

TaskFuncOut = TypeVar(
    "TaskFuncOut", bound=Union[Type[pydantic.BaseModel], List[Type[pydantic.BaseModel]]]
)

into two types TypeVars then

refactored to

TaskFuncOutParam = TypeVar("TaskFuncOutParam", bound=Type[pydantic.BaseModel])
TaskFuncOut = Union[TaskFuncOutParam, List[TaskFuncOutParam]]

and it worked 😆 and it did not work

letss see

TaskFuncInput = TypeVar(
    "TaskFuncInput",
    Type[pydantic.BaseModel],
    List[Type[pydantic.BaseModel]],
)
TaskFuncOut = TypeVar(
    "TaskFuncOut",
    Type[pydantic.BaseModel],
    List[Type[pydantic.BaseModel]],
)
TaskFuncType = Callable[[TaskFuncInput], TaskFuncOut]

and call is

task_factory(
    function=query_db,
    type=workflow.TaskQuantityType.SINGLE,
    input=QueryDBParams,
    output=List[User],
    needs=workflow.REQUEST_INPUT,
),

(function) task_factory(
    function: TaskFuncType,
    input: TaskFuncInput@task_factory,
    output: TaskFuncOut@task_factory,
    needs: TaskNeededType,
    type: TaskQuantityType
) -> Task
Value of type variable "workflow.TaskFuncOut" of "task_factory" cannot be "Type[List[Any]]"  [type-var]mypy

when i insert List[User]... it for some reason was not accepted as i expected. internal type of value in List dissapeared 🤔

for this probably thing we need unconstrainidness

[cunning plover]

BaseModelInList = Any
TaskFuncInput = TypeVar(
    "TaskFuncInput",
    Type[pydantic.BaseModel],
    List[BaseModelInList],
)
TaskFuncOut = TypeVar(
    "TaskFuncOut",
    bound=Union[Type[pydantic.BaseModel], Type[List[BaseModelInList]]],
)
TaskFuncType = Callable[[TaskFuncInput], TaskFuncOut]

Ergh, no idea how correctly to bind BaseModelInList correctly. this hack works for temporal solution though

more minimal setup to repeat the problem

BaseModelInList = Type[pydantic.BaseModel]
TaskFuncOut = TypeVar(
    "TaskFuncOut",
    bound=Union[Type[pydantic.BaseModel], Type[List[BaseModelInList]]],
)
TaskFuncType = Callable[[TaskFuncInput], TaskFuncOut]


def func_test_typing(function: TaskFuncOut) -> None:
    pass


class User2(pydantic.BaseModel):
    name: str


func_test_typing(List[User2])

(function) func_test_typing(function: TaskFuncOut@func_test_typing) -> None
Value of type variable "TaskFuncOut" of "func_test_typing" cannot be "Type[List[Any]]"  [type-var]mypy

hmm problem in List[User2] transformed to Type[List[Any]], while i expected it to be Type[List[BaseModel]]

[woven cliff]

is there any decent way to type this for the self.X.url stuff (since it can be None) without using if statements for every line (i can if i have to but why not ask if there's a better way), i was thinking i could do something with the has_x methods but i dont know, any suggestions?
https://paste.pythondiscord.com/aqiqudiwev

[cunning plover]

Best way to make X with URL always present 🙂 stop using None

[hearty shell]

What if a Guild has no banner?

[woven cliff]

Yeah, the thing is icon, splash, and banner can all be None

i could do

if self.icon.url is not None

for all of them, but i find it as an ugly solution, just wanted to see if there are better ways of doing it

[hearty shell]

PEP 505

Although I dont see how that is type related, the representation you have is fine I think considering you have to play with the discord library

Which returns None for a bunch of things

[woven cliff]

i thought it'd be a better place to ask since it is a bit type related, but yeah i get that

[dull lance]

This may be overkill but you can apply the State pattern and encapsulate the state of each field, so you get two classes: one class for the state where the field is not None and another class where the field is None. In each class you define the operations to be done relating to that field. These operations no longer have to differentiate between whether it is None, since you separated the logic into the two classes.

Again, you should think about whether this is worth the effort before commiting to this xD

[woven cliff]

part of me wants to try it, but at the same time its a bit much 😭

[trim tangle]

Can you share your code maybe?

[woven cliff]

i did here, i know there is a very simple solution but i decided to ask anyway

[trim tangle]

Oops sorry I missed

[trim tangle]

Where would you use this object?

Besides turning it into a string

I meant, can you show a usage example for this object?

[woven cliff]

so far its only being used in a guild info command in an embed

embed.add_field(name="Graphics", value=GuildGraphics.from_guild(guild), inline=True)

[trim tangle]

So add_field is a method from your own code?

[woven cliff]

i was only asking out of sake of curiosity for typing itself tbh, i know i can just do if self.icon: do stuff, etc

add_field is a method from the Embed class in discord.py

[trim tangle]

Yeah I'd do something like this

@dataclass(frozen=True)
class GuildGraphics:
    guild_id: int
    icon: discord.Asset | None
    splash: discord.Asset | None
    banner: discord.Asset | None

    @classmethod
    def from_guild(cls, guild: discord.Guild, /) -> GuildGraphics:
        guild_id = guild.id
        icon = guild.icon
        splash = guild.splash
        banner = guild.banner

        return cls(guild_id=guild_id, icon=icon, splash=splash, banner=banner)

    def to_text(self) -> str:
        ret = ""

        if self.icon:
            ret += f"\n**Icon:** [click here]({self.icon.url})"
        if self.splash:
            ret += f"\n**Splash:** [click here]({self.splash.url})"
        if self.banner:
            ret += f"\n**Banner:** [click here]({self.banner.url}))"

        return ret.strip() or "(no graphics)"

I'd probably also make both of the methods as free functions

What kind of object does add_field expect?

Oh wait, it accepts a string

In that case you shouldn't pass in a GuildGraphics object

[trim tangle]

Yeah I'd do something like this

def report_graphics(guild: discord.Guild) -> str:
    ret = ""

    if guild.icon:
        ret += f"\n**Icon:** [click here]({guild.icon.url})"
    if guild.splash:
        ret += f"\n**Splash:** [click here]({guild.splash.url})"
    if guild.banner:
        ret += f"\n**Banner:** [click here]({guild.banner.url}))"

    return ret.strip() or "(no graphics)"

[woven cliff]

fair originally i wanted to have other stuff that would benefit from the class but those dont matter much + dont really go with it, either way thanks

[trim tangle]

Of course there's a trade-off. Making an intermediate representation decouples the formatting of this information from how you gather it (e.g. maybe you want to get it from a local cache instead of banging on Discord's servers)

But in this case I think a function is fine

[stray summit]

Hmm, i really wish it was easier to write depend types plus validate partial application

I have a get function like def get(name: str, *, default: D|None=None, convert: Callable[[str], T] |) -> str|D|T|None

I need dozens of overloads for the combinations
I'd like to express them more sane

Either as special Union that automatically picks the first usable type, or as concept /trait

[trim tangle]

What does the function do?

And what is convert?

[stray summit]

On missing, return default, on found apply convert if given, then return

[trim tangle]

What happens if convert is not given? Is None returned on missing value?

Also, is this an existing public function?

[stray summit]

If convert is not given, a Str is returned

This is part of the api of iniconfig, the dutiful parser for pytest.ini

[trim tangle]

I mean, if convert is not given and the value is missing

[stray summit]

When missing the value, default is returned, which is none unless a different value is passed

[trim tangle]

What if both default and convert are given?

[stray summit]

If missing, return default, else apply convert and return the result

[trim tangle]

So:

• if default is not provided or is None:
  • if convert is provided, return T | None
  • if convert is not provided, return str | None
• if default is provided:
  • if convert is provided, return T | D
  • if convert is not provided, return str | D

That sounds like 4 overloads

[stray summit]

I had to provide 6 to ensure matching combinations with none, and I implemented it bad enough that delegation can't be typed in the same way

[trim tangle]

Why 6?

[stray summit]

To handle none in combination with convert and default

I still haven't figured why I needed the additional ones

Just that it breaks if I leave them

[trim tangle]

() -> str | None
(default: D) -> str | D
(convert: (str -> T)) -> T | None
(default: D, convert: (str -> T)) -> T | D

Am I missing something?

[stray summit]

Explicit pass in of none from delegating functions

[oblique urchin]

(convert: None = ..., default: D)

for overload 2

[stray summit]

It wouldn't let me have that as part of a declaration with a default

[trim tangle]

For default, it will work as intended with D=None. For convert, just pass in str instead of None

[stray summit]

Hmm, let me try to reduce it that way

[trim tangle]

And for overload 1 yeah.

But I don't see the need to support None in that argument

Besides backwards compatibility

[oblique urchin]

could be useful for wrapper functions

[trim tangle]

But you can use str as a default

[stray summit]

Delegates that use the default and pass it over

The alternative is to ensure delegates share the default

[trim tangle]

Or lambda x: x

[stray summit]

Which means sync needed

[trim tangle]

Here you also evidently have to ensure delegates share the default 🙂 since you can't change this function's default without changing the delegates

[stray summit]

The none as sentinel for a missing argument passes relatively straightforward

It just types inconvenient

Well, a unittest to ensure parity seems less painful than overly complicated overload.

[trim tangle]

Well, you can do what Jelle suggested

Add a convert: None = ... to overloads without convert

[stray summit]

hmm, ah - error: Incompatible default for argument "convert" (default has type "Type[str]", argument has type "Callable[[str], _T]"

it seeems i can only pass cast(Callable[[str], _T], str) as the default, im not sure whether it understands the details

[trim tangle]

Can you show the code maybe?

[stray summit]

https://github.com/pytest-dev/iniconfig/blob/main/src/iniconfig/__init__.py#L46-L87

[trim tangle]

https://pyright-playground.decorator-factory.su/?gzip=H4sIAAnr62MC_61SQWrDMBC87ysWn6yi-gGGlJaaHnsyvYRQFGcVTGTJtZWAoY_vKq5MCCVNTHSSdnZmdsTqzjVYOWOo8rWzfabWFdZN6zqPr8oYtTYEOjT5oa3tNmLl0NKH6iS-2EGiO1BnnNoAQIGLCKZJkQgoTwslFwAqo_oe35zLAfk8T_Tw2pDGLfm0J6MlWtVQjr1npwfJg1pu9Tm-O0usm2WZwMengOP3sTgqhsMYzJWPyZdLhlYSy9XRpryHCWNqb9ik-C9PcX-fC8HO3OL9Wp9xeonR6yQNr8hZEgiCOtXOYR72QIx4RwdS5pM3jQKWBcvka6-qXSIETNNcaJuiLsKPXk36zXAb6dRppkQkpWIGZWbWSKqtZ848w5F7-9_-pfADC0lNtYIEAAA%3D

(sorry, it doesn't run on mobile)

fixed link*

[trim tangle]

mirror

[trim tangle]

Why does mypy not like this? Is this a bug?

from typing import TypeVar

T = TypeVar("T")

def identity(t: T) -> T:
    return t

def f(s: str):
    parts = s.split()
    hmm = list(map(identity, parts))  # error: Argument 1 to "map" has incompatible type "Callable[[T], T]"; expected "Callable[[str], T]"  [arg-type]
    reveal_type(hmm)
    

[trim tangle]

@stray summit were you able to make it work?

[stray summit]

only partially - i did not get the types for the function delegating to it right yet, and my net chance to try is Friday ^^

[heady flicker]

Type hinting experts, would you explain why expr ends up being "bool" in "foo" function?

https://pyright-playground.decorator-factory.su/?gzip=H4sIALfa7GMC_81TwWrDMAy9-ysEu6wQcsixsMtg17VQeg5JqmwajhRkLTTQj5_SbIytWwM7zQfZlp6erGe7VenAxp74qcSjIScSTkBdL2pQpYRqJeOAGkJoojtgJx1u7Bl1U79gY-sAPnqPOGJnSo3F8V4kYsUPx14xTZR7dgN38CiMcILa42Gr1JHRgJewBZ4TEJvbNko1zck0bMUB9ZXSi5zfGvN2DthCK3KLjl3Db-ddzRJQC5SIk1Xc4DklW6j5njgNRXtVPm8x_sR0vb0vTANWsfQ7nVNXcAN79pW3hIc4ZrP4C3oVfxDMPXmef8pW_H_digXhLp5Z5n8CPsQM4Q14ElDjPwMAAA%3D%3D

Feels like a bug in pyright

[trim tangle]

What did you expect it to be?

also what's up with the ...

[heady flicker]

I expected the result to be the same as in foo2

[heady flicker]

No idea. It was my wild guess that fixed pyright's handling of it.

[trim tangle]

StrictlyBooleanExpressionUnion = None | bool
PrimitiveExpressionUnion = StrictlyBooleanExpressionUnion | int | float | str
PossiblyBooleanExpressionUnion = StrictlyBooleanExpressionUnion | SomeOtherObject


#   foo(expr: PrimitiveExpressionUnion):  [expand]
#   foo(expr: StrictlyBooleanExpressionUnion | int | float | str): [expand]
def foo(expr: bool | None | int | float | str):
#   if isinstance(expr, StrictlyBooleanExpressionUnion): [expand]
    if isinstance(expr, None | bool):
        return
#   elif isinstance(expr, PossiblyBooleanExpressionUnion): [epand]
    elif isinstance(expr, None | bool | SomeOtherObject):
        reveal_type(expr)

The only way expr is both a PossiblyBooleanExpressionUnion and a PrimitiveExpressionUnion is if it's bool

so... actually expr should be Never

which does sound like a bug, but a different one 😄

[heady flicker]

Oh, right, yes.

So I am not insane and It is a bug

Thanks!

[trim tangle]

Why did you expect it to be str | int | float though?

[heady flicker]

That was a brainfart from my side, actually. I did expect it to be Never but then I rewrote it from production code into a simpler example and got confused.

Take a look at this example too:

def foo(expr: None | bool | int | float | str | SomeOtherObject):
    if isinstance(expr, None | bool):
        return
    elif isinstance(expr, None | bool | SomeOtherObject):
        reveal_type(expr)

Here pyright thinks that it's bool | SomeOtherObject

[trim tangle]

💀

stuff like this makes me legit want to write my own type checker

[heady flicker]

Oh, funny

Pyright is right, we are wrong

def foo(expr: bool | int | SomeOtherObject):
    if isinstance(expr, bool):
        return
    elif isinstance(expr, bool | SomeOtherObject):
        reveal_type(expr)

[trim tangle]

hm

[heady flicker]

It makes a lot of sense:
isinstance(0, bool) is False
isinstance(False, int) is True

Or maybe not a lot of sense

Need to think about this

[trim tangle]

no, there's no way expr is a bool in the second branch

[heady flicker]

Okay, yeah, no way

Hm... Mypy has the same issue

I think this has to do with type intersections somehow.

[hearty shell]

I was testing this myself and at some point I 100% thought it was bugging with floats too but then I remembered isinstance(42, float) or isinstance(True, float) are both false

😵‍💫

One funny bug is that

if isinstance(expr, object):
    return
elif isinstance(expr, None | bool):
    reveal_type(expr) # Type of "expr" is "None"

[trim tangle]

uhhhhhhhhh

☠️

[heady flicker]

I asked Alex Traut about this, and he described this as "is-designed"

Which technically makes sense because essentially an int is type narrowed to a bool.

[heady flicker]

So we checked that it's not a bool, but then it can still be an int, and an int can be safely casted into a bool, which makes a lot of sense from the type system's perspective.

[soft matrix]

you are aware of

!d bool

[rough sluice]

bool

class bool(x=False)```
Return a Boolean value, i.e. one of `True` or `False`. *x* is converted using the standard [truth testing procedure](https://docs.python.org/3/library/stdtypes.html#truth). If *x* is false or omitted, this returns `False`; otherwise, it returns `True`. The [`bool`](https://docs.python.org/3/library/functions.html#bool "bool") class is a subclass of [`int`](https://docs.python.org/3/library/functions.html#int "int") (see [Numeric Types — int, float, complex](https://docs.python.org/3/library/stdtypes.html#typesnumeric)). It cannot be subclassed further. Its only instances are `False` and `True` (see [Boolean Values](https://docs.python.org/3/library/stdtypes.html#bltin-boolean-values)).

Changed in version 3.7: *x* is now a positional-only parameter.

[soft matrix]

The bool class is a subclass of int

im pretty sure there werent even dreams of a type system when bool was implemented as an type

[oblique urchin]

maybe there were! bool was implemented surprisingly late (2.2 I think)

[soft matrix]

whats was jukka's language called do you remember?

[leaden oak]

mypy, no?

[heady flicker]

I am. My issue here was that the conversions pyright did were quite confusing.

[soft matrix]

no it was a precursor to mypy and typed python

[oblique urchin]

it was already called mypy

[soft matrix]

https://github.com/JukkaL/alore was what i was thinking of

[leaden oak]

Huh, I thought mypy was originally a custom implementation and then got transformed into a type checker later on, but looks like I misremembered

[soft matrix]

well this doesnt look like its 2001 levels of old

but maybe jelle knows more about this than i do

[oblique urchin]

this is also before my time but https://peps.python.org/pep-3107/#use-cases has some very early links on typing in Python

[soft matrix]

huh cool i didnt realise there was stuff like 8 years before i thought

[hearty shell]

It does make sense yeah, otherwise the type checker would have to start carrying "lower bound" constraints, like after isinstance(expr, bool) with expr: bool | int, expr would have be reduced to expr: T, T :< Int & T >: bool, so unless there is some other system to carry though that information as something else then a lower bound it makes sense why the type checker is not able to guess that

[wooden terrace]

Does anybody have experience testing with multiple typecheckers/linters when writing a module?

As in "I want to make sure my module plays nice with multiple popular linters"

[trim tangle]

I usually try to get this thought out of my head 🫠

hmmm something like tox?

[oblique urchin]

yes, I routinely use mypy, pyright, and pyanalyze

[trim tangle]

||rename to pyanal for consistency with mypy internals 🥴 ||

[wooden terrace]

I've used mypy and pyright. Not familiar with pyanalyze. Will have to check it out.

[oblique urchin]

I wrote it so I'm biased in its favor 🙂

[wooden terrace]

Guess I know who to ask design/hard questions then lol

[trim tangle]

Jelle regrets decision

[wooden terrace]

Promise I'll comb through docs and the bowels of the internet before that lol

[trim tangle]

||I bet there's a lot of bowels behind pyanal||

ok I should stop with the 10y.o. level jokes

[wooden terrace]

https://tenor.com/view/marceline-adventure-time-flirty-gif-20739264

[wooden terrace]

life is too short to be an adult all the time

[digital void]

why do we have circular import errors. It makes type hinting impossible at times. It makes me want to die.

[soft matrix]

!d typing.TYPE_CHECKING

[rough sluice]

typing.TYPE\_CHECKING

typing.TYPE_CHECKING```
A special constant that is assumed to be `True` by 3rd party static type checkers. It is `False` at runtime. Usage:

```py
if TYPE_CHECKING:
    import expensive_mod

def fun(arg: 'expensive_mod.SomeType') -> None:
    local_var: expensive_mod.AnotherType = other_fun()
```  The first type annotation must be enclosed in quotes, making it a “forward reference”, to hide the `expensive_mod` reference from the interpreter runtime. Type annotations for local variables are not evaluated, so the second annotation does not need to be enclosed in quotes.

[soft matrix]

might help here

replace expensive_mod with circular_mod

[digital void]

my IDE always tells me that it can't find the module

I'm using mypy so I thought it would be able to do it 😦

oh wait, I didn't read that "if TYPE_CHECKING" bit

that does seem useful

[tranquil turtle]

You probably also need from __future__ import annotations

[snow plover]

class BaseFoo:
  pass


class Bar(BaseFoo):
  pass

a = Bar  # note: not initialized

# how would i type hint for non-initialized class?

how to type hint for non-initialized class? is it possible, is it necessary?
(please ping if you respond)

[void panther]

type[Bar]

[snow plover]

awesome, thanks

[cunning plover]

is there any easy short syntax for stuff like

abc = workflow_id(123) # or abc: workflow_id = 123
asd = 123

def foo(value: workflow_id) -> None:
    pass


foo(asd) # Mypy error, because was supplied int instead of workflow_id

?

found

from typing import NewType

UserId = NewType('UserId', int)
some_id = UserId(524313)

[soft matrix]

i think youre looking for something like an as keyword

oh yeah i thought you were already using new type

:P

[trim tangle]

Yeah NewType exists 👍

[cunning plover]

not yet, but will be now

WorkflowId = NewType("WorkflowId", str)

abc = WorkflowId("abc")
asd = "asd"


def foo(value: WorkflowId) -> None:
    pass


foo(asd) # error

perfectly works. 🙂

safeguarding me from too generic types

[trim tangle]

One thing I'm missing in NewType is the ability to limit the instantiation of a thing to one module. Because with NewType you can just do UserId(42069) and you can pull any number out of your... ear

[soft matrix]

https://github.com/python/typing/issues/953 fwiw

imo not a great issue but has some interesting ideas

[spiral fjord]

a: Str | Int = 0  # SUS ALERT```

[trim tangle]

Or at least something like the classic pattern of... I forgot the name ```py
class Email:
def init(self, raw: str, /) -> None:
if EMAIL_REGEX.match(raw) is None:
raise ValueError("skill issue")
self._raw = raw

def raw(self) -> str:
    return self._raw

[soft matrix]

phantom types

[cunning plover]

Problem.

WorkflowId = NewType("WorkflowId", str)

abc = WorkflowId("abc")
asd = "asd"


def foo(value: str) -> None:
    pass


foo(abc)  # NO ERROR

this does not yield error. i expected to be. How to go around of it? :/

[trim tangle]

You need to do value: WorkflowId

right now you're accepting any string

[cunning plover]

i know, i want WorkflowId become protected from being used in str

[trim tangle]

oo

[cunning plover]

i mean workflowID should not autoconvert to str. i was explicitely converting it to str with str(value)

[soft matrix]

NewType(..., UserString)

:)

[trim tangle]

❌

[trim tangle]

Then I'd just make a class. Something like a dataclass or a NamedTuple

though a NamedTuple is also a tuple so I'd avoid that

[cunning plover]

sad.

[trim tangle]

Well... it's 3 lines instead of 1

[spiral fjord]

What is the usecase for not allowing string subtypes to be used as strings?

[trim tangle]

It makes little sense to add two emails

[trim tangle]

you mean opaque types?

yeah

[cunning plover]

ergh, it is use case, easily refactoring code where you used workflow_id through code as str, and wishing to replace everywhere to strict WorkflowId type, which prevents it being used in wrong function arguments by accident (in older functions which still accept it as str type)

[soft matrix]

i swear theyre phantom types

[trim tangle]

hmm we might be talking about different things

https://wiki.haskell.org/Phantom_type

though I've seen them used to indicate unit in F# or something

[soft matrix]

yeah thats what the example in the rust book has them for

[trim tangle]

You could do something like ```py
_UserId = NewType("_UserId", int)
UserId = NewType("UserId", _UserId)

async def get_all_user_ids() -> list[UserId]:
...
``` now you can be sure that if you've got a UserId, it must've came from this function. Well... as long as you don't touch the no-touchie stuff

[soft matrix]

but then you have return [UserId(_UserId(x)) for x in ...] 😢

[trim tangle]

that is true

[soft matrix]

well actually you should just cast there

but still NewTypes in NewTypes really is very weird in python

[trim tangle]

ah yes, those precious nanoseconds won't save themselves

[cunning plover]

Note that these checks are enforced only by the static type checker. At runtime, the statement Derived = NewType('Derived', Base) will make Derived a callable that immediately returns whatever parameter you pass it. That means the expression Derived(some_value) does not create a new class or introduce much overhead beyond that of a regular function call.
https://docs.python.org/3/library/typing.html
documentation promises us it is super duper fast just one func call

[trim tangle]

Yeah it's just an identity function at runtime

Haskell/Rust/Elm/Purescript have "newtype" which is actually pretty neat

It's not a subtype of the wrapped type, and you can't "twiddle with it"

[soft matrix]

yes but call overhead of cast is less than the overhead of calling the NewType.call on every element

[cunning plover]

golangs new type is pretty neat too

type Dbpath string

it forbids using new types in the place of its parent(string) until you made explicit conversion back string(dbpath)
one line = a lot of protection

[trim tangle]

here it doesn't provide even that

Which is useful e.g. if you want to be able to change the implementation of an Id from an int to a string

[maiden lagoon]

I need a function that accepts
"S", str
Or
"I",i

So
foo("S","5")
foo("I",3)
are valid but not
foo("S",3)

how van I solve it with type hinting?

[brisk hedge]

Using Overloads and Literal, so

@overload
def foo(x: Literal["S"], y: str): ...
@overload
def foo(x: Literal["I"], y: int): ...
def foo(x, y):
    # Your original function here

overloads are a way to give more precise types for each specific way to call a function

[tranquil turtle]

It is a bit similar to private subclasses in CPP. It is not visible from outside that one class is a child of another.

[kind kestrel]

how do I keep my linter happy here ? (thinking about it, it's probably by type hinting the lambda expression ? )

[hallow flint]

i think filter(None, spell_words) should work

(although it reads a little less obvious than the spelled out lambda)

[kind kestrel]

yes you are right

and it makes linter happy too. thanks 😄

I need to learn to type hint lambdas though 🙂 good day !

[tranquil turtle]

def is_not_none(x: T | None) -> TypeGuard[T]: return x is not None

filter(is_not_none, spell_words)
``` this should work

[stray summit]

anyone aware of a nice pattern to enable optional generics for libraries that support python3.8+

or alternatively managing descriptors in a way that type-annotations can nicely pass over

aka enable

class MyClass(...):
   foo = MagicProperty[int]() #enable this
   bar = MagicProperty() # keep allowing that

[dull lance]

didn't even realize that is something you could do in python

MagicProperty[int]()

[maiden lagoon]

!e

[rough sluice]

Missing required argument

code

Command Help

!eval [python_version] <code, ...>
Can also use: e

Run Python code and get the results.

This command supports multiple lines of code, including code wrapped inside a formatted code block. Code can be re-evaluated by editing the original message within 10 seconds and clicking the reaction that subsequently appears.

If multiple codeblocks are in a message, all of them will be joined and evaluated, ignoring the text outside of them.

By default your code is run on Python's 3.11 beta release, to assist with testing. If you run into issues related to this Python version, you can request the bot to use Python 3.10 by specifying the python_version arg and setting it to 3.10.

We've done our best to make this sandboxed, but do let us know if you manage to find an issue with it!

[dull lance]

the more you know

[maiden lagoon]

!e from typing import overload,Literal,NewType,Tuple
s=NewType("s",Tuple[Literal["S"],str])
i=NewType("i", Tuple[Literal["I"],int])

@overload
def foo(x:s)->None:
print("valid String")
@overload
def foo(x:i)->None:
print("valid Integer")
def foo(*args):
print("somethings wrong here")

a= s(("S","i"))
b = i(("I",5))
print(type(a))
foo(a)
foo(b)

[rough sluice]

@maiden lagoon :white_check_mark: Your 3.11 eval job has completed with return code 0.

001 | <class 'tuple'>
002 | somethings wrong here
003 | somethings wrong here

[soft matrix]

sounds like you need

[maiden lagoon]

I didnt expect this , I expected valid integer or valid String

[soft matrix]

!pep 696

[rough sluice]

**PEP 696 - Type defaults for TypeVarLikes**

Link

Status

Draft

Python-Version

3.12

Created

14-Jul-2022

Type

Standards Track

[soft matrix]

and also maybe https://github.com/python/cpython/pull/101827 when it comes out

[stray summit]

@soft matrix well, im aware of that, and i cant use that , im on py3.8

[dull lance]

!e

my_list = list[int]()
print(my_list)

[rough sluice]

@dull lance :white_check_mark: Your 3.11 eval job has completed with return code 0.

[]

[soft matrix]

import TypeVar from typing_extensions?

[dull lance]

I mean, you could already do that, apparently

[stray summit]

@dull lance that example you posted will fail on python3.8

[dull lance]

oh, you need it to work on py38

hmm

[soft matrix]

also you know this valid on 3.7+ if you subclass typing.Generic and dont use GenericAlias?

[dull lance]

do you need to do it only for a specific class or does it need to be applied to all generic types?

[soft matrix]

overload does nothing at runtime

if you want this to work it probably works with functools.multipledispatch

[stray summit]

i need to use this pattern for 2 classes

ok, its i get errors for using typing extensions typevar atm, its not supporting default, and its not supporting using Any

so the pep is not usable in my code atm

[soft matrix]

you need to update it then

its in 4.4.0 iirc

although the default isnt Any cause some people may wish to have it be unknown

the value when nothing is passed is None

[stray summit]

hmm, stil lget errors about the default parameter

hmm, its in 4.5 it seems, still errors tho, perhaps a mypy issue

[soft matrix]

its in mypy

yeah well the pep hasnt been accepted so its not in mypy yet

use pyright or my fork for it https://github.com/Gobot1234/mypy/tree/TypeVar-defaults

[stray summit]

hm, i cant use the fork and i cant use pyright 😦

btw, any reason for not making those changes a pr to mypy?

oh, that comment 🙃

[lone timber]

hey guys, I've developed a data type declaration and parsing library based on Python type annotations called utype (https://utype.io/) , which enforce types and constraints for classes and functions at runtime, hopes this can be a good help for you

[rare scarab]

How does it compare to pydantic?

Do required arguments play nice with static type checkers? i.e. will pyright or mypy detect that username is required?

[rare scarab]

Do you support foo: Annotated[str, utype.Param()] instead of foo: str = utype.Param()?

[cunning plover]

Q = TypeVar("Q", bound=pydantic.BaseModel)
W = TypeVar("W", bound=pydantic.BaseModel)
E = TypeVar("E", bound=pydantic.BaseModel)
R = TypeVar("R", bound=pydantic.BaseModel)
T = TypeVar("T", bound=pydantic.BaseModel)
Y = TypeVar("Y", bound=pydantic.BaseModel)
U = TypeVar("U", bound=pydantic.BaseModel)
I = TypeVar("I", bound=pydantic.BaseModel)

class TaskSingle(iTask, Generic[Q, W, E, R, T, Y, U, I]):
    __slots__ = ()

    def __init__(
        self,
        name: str,
        func_run: Callable[[Q], W]
        | Callable[[List[E]], List[R]]
        | Callable[[T], List[Y]]
        | Callable[[List[U]], I],
        input_type: Type[Q | List[E] | T | List[U]],
        output_type: Type[W | List[R] | List[Y] | I],

I am trying to write construction like this. Is it possible?

my test fail to acknowledge it

i try to allow for input and output pair to be matching in their quantity

single value in callable input, and callable output
means input and output is matching singular too

if they both list, (E,R), were supplied => then both lists

[oblique urchin]

that type seems far too complicated to be comprehensible. You might be able to make it work with a lot of overloads though.

[cunning plover]

that will be a problem of my future self. For now this syntax does not work at all (failes to be comprehended by mypy)

it breaks when i try to add T and List[Y] pair

        func_run: Union[
            Callable[[Q], W],
            Callable[[List[E]], List[R]],
            Callable[
                [T],
                List[Y],
            ],
        ],
        input_type: Type[Q | List[E] | T],
        output_type: Type[W | List[R] | List[Y]],

previous test becomes failing when List to List supplied
func_run=list_to_list

(function) def list_to_list(input: List[Input]) -> List[Output]
Argument "func_run" to "TaskSingle" has incompatible type "Callable[[List[Input]], List[Output]]"; expected "Union[Callable[[<nothing>], <nothing>], Callable[[List[Input]], List[<nothing>]], Callable[[<nothing>], List[<nothing>]]]"  [arg-type]mypy

and output_type=list[Output] yields

(class) list
Built-in mutable sequence.

If no argument is given, the constructor creates a new empty list. The argument must be an iterable if specified.

Argument "output_type" to "TaskSingle" has incompatible type "Type[List[Any]]"; expected "Union[Type[<nothing>], Type[List[<nothing>]]]"  [arg-type]mypy

hmm. perhaps it does not work like wished it would

i thought python will match all possible combinations until it would find satisfying one

among

class TaskSingle(iTask, Generic[Q, W, E, R, T, Y, U, I]):

what if it is not trying all combinations

wait a second. it has good advice though

[trim tangle]

What problem does it aim to solve?

Compared to e.g. typeguard

[eager vessel]

Or why not use pydantic? 🤔

[lone timber]

utype has a more decent way to handle function parsing (https://utype.io/guide/func/) including arguments and return values (for sync, async, generator and async generator functions), and converter function for all types can be registered and override (https://utype.io/guide/type/#register-type-transformer) so that developer can tune his/her preferences of type cast; and we have supported all logical operators (&, |, ^, ~) for complex type conversion (https://utype.io/guide/type/#logical-operations-of-type)

btw, utype is fully compatible with pydantic (and attrs/dataclasses, as shown in https://utype.io/guide/type/#compat-pydantic) since all types can be registered in utype

[eager vessel]

You can always create a dto/model and pass it into a function
Assigning default values to function parameters is a problem though 🤔

@utype.parse
def login(
    username: str = utype.Param(regex='[0-9a-zA-Z]{3,20}'),
    password: str = utype.Param(min_length=6)
):
    ...

login() # Valid call

[lone timber]

Sure, foo: Annotated[str, utype.Param()] will be on the next version

[trim tangle]

@lone timber What if I want to use the same username value across many functions? Do I have to specify this Param every time?

And even if I save this Param into a constant, it will apply this regex on all levels, won't it?

[lone timber]

import utype

class Username(str, utype.Rule):
    regex = '[0-9a-zA-Z]{3,20}'

try:
    Username('@invalid')
except utype.exc.ParseError as e:
    print(e)
    # Constraint: <regex>: '[0-9a-zA-Z]{3,20}' violated

you can just make a new type called Username, and use it to annotate all the username params

[trim tangle]

I still have this question though

what is this library for?

Is it for parsing raw data (like JSON transferred over network)? Is it to help public functions of a library to fail fast?

[trim tangle]

so here, as I understand, the function will still receive a plain str object if it declares a Username argument?

[lone timber]

Yes, utype.Rule is just a constraint validator, str is the output type

[trim tangle]

So if I pass this username into other functions that declare their input as Username, it will check the regex every time 🤔

[lone timber]

Yep

[trim tangle]

same with, say, list[list[list[int]]]

[lone timber]

you can combine the nested type with Rule to enable the parsing ability, say assert (Rule & list[list[list[int]]])('[[["1"]], [[4.5]], []]') == [[[1]], [[4]], []]

[trim tangle]

what I meant is, if a list contains a lot of items it's pretty expensive to check it at every call level

especially given that you've already validated that piece of data

e.g. if you're making a binary search function that works with Sequence[int], its time complexity will change from O(log n) to O(n)

[lone timber]

oh i see, i think in that way, using @utype.apply to apply constraints to an inherited type (https://utype.io/guide/type/#utypeapply-decorator) can be the cure

so that if the constrains of the type is validated and the input is converted to the declared type (like Month instead of int), next time it will directly pass the validation

but the main target of utype is public functions and classes, to convert the unknown input of network-requests/user-inputs, if the parsed data is going through the internal functions (or you can guarantee the types and constraints), it won't need the @utype.parse decorator

[trim tangle]

So my initial reaction is: I think the library is trying to solve very different problems in one package.

1. The first potential problem it solves is parsing unstructured data, like JSON or YAML. For that purpose perhaps it's pretty good. (though similar solutions exist, like pydantic and dataclass_factory)

2. Enforcing constraints on public interface boundaries. Applying the same rules as for parsing loose serialized data won't really work here. For example, it probably makes sense to convert strings into ints if you're accepting some moldy JSON. That has some legitimate use cases, e.g. JavaScript might want to do that to pass around bigints, since a JavaScript number only fits integers below 2**53. But if someone passes a string to my library function that accepts an integer, it's 100% a programming error, and there's something very wrong on the user side. Same with e.g. converting int to list[int] or "5.2" to 5.

It also changes the semantics of many functions. For example, these functions are very different even if you ignore the type coercions:

def f(x: list[int]) -> list[int]:
    x.append(42)
    return x

@utype.parse
def f(x: list[int]) -> list[int]:
    x.append(42)
    return x

This task is generally somewhat problematic because for a lot of stuff, you simply can't check the type early at runtime. For example, if you accept a function or a lazy iterator.
In general, I would just tell the users of a library to get a static type checker like the rest of the world 🙂 it's much more robust, and can catch errors before runtime.

[eager vessel]

Shouldn't library user be responsible for parsing user provided inputs? 🤔

[trim tangle]

For this purpose, and also for enforcing internal contracts, I would use "value objects" that encapsulate the constraint:

class Username:
    def __init__(self, raw: str) -> None:
        if not re.fullmatch(r"[a-zA-Z0-9-]{3,20}", raw):
            raise ValueError("Invalid username")
        self._raw = raw

    def raw(self) -> str:
        return self._raw
``` this way:
1. You don't duplicate the validation (both in the runtime sense and in the conceptual/logical sense)
2. You don't have implicit coercion, and you don't change the semantics of functions
3. You can never forget to enforce the constraint by writing `def foo(username: str)`, because your type checker will yell at you if you try to call `foo` with a `Username` object.
4. You get guarantees for internal functions

[eager vessel]

If library public api would be using utype or similar tool then all inputs would be validated, regardless if it's a user provided input or another library interacting with it

[trim tangle]

wdym

[eager vessel]

For example if I have a function that uses utype:

@utype.parse
def f(x: list[int]) -> list[int]:
    x.append(42)
    return x

If a different library want to consume that public api x would have to be validated again

[lone timber]

i understand your concerns, in fact dealing with user inputs is indeed a complicated task, so the behaviour in the earlier example is just the "default" preferences for some loose validation, utype has provided a parsing Options to tune the strictness of the parsing (https://utype.io/references/options/#transforming-preferences), some shortcut options like no_explicit_cast or no_data_loss can solve the common issues liked you mentioned, most importantly, you can register your own type converter to impose the strictness of your preferences

[trim tangle]

IMO using these coercions by default is a bad idea

Also, how do you configure that for functions?

[lone timber]

@utype.parse also have a options param to accept Options instance

https://utype.io/guide/func/#configure-function-parsing

[lone timber]

emm, i wlll consider make the "strict" behaviour as the default behaviour in the next version

but user can still choose to disable no_explicit_cast or no_data_loss to get the lax validation

[trim tangle]

well... for deserializing data some of that might be a good idea, but not for validating public interfaces 🙂

[upbeat olive]

print("hello")

[mystic harness]

its possible to "overload" a variable type? i have a generic cache class (GenericCache[str, Union[App, List[App]]]) that can cache a Union[App, List[App]]. The thing is: it will only cache a List[App] if the key is a Literal["all"], otherwise the VT will be App

[trim tangle]

can you show the code perhaps?

[mystic harness]

https://paste.pythondiscord.com/ucutekapet

the typechecker doesnt know when the acessed key will be a List, but the program has a very clear rule: if the key is a Literal["all"], then VT (Value Type) will be a list, otherwise, a App instance

[trim tangle]

I would make two methods: get_app(target: str) and get_all_apps()

and then cache "all apps" separately

[mystic harness]

theres no other solution?

wasnt planning on splitting the method in 2

[trim tangle]

You probably can hack something together. But I think it's just easier to use than the special "all" key

If you want to keep it as one method, I'd just add # type: ignore or something like that

[mystic harness]

alr, thanks

[tranquil turtle]

def f(x: Any, t: TypeAlias) -> <instance of T>: ... # where T is a TypeAlias

f(x, int) # int
f(x, 'str') # str
f(x, list[int]) # list[int]
f(x, 'list[int]') # list[int]
``` is this possible?

T = TypeVar('T', bound=TypeAlias)

def f(x: Any, t: type[T]) -> T: ... # doesnt work because mypy has no idea what type[T] means
def f(x: Any, t: T) -> T: ... # work not as it should, it works like T is a typevar with no bound

[trim tangle]

https://pyright-playground.decorator-factory.su/?gzip=H4sIAOEx8WMC_zWMQQrCMBBF93OKT1cJ1BwgoODGEwQ3IlJw0gZsEmIQAx7eaauz-8N7z5c0o7Yc4ogw51QqjrH1cC3zeShEDvv_UJ3rNNGdPbx62w2sdtH54q4auwOcJcgZY4hOKYkcYsUHzyqtVR1_qt7ISRCp9RBar5_CLx4et6WqJv0FbQClzaEAAAA%3D

from typing import Any, TypeVar

T = TypeVar("T")

def f(x: Any, t: type[T]) -> T:
    ...

#

Foo = int | str

def g(x: Any):
    h = f(x, Foo)
    reveal_type(h)

[trim tangle]

ah

[trim tangle]

What about ```py
Foo: TypeAlias = "int | str"

[tranquil turtle]

I am just curious. I don't think it can be used in real code

[tranquil turtle]

yes, but i want to use it as "inline TypeAlias"

[trim tangle]

ah yeah no that won't work

[cerulean latch]

What is typing.AnyStr?

[oblique urchin]

it's a helper for functions that operate on both bytes and str objects in a similar fashion

it's not frequently useful for new code

[cerulean latch]

ah okay

i was trying to import Any from typing and vscode showed AnyStr so i was just wondering, thanks :>

[trim quartz]

so I have a class Foo, an inner class Bar with property x that point to the outer class, in this case Foo

class Foo:
    class Bar:
        x: "Foo"

class Baz(Foo):
    pass

it works, but not when I subclass Foo
is there anyway to automatically make Baz.Bar.x hint to Baz instead of Foo?

[eager vessel]

Can paramspec be used to specify to make function "partial"? 🤔
I want to make all arguments optional but at the same time preserve type information

[trim tangle]

Why do you have a nested class?

[midnight portal]

#! /usr/bin/env python3

import pandas as pd
import csv

df1 = pd.read_csv(r"whitelist", dtype=str)
df2 = pd.read_csv(r"adresser_alle.csv", dtype=str)

# Merge dataframes (EF whitelist + DAWA adresser)
df3 = pd.merge(df1, df2, on = ['geographicAddressId'], how = 'left')
df3.set_index('geographicAddressId', inplace = True)

# Skriv til ny CSV file
df3.to_csv('CSV4_output.csv')

# Åbn 'CSV3_output' og gem items til ny fil
with open('CSV4_output.csv', "r") as a:
    reader1 = csv.DictReader(a)
    for item in reader1:
        print(item['geographicSiteId'], item['wgs84koordinat_bredde'], item['wgs84koordinat_længde'])


a.close()

Problem: File one contains some uppercase letters on the index ID, but file two is lower case. How do I use str.lower to make all ID lower case? 🙂

[trim quartz]

I'm still in the middle of redesign, but these are template classes

class FooSub(Foo):
    class Bar(Foo.Bar):
        async def callback(self):
            # do something different
    
    def create_bar(self):
        return Bar()

the callback design favors subclassing/templates, and these templates are deeply connected to each other that I need to group them somehow

but that's beside the point, as it's not guaranteed to be final
I just want to know the limit of what I can push the type hint system

[gritty crater]

How'd I go about type hinting this: py def maybe_round(num: T) -> T | int: if int(num) == num: return int(num) return numwhere py T = TypeVar('T', bound=float). The return type is what's bothering me, since in both cases, the return value == num, so technically, -> T should be valid, but the data type here would just be wrong.

[trim tangle]

This function will blow up if you pass in None or [1, 2, 3]

oh wait, T is bound to float

[trim tangle]

the return value == num, so technically, -> T should be valid, but the data type here would just be wrong.
🤔 not sure what you mean

[trim tangle]

Can you show a more complete example maybe?

[peak echo]

I think they mean if they pass float, the func should return int or float, if they pass int it returns int

[trim tangle]

that's how it works now, isn't it?

[peak echo]

That actually makes sense except typevar shouldn't be bound to float

Better to restrict it to int and float I think

[trim tangle]

🤔

well then ```py
def maybe_round(num: float) -> float:
if int(num) == num:
return int(num)
return num

[gritty crater]

The intended use of this is to return an int if the float ends with .0, otherwise just return the num

So that where possible, the user would see 23 instead of 23.0, but 23.1 would be left as 23.1

[trim tangle]

then it's a formatting issue, not a numeric issue 🙂

!e

pi = 3.14
not_pi = 42.0
print(f"{pi:g}")
print(f"{not_pi:g}")

[rough sluice]

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

001 | 3.14
002 | 42

[gritty crater]

Ahh, tyvm, forgot about f-string formatting lmao

[trim tangle]

You can do it with .format as well btw

in 99% of the cases where you want round you don't actually need rounding

also it should be in math

[stray summit]

whats the correct way to type a protocol where there ought to be either a readonly property or a attribute of a certain type/base type

[trim tangle]

hm, what do you mean?

[stray summit]

i have protocol i need/want to use where the implementing classes may either have a direct attribute, or a property returning the correct type, i observed trouble with that

[trim tangle]

from typing import Protocol
from dataclasses import dataclass

class HasSound(Protocol):
    @property
    def sound(self) -> str:
        ...

@dataclass(frozen=True)
class Worm:
    sound: str

class Duck:
    @property
    def sound(self) -> str:
        return "quack"

class Dog:
    def __init__(self) -> None:
        self.sound = "woof"

def squeeze(animal: HasSound) -> None:
    print(animal.sound.upper())

squeeze(Duck())
squeeze(Dog())
squeeze(Worm("idk what sound worms make"))

[stray summit]

seems like i need to extract a example, your minimal one works

[trim tangle]

works on my machine 😎

[trim quartz]

it's not that
what I want is the inner property always hinted as the outer class, even on inheritance
kinda like Self type, but on nested classes

[peak echo]

Sorry that was a wrong reply

[cunning plover]

is there a good way to import Typeguard between versions?

if not sys.version.startswith("3.10"):
    from typing_extensions import TypeGuard
else:
    from typing import TypeGuard  # type: ignore

so it would be correctly working between versions for Mypy

[tranquil turtle]

if sys.version>=(3,10)

[tranquil turtle]

Typecheckers understand only this version check

[cunning plover]

Unsupported operand types for >= ("str" and "Tuple[int, int]") [operator]mypy

[tranquil turtle]

.version_info

[cunning plover]

>>> import sys
>>> sys.version
`Unsupported operand types for >= ("str" and "Tuple[int, int]")  [operator]mypy`

[cunning plover]

if sys.version_info >= (3, 10):
    from typing import TypeGuard
else:
    from typing_extensions import TypeGuard

nice. mypy catches that 🙂

[tranquil turtle]

It is documented in pep484

[trim tangle]

nooo xunil my beloved

[tranquil turtle]

swodniw

[soft matrix]

i cant tell if im going a little bit crazy but i have a class ```py

Type is an enum

class ID[TypeT: Type = Type]:
if TYPE_CHECKING:
# this is hardly ideal without HKT but as far as I'm concerned you shouldn't be constructing your own subclasses of ID without specifying TypeT anyway
# so this should work for now
@overload
def new( # type: ignore
cls,
id: Intable,
*,
type: TypeT,
universe: Universe = ...,
instance: Instance = ...,
) -> Self:
...

    @overload
    def __new__(
        cls,
        id: Intable,
        *,
        universe: Universe = ...,
        instance: Instance = ...,
    ) -> ID[Type]:
        ...

def __init__(
    self,
    id: Intable,
    *,
    type: TypeT = ...,
    universe: Universe = ...,
    instance: Instance = ...,
):```

and with it i want to be able to do

ID_ZERO = ID(0, type=Type.Individual)  # type should be ID[Literal[Type.Individual]]
ARBITRARY_ID = ID(1234567890)  # type should be ID[Type]
```what am i doing wrong here?

currently both are ID[Type]

and idk why

[stray summit]

@soft matrix isnt that the draft experimental syntax that might make it to 3.12 but aint in stone yet?

[soft matrix]

yeah its just easier for me to write

the bit that i really dont understand is this

[tranquil turtle]

does mypy and black support this 3.12 syntax?

[soft matrix]

no

[tranquil turtle]

:-(

[soft matrix]

eric 💪

[oblique urchin]

are you running Eric's fork of CPython?

[soft matrix]

no im not actually running this code

i do have the fork though

its nice

[oblique urchin]

I think that's a bug, maybe report it to Eric?

[tranquil turtle]

so you are writing stubs in new syntax?

[soft matrix]

no just a little demo

my actual thing is https://github.com/Gobot1234/steam.py/blob/main/steam/id.py#L225 but thats a lot

[rough sluice]

steam/id.py line 225

class ID(Generic[TypeT], metaclass=abc.ABCMeta):```

[trim tangle]

damn I really hate type hints

I wish they never existed

[tranquil turtle]

i love typehints but hate typecheckers because they complain a lot

[trim tangle]

I kinda think they give you a false sense of security/soundness... And Python hasn't gotten as much awesome tooling static languages have

But they also prevent you from using previously perfectly fine python features and libraries

[soft matrix]

all it takes is one bad type: ignore and your code blows up

[oblique urchin]

@soft matrix do you understand what https://github.com/python/typing_extensions/issues/118 might be about?

sounds to me like they found a mypy bug

[trim tangle]

...and the type checkers themselves are more complicated and error-prone than actual compilers for statically typed languages

so you kinda get the worst of static languages and the worst of dynamic languages

[soft matrix]

yeah this looks like a mypy bug to me

seeing Self? is generally a sign bad things are happening

although i cant really speak much on the implementation in mypy as my solution was never merged and ivan(???) i think ended up implementing it in a completely different way

[oblique urchin]

yes, that's right. Thanks, I closed the issue and redirected them to mypy

[soft matrix]

gosh that reminded me i need to backport typing.Protocol

[green gale]

i don't think this is necessarily a fault of the type checkers

like yes, they are less good, but python is also really hard to statically analyze

statically typed languages are built with good static analysis in mind. python wasn't.

[trim tangle]

indeed, so why put a square peg into a round hole?

[green gale]

because it's better than the alternative

which is almost no static analysis

the number of bugs that are avoided by what static analysis tools do isn't really something you can discount

[trim tangle]

Is there any research on this?

specifically for python

[green gale]

well, no, i suppose not (that i'm aware of). i'm speaking mostly from personal experience.

[trim tangle]

exactly

the benefit is all anecdotal

[green gale]

and this seems to be an experience shared by tons of python developers, given the fact that we use them so widely

[green gale]

and yet there's so much of it. i don't disagree that it could be better, but i don't think what we have now is bad.

[trim tangle]

homeopathy has many supporters (as many other placebo-based cures)

So I guess my question is: if static analysis is important for a given project, why pick Python - a language that's very hard to do static analysis on, and where many existing libraries don't play well with static analysis?

Python was widely used even before PEP 484. Though mypy was, of course, still a thing

[green gale]

if static analysis is important for a given project, why pick Python
i don't think this is it. i think we use python because of all the rest (e.g. the libraries), and we do our best to add static analysis to make the best of it.

[trim tangle]

well, what's good about Python for example?

According to the documentation:

Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python’s elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms.

Of course, some degree of static analysis might be helpful for developers. For example, catching typos in variable names or imports.

[green gale]

as mentioned, libraries, fast iteration, reasonable simplicity, etc.

[trim tangle]

libraries
Many existing libraries, like zope, SQLAlchemy, numpy, attrs do not play well with type checkers. They leverage Python's dynamic nature -- isn't that the whole point of dynamic typing, to allow for very flexible constructs? Even standard library constructs like asyncio.gather or dataclasses/namedtuple have to use hacks or be hardcoded into the type checker to work.
fast iterations
Spending extra time on type-annotating already working Python code does not save time. I find myself constantly fighting type checkers instead of them helping me.
simplicity
Type annotations clutter the code. Sometimes you'll also have to work around the more simple solution to satisfy the type checker, and get a more complicated solution instead (simple example: using object() as a sentinel). I have never seen the inverse

re: fast iterations
I do believe that static types might increase development speed, especially on larger projects. But that's the case when the types help you (see Rust, Haskell, PureScript, Elm, etc.) instead of you having to fight them

[green gale]

libraries
i'll agree with you there. it's suboptimal here, but for the libraries where it does work, it's a huge help.
fast iterations
i write annotations as i write code, and it's usually fine, so i can't really make a statement here. in fact, when i don't use type annotations, i have a tendency to get things wrong really easily.
simplicity
yes, they might introduce some complexity, and i think that's fair. personally i don't find i encounter that a lot, but i'm not sure if it's just the way i tend to write code or the types of things i write.

ultimately, i say this as someone who has anecdotally had a great time with them. but, i don't usually use python for massive projects (usually shorter scripting and whatnot), and i don't have a stack of good evidence somewhere.

[trim tangle]

when i don't use type annotations, i have a tendency to get things wrong really easily.
perhaps you've un-learned the skill of writing dynamically annotated code 🙂
and, of course, there are other techniques, like test-driven development (you still need to write tests, right?)

[green gale]

i do use statically typed languages for larger things mostly, so some things carry over. i do occasionally find myself wanting python's type hints to be more expressive, so i understand what you mean here.

[trim tangle]

I've seen too much Mapping[str, Any] in my short life 💀

[soft matrix]

    async def manifests(
        self,
        ...
        passwords: Mapping[
            AppT, str | None
        ] = {},  # workaround for no covariant in the key mapping. Shouldn't fall victim to https://github.com/python/typing/pull/273
        password_hashes: Mapping[AppT, str] = {},
    ) -> AsyncGenerator[Manifest, None]:```i like it where there are undocumented work arounds for things like this

AppT is TypeVar("AppT", bound=App)

[trim tangle]

peak typing 😎

[hallow flint]

my goal when writing code is to only make complicated mistakes. type hints let me avoid simple mistakes. if you're spending too much time fighting the type checker, lower your definition of complicated 🙂

[trim tangle]

I dongetcha

[plain dock]

does python's type hint system - or any language's type system, for that matter - support labelling functions as to their functional class with respect to the domain and codomain/image: e.g. bijective, injective, surjective, involutory, etc?

[soft matrix]

is this haskel speak?

[trim tangle]

in Agda and some other dependently typed languages you can provide a proof that a function is e.g. bijective

https://agda.github.io/agda-stdlib/Function.Bijection.html

[soft matrix]

oh its a fancy word for one to one

[trim tangle]

don't ask me what this does cause I have no fucking clue

😂

[soft matrix]

how can you prove that?

[trim tangle]

obviously, without K

[plain dock]

it depends on how you define one-to-one, but mostly yes: wiki has a grid on this page: https://en.wikipedia.org/wiki/Bijection,_injection_and_surjection

[trim tangle]

1. Prove that the function is surjective
2. Prove that the function is injective
   Hence, the function is bijective

[plain dock]

involutory means it's self-inverting, like how -(-x) is just x for the "function" -

[trim tangle]

or e.g. prove that there's another function which is the inverse of the original

[trim tangle]

Oh actually this module is kinda deprecated.
See this: https://agda.github.io/agda-stdlib/Function.Definitions.html

Yeah this definitely makes more sense

Congruent : (A → B) → Set (a ⊔ ℓ₁ ⊔ ℓ₂)
Congruent f = ∀ {x y} → x ≈₁ y →  f x ≈₂ f y

Injective : (A → B) → Set (a ⊔ ℓ₁ ⊔ ℓ₂)
Injective f = ∀ {x y} → f x ≈₂ f y → x ≈₁ y

Bijective : (A → B) → Set (a ⊔ b ⊔ ℓ₁ ⊔ ℓ₂)
Bijective f = Injective f × Surjective f

especially if you ignore the type signatures

[slender timber]

the mypy docs don't mention --enable-incomplete-feature this command line option at all

# in some class,
_NOTE_NAMES = ("C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B")
@property
def key(self) -> str:
    """Note name with octave, for e.g. 'C5' or 'A#3' ranging from C0 to B10."""
    return self._NOTE_NAMES[self["key"] % 12] + str(self["key"] // 12)

pylance says return type is unknown

tf is wrong with pylance these days

[soft matrix]

try restarting the lsp

[soft matrix]

i also wonder why 🤔 🤔 🤔

(i dont think pointing people to incomplete features is a great idea)

[plain dock]

it seems they now prefer enumerating the features you want: https://github.com/python/mypy/blob/master/mypy/main.py#L1310-L1315
of which there are currently 2: https://github.com/python/mypy/blob/master/mypy/options.py#L71-L73

[rough sluice]

mypy/main.py lines 1310 to 1315

if options.enable_incomplete_features:
    print(
        "Warning: --enable-incomplete-features is deprecated, use"
        " --enable-incomplete-feature=FEATURE instead"
    )
    options.enable_incomplete_feature = list(INCOMPLETE_FEATURES)```
`mypy/options.py` lines 71 to 73
```py
TYPE_VAR_TUPLE: Final = "TypeVarTuple"
UNPACK: Final = "Unpack"
INCOMPLETE_FEATURES: Final = frozenset((TYPE_VAR_TUPLE, UNPACK))```

[slender timber]

i need Unpack thats all

[grave fjord]

is there a way to forbid untyped function definitions in a typed function definition?

[dull lance]

I think mypy checks against that specifically

[tranquil turtle]

from mypy --help: ```
Untyped definitions and calls:
Configure how untyped definitions and calls are handled. Note: by default, mypy ignores any untyped function
definitions and assumes any calls to such functions have a return type of 'Any'.

--disallow-untyped-calls Disallow calling functions without type annotations from functions with type annotations
(inverse: --allow-untyped-calls)
--disallow-untyped-defs Disallow defining functions without type annotations or with incomplete type annotations
(inverse: --allow-untyped-defs)
--disallow-incomplete-defs
Disallow defining functions with incomplete type annotations (inverse: --allow-incomplete-
defs)
--check-untyped-defs Type check the interior of functions without type annotations (inverse: --no-check-
untyped-defs)
--disallow-untyped-decorators
Disallow decorating typed functions with untyped decorators (inverse: --allow-untyped-
decorators)

mypy -V
mypy 1.0.0+dev.154fee110f274fe6214eff856b65d437ee299fdb (compiled: no)

[jade viper]

Hey guys, do you know if Django QuerySet support subobject type hints? As in something like: response_qs: QuerySet[UserAction]

Found this: https://pypi.org/project/django-hint/#:~:text=Django_hint is a module to,and with pylint in VSCode which doesn't make me hopeful lol

[eager vessel]

Django doesn't use type hints

There's a django-stubs package but there are still problems with querysets IIRC

Also not sure what you mean by a "subobject" 🤔

[analog fable]

django-stubs has worked fairly well for me

[cerulean latch]

i dont really know what this pylance errors means?

    async def get_user(self, user: Union[str, int]) -> Dict[str, Any]:
        headers = await self._make_headers()
        resp = await self._request("GET", self.API_URL + f"/users/{user}", headers=headers)

        return await resp.json()
``` idk much about type hinting or type checkers or anything like that so im quite confused, this is my get_user

[oblique urchin]

it doesn't like that you're changing the type of value placed in user

[soft matrix]

you should probably call the param id?

or name or something

[oblique urchin]

it's initially declared as str | int, so you get an error when you assign a value of a different type to it

[cerulean latch]

OH YEAH

im stupid, the name of the param was conflicting with the name of the variable

[soft matrix]

yeah

thats what jelle is saying

[cerulean latch]

ah

well thank you both! (this code is a few months old honestly and i havent touched it since September but i wanted to try to make sure everything was type checker friendly) so i appreciate this

[reef yacht]

you should really consider renaming the parameter user there as something else. It's clearly not a user, because if you had a user, you wouldn't need to get a user. Or the function name is incorrect... or both honestly.

The type checker can only help you so much, because it doesn't understand the names are wrong. Just making the type checker happy won't make your code readable.

[cedar sundial]

I'm working with numpy in a project and I am slightly confused about the types if anyone has any insight. I'm following this type signature (https://numpy.org/devdocs/reference/typing.html#numpy.typing.NDArray) and I can't figure out if Any can be substituted with any other type as the first type parameter

[chrome hinge]

The typing.Any at the start of the definition is, I believe, the shape, and can't be specified.

(typehinting the shape isn't supported yet, though they are working on it - supposedly the new 3.11 typing features make it possible)

[mystic gazelle]

Protocol is messed up down 31 to 39

[soft matrix]

wdym?

[trim tangle]

? ? ?

[cedar sundial]

Thanks, that makes a bit more sense

[soft matrix]

@trim tangle pyright playground is down :(

[trim tangle]

thanks, should be fixed now

[soft matrix]

thank you

[heady flicker]

A question: In typing.get_origin, we have the following checks.

How would you typehint the "if tp is Generic: return Generic" part?

[soft matrix]

you cant 🤔

oh unless (type[Generic]) -> type[Generic] works

[heady flicker]

I'd assume that type[Generic] would allow subclasses of Generic which is bad

[soft matrix]

i dont think it does cause Generic is erased from bases at type time

[heady flicker]

Yeah, it does. Sadly, pyright doesn't even allow typehinting args as Generics.

So no luck for me.

[cunning plover]

Hey. I want to type that expected value is class instance that inherits str, and Enum at the same time

class ActionType(str, Enum):
  banana = "banana"
  kivi = "kivi"

how is it possible to do?

(this type of value, inhering those two things is expected by pydantic models as Choice field 🙂 )

[lunar wharf]

is StrEnum kinda what you want?

[cunning plover]

yeah.

[cunning plover]

class Something:
  abc: StrEnum = ActionType.banana

to be accepted as correct by Mypy

[soft matrix]

you should either use Final or a literal here to preserve the type

oh actually it might use bidirectional inference to just do the literal for you

[cunning plover]

i mean, i want any type subclassing (str, Enum) to satisfy
like example above: ActionType subclasses str, Enum, which means it is valid to this type T. class Asdfgh(str, Enum) should fit this type T too.
Gdfgfg(Enum) should fail, and Werwer(str) should fail. and Klklk should fail.

[soft matrix]

oh i thought you were using this in a constant setting

[hallow flint]

static type checking of code that does runtime type checking is currently an uphill battle

[trim tangle]

against a vertical wall

[cunning plover]

And I already started to conquer Java mountain for this reason 😁

No point to fight battle you can't win

[slender timber]

why pyright why?

[soft matrix]

restart lsp?

[trim tangle]

I think the inferred right hand side type always takes precedence over the explicit type

which... sounds wrong but yeah it's probably "as designed"

[slender timber]

it happens everytime

[soft matrix]

i think there should be bidirectional inference for unknowns right?

[slender timber]

its only pyright btw

[soft matrix]

thats bidirectional inference the other way

im 99% sure this is a bug

it is joining Literal[42] and int | str right?

that makes sense to be Literal[42]

but joining something with unknowns it should surely take presendence from the more specific type

[trim tangle]

🤔

Yeah that does sound wrong

[trim tangle]

I think it doesn't make sense. Why would a type checker override my very explicit type?

[fading temple]

It is worth it to import a class from other file just for typehint the return type of a function?

[soft matrix]

thats a good question

[trim tangle]

yes

[soft matrix]

i can see arguments for both but i just use cast where i have to fight the inference

ive given up caring 😎

[trim tangle]

in fact @soft matrix it leads to hairballs like this

[fading temple]

okay

[soft matrix]

i mean that code runs fine ;)

but yeah i see your point

oh wait

[trim tangle]

set_to_string("foo")
add()

[soft matrix]

yeah

i thought you meant to call them inside the function

[trim tangle]

I'd classify this as a strong false negative but I think that will be classified as "as designed" so I just skip the reporting phase and transition to the coping phase 😂

[glass lily]

from typing import ParamSpec
from typing import TypeVar
from typing import Generic

P = ParamSpec('P')
R =   TypeVar('R')

class VisitorBase(Generic[P, R]):
  
    def visit_generic(self, node: 'Node', *args: P.args, **kwargs: P.kwargs) -> R:
        ...

    def visit_a(self, node: 'A', *args: P.args, **kwargs: P.kwargs) -> R:
        return self.visit_generic(self, node, *args, **kwargs)

    def visit_b(self, node: 'B', *args: P.args, **kwargs: P.kwargs) -> R:
        return self.visit_generic(self, node, *args, **kwargs)

    def visit_c(self, node: 'C', *args: P.args, **kwargs: P.kwargs) -> R:
        return self.visit_generic(self, node, *args, **kwargs)

class VisitorConrete(VisitorBase[int, str]):
    
    def visit_generic(self, node: 'Node', some_int: int) -> str:
        return str(some_int)

    def visit_a(self, node: 'A', some_int: int) -> str:
        return str(some_int + 1)

    def visit_b(self, node: 'B', some_int: int) -> str:
        return str(some_int + 2)

    def visit_c(self, node: 'C', some_int: int) -> str:
        return str(some_int + 3)

class Node():
    
    def visit(self, visitor: VisitorBase[P, R], *args: P.args, **kw: P.kwargs) -> R:
        return visitor.visit_generic(self, *args, **kwargs)

class A():
    
    def visit(self, visitor: VisitorBase[P, R], *args: P.args, **kw: P.kwargs) -> R:
        return visitor.visit_a(self, *args, **kwargs)

class B():
    
    def visit(self, visitor: VisitorBase[P, R], *args: P.args, **kw: P.kwargs) -> R:
        return visitor.visit_b(self, *args, **kwargs)

class C():
    
    def visit(self, visitor: VisitorBase[P, R], *args: P.args, **kw: P.kwargs) -> R:
        return visitor.visit_c(self, *args, **kwargs)

What's the right way to type-hint the visitor pattern with an abstract base, and arbitrarily potential arguments and return on base classes?

[proud wharf]

If I have an abstract class with an __init__ function like this: py @abc.abstractmethod def __init__(self, name_list: Union[ Union[Sequence[Any], AbstractSet[Any]], Reversible[Any] ]) -> None:

does it makes sense for other sub classes that inherit from it to type-hint like this: py def __init__(self, name_list: Union[Sequence[Any], AbstractSet[Any]]) -> None: self.name_list: Iterator[Any] = iter( random.sample(name_list, len(name_list)) )
and py def __init__(self, name_list: Reversible[Any]) -> None: self.name_list: Iterator[Any] = reversed(name_list) I'm using mypy and its giving me all these suggestions but this is the one that it doesn't say anything about

it's my first time type-hinting

I don't know if it makes sense to have the abstract class' __init__ function to be type hinted like that and have the other classes type-hinted another way??

[trim tangle]

Why even define the __init__ in the abstract class?

[proud wharf]

I'm not sure if I was doing it right

[trim tangle]

Could you show more code?

[proud wharf]

class CARequestNameSorter(abc.ABC):
    @abc.abstractmethod
    def __init__(self, name_list: Union[
            Union[Sequence[Any], AbstractSet[Any]],
            Reversible[Any]
        ]) -> None:
        raise NotImplementedError
    
    @abc.abstractmethod
    def get_next_name(self):
        raise NotImplementedError # this is a base class

class CARequestNameSortRandom(CARequestNameSorter):
    def __init__(self, name_list: Union[Sequence[Any], AbstractSet[Any]]) -> None:
        self.name_list: Iterator[Any] = iter(
            random.sample(name_list, len(name_list))
        )
        
    def get_next_name(self):
        return next(self.name_list)

class CARequestNameSortLastToStart(CARequestNameSorter):
    def __init__(self, name_list: Reversible[Any]) -> None:
        self.name_list: Iterator[Any] = reversed(name_list)
        
    def get_next_name(self):
        return next(self.name_list)``` here

I may have made some mistakes in the design but it should give an idea

[trim tangle]

What's the point of these three classes?

Sounds like you have the same interface as an iterator

[proud wharf]

I'm gonna add more later

[trim tangle]

what for example?

[proud wharf]

matchers and stuff

[trim tangle]

well, right now you don't have those extra stuff

could you show how that would impact the classes?

[proud wharf]

I'm not sure I understand what you mean

[trim tangle]

If you don't have any other methods, I would do: ```py
T = TypeVar("T")

def random_iterator(it: Iterable[T]) -> Iterator[T]:
...
``` and reversed already exists

If you think you will need more stuff later, you can add that stuff later 🙂

[proud wharf]

I can try, I still need to add matching and other things to evaluate the list

each have their own implentation

[trim tangle]

Maybe you could tell more about the problem domain? What are you doing?

[proud wharf]

It's hard to explain, are you familiar with Qt?

[trim tangle]

Nope

Well, I've heard about it 🙂

[proud wharf]

Well there's a class called QNetworkProxyFactory (https://doc.qt.io/qt-5/qnetworkproxyfactory.html#queryProxy) and I need to overload a function of it called queryProxy, and I'm using this abstract class to basically have a way to sort, test, and match the proxy and name list so I can decide which are the right proxies to use for a specific domain or request

The classes I just sent are just sorters for the proxy factory super class to use to sort the proxy and name list

they're going to have more implementation later I'm just trying to get a hang of the type-hinting

[trim tangle]

ah alright

so my answer is: don't put __init__ in the abstract class

[proud wharf]

Sorry my bad I'll remove it

[trim tangle]

Also, you might want to use generics instead of Anys
https://decorator-factory.github.io/typing-tips/tutorials/generics/

[proud wharf]

Thanks I'll read more into it

just getting started with this

[trim tangle]

yeah I understand, there aren't any resources that cover all the stuff you need to know

Maybe you could take a look at the mypy documentation:
https://mypy.readthedocs.io/en/stable/

[proud wharf]

Sure anything I'll need to learn how to do this

lots of very helpful tools out there

[paper salmon]

perhaps one of these approaches could work for you
TypeVarTuple (arbitrary args allowed but no kwargs, also not yet supported by mypy): https://paste.pythondiscord.com/xositudodi
TypeVar (only one arg, but payload can be whatever is needed): https://paste.pythondiscord.com/eyekucugoj

[glass lily]

Ohhh clever

I was thinking of implementing a VisitorContext class as payload

It also occurred to me that you could store the "arguments" inside the visitor, and when you want to "pass in" new arguments you just create a new visitor with a different internal state

That solves the arguments problem, but not the return type problem. That's simple enough to solve with a single typevar in the generic though.

That said, if you're got one argument in the generic, why not two? In theory is should be possible in any language that supports generics/templating

[paper salmon]

isnt that what our examples already show? two generic typevars

[glass lily]

In one form or another, yeah

Their both quite clever! I guess my only question is — and this stems from crippling imposter syndrom

Is there any reason one couldn't use a paramspec instead of a namedtuple or a payload class?

Then just pass arbitrary arguments directly?

[paper salmon]

afaik paramspec is only meant for use with callable types, but you're not passing a callable object around so its usage is invalid

[glass lily]

Derp

Good call

[cerulean pecan]

is it pointless to do ```py
variable: MyType = function()

because the type can be inferred?

[acoustic thicket]

from the typechecking perspective yes, from the readability perspective it depends on you

[cerulean pecan]

what's the point of type hinting if everything can be inferred anyway

[hallow flint]

well if function doesn't have a return type annotation, it won't be inferred

[glass lily]

I would presume because a) not everything can be inferred, and b) readability counts

It counts for far, far more than anything else as far as I'm concerned

[cerulean pecan]

thanks

that makes sense

[glass lily]

I suppose a third reason might be that inference may only be a half measure

[slender timber]

ok i am having a weird issue with mypy in my ci

only python 3.7 shows an error, python 3.8+ dont

https://github.com/demberto/PyFLP/actions/runs/4274608056/jobs/7441344707

i only removed this in the previous commit in hope that it will be finally solved but meh

type checking is increasingly becoming the only pita i have

[grave fjord]

What version of mypy are you using? https://mypy-play.net/?mypy=latest&python=3.7&gist=9f717993daa010c30ab5569315c2bc18

[slender timber]

1.0.1

[grave fjord]

Weird

[slender timber]

yea and i checked locally it happens w/ 3.7 too

https://github.com/timrid/construct-typing/blob/acc3fa344396eeb986859e94a1e761eff67c9a1c/construct-stubs/core.pyi#L200-L202
SimpleAdapter in my code is based off of this

[rough sluice]

construct-stubs/core.pyi lines 200 to 202

class Adapter(
    Subconstruct[SubconParsedType, SubconBuildTypes, ParsedType, BuildTypes],
):```

[slender timber]

https://github.com/timrid/construct-typing/blob/acc3fa344396eeb986859e94a1e761eff67c9a1c/construct-stubs/core.pyi#L168-L174

[rough sluice]

construct-stubs/core.pyi lines 168 to 174

SubconParsedType = t.TypeVar("SubconParsedType", covariant=True)
SubconBuildTypes = t.TypeVar("SubconBuildTypes", contravariant=True)

class Subconstruct(
    t.Generic[SubconParsedType, SubconBuildTypes, ParsedType, BuildTypes],
    Construct[ParsedType, BuildTypes],
):```

[slender timber]

https://github.com/timrid/construct-typing/blob/acc3fa344396eeb986859e94a1e761eff67c9a1c/construct-stubs/core.pyi#L95-L98

[rough sluice]

construct-stubs/core.pyi lines 95 to 98

ParsedType = t.TypeVar("ParsedType", covariant=True)
BuildTypes = t.TypeVar("BuildTypes", contravariant=True)

class Construct(t.Generic[ParsedType, BuildTypes]):```

[grave fjord]

But you can't actually run Adapter[T, T, U, U]

[slender timber]

wdym

oh yea

we cant

this (bug?) was introduced in mypy 0.991

typealias vs subclass thing

[hallow flint]

if you can narrow it down, file a mypy issue and tag me

[cedar sphinx]

hi how do i type hint the return two variables seperated by a comma in a function signature?

e.g. def test() -> int , int:

[trim tangle]

def test() -> tuple[int, int]:

When you do return a, b you're really returning a tuple

!e

def test():
    return 1, 2

print(test())

[rough sluice]

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

(1, 2)

[cedar sphinx]

@trim tangle thx

[cunning plover]

function_type = Callable[[Any], None]
def action(
    _func=Optional[function_type],
    *args: List[Any],
    name: str = "",
    asynced: bool = False,
    capacities: Optional[List[enum.Enum]] = None,
) -> Any:
    def decorator_repeat(func: function_type) -> Any:
        @functools.wraps(func)
        def wrapper_repeat(*args: Any, **kwargs: Any) -> Any:
            return func(*args, **kwargs)

        return wrapper_repeat

    if _func is None:
        return decorator_repeat

    return decorator_repeat(_func)

trying to type decorator with arguments, but mypy still complains it is not typed Function is missing a type annotation for one or more arguments [no-untyped-def]

what i can be mssing?

a bit complicated to type decorator 😁 (for this reason trying to have at least with Any to get it right first)
as end goal trying to decorate it for

class Asd:
  @flowey_action(name="send_mail", asynced=True, capacities=[Capacity.AllowSendMail])
  def send_mail(self, queryset: QuerySet[DummyObject]) -> None:

class Gfds:
  @flowey_action
  def send_mail(self, queryset: QuerySet[DummyObject]) -> None:

[trim tangle]

On what line is it giving an error?

[acoustic thicket]

you mistyped _func=Optional instead

of _func: Optional

[trim tangle]

oh yea

[cunning plover]

Ops. Thanks