#type-hinting

1 messages · Page 32 of 1

jade viper
#

Oh yes, my bad

#

With the decorator, the type hint should be:

(date: Sequence[DateT]) -> Sequence[DateT]:

trim tangle
#

are you on 3.12?

jade viper
#

Yes

restive rapids
# trim tangle are you on 3.12?

hmh, i think they would need to "apply" Sequence to each arg in the paramspec which is not possible ? maybe it should only work for 1 argument functions?

trim tangle
jade viper
#

And the return

restive rapids
#

you cant do anything with that target in the typehint

trim tangle
jade viper
#

Then you can consider only one target

jade viper
#

Can't do it then? 😦

trim tangle
#

Yeah, you can't match string literals with parameters in this way

#

How many functions do you want to decorate with this?

restive rapids
#

maybe [A, *B, C](Callable[[A, *B], C]) -> Callable[[Sequence[A], *B], Sequence[C]] to apply it to the first arg

trim tangle
#

Yeah, you can probably do that

trim tangle
tranquil ledge
#

Seems like you're looking for a TypeGuard/TypeIs function of some kind. Those constructs allow you to tell the type checker how a custom function might narrow a particular kind of input, but it won't verify your narrowing logic for you.

tacit sparrow
#

And type checkers can't verify that your overloads are correct

Can you show an example? Haven't met any issues with overloads yet. Pyright seems to complain if an overload doesn't fit in the original function signature

trim tangle
tranquil ledge
#

Depending on what spawned the list, where it's passed, etc.

trim tangle
#

The overall signatures make sense, but the actual implementation doesn't follow "str maps to str and None maps to None"

#

When you use overloads, the typechecker trusts you that something like this won't happen

tacit sparrow
tacit sparrow
trim tangle
#

(in my example it's possible to use a MaybeStr = TypeVar("MaybeStr", str, None), but I'm not too sure)

tacit sparrow
#

but if it would type check something like this it would help to spot an error

def double(x: str) -> str:
    if random.random() > 0.5:
        return str(x) + "!"
    else:
        return None
trim tangle
#

This would be incorrect because you're returning str | None while the annotation says you're returning a str

tacit sparrow
#

I mean that in theory it could type check those two for issues and tell the user that both overloads are wrong.
Though no idea how it would work in terms of UI 😄

def double(x: str) -> str:
    if random.random() > 0.5:
        return str(x) + "!"
    else:
        return None

def double(x: None) -> None:
    if random.random() > 0.5:
        return str(x) + "!"
    else:
        return None
trim tangle
#

in this case you can just replace it with a TypeVar as I mentioned

tacit sparrow
#

sorry, still don't understand why type checker wouldn't ensure that running implementation with any of the available overloads wouldn't produce type errors.

trim tangle
rough sluiceBOT
#

python/pyspark/sql/classic/dataframe.py lines 509 to 519

@overload
def repartition(self, numPartitions: int, *cols: "ColumnOrName") -> ParentDataFrame:
    ...

@overload
def repartition(self, *cols: "ColumnOrName") -> ParentDataFrame:
    ...

def repartition(  # type: ignore[misc]
    self, numPartitions: Union[int, "ColumnOrName"], *cols: "ColumnOrName"
) -> ParentDataFrame:```
trim tangle
oblique urchin
#

I wonder if we should use Unpack[] for that

trim tangle
#

Well... this is a loving "abomination". I understand that a lot of Python functions just have complex interfaces

#

and 3.11 adds another argument :(

oblique urchin
#

yeah, with Unpack[] we can at least avoid repeating the long list of parameters that don't affect the return type 🙂

tacit sparrow
#

Can someone please test this one in Python 3.12, does numpy array have a typing error for you too?

from typing_extensions import Buffer

class ClassA:
    def accept_buffer2(self, buffer: Buffer):
        pass

a = ClassA()

from array import array
import numpy as np
array_buffer = array("l", [1, 2, 3, 4, 5])
np_buffer = np.array([1, 2, 3, 4, 5], dtype=np.int32)
print(memoryview(array_buffer))
# typing issue
print(memoryview(np_buffer))

a.accept_buffer2(array_buffer)
# Argument of type "NDArray[signedinteger[_32Bit]]" cannot be assigned to parameter "buffer" of type "Buffer" in function "accept_buffer2"
#   "ndarray[Any, dtype[signedinteger[_32Bit]]]" is incompatible with protocol "Buffer"
#     "__buffer__" is not present
a.accept_buffer2(np_buffer)
tacit sparrow
#

okay, I've found it, on 3.12 it should work

rough sluiceBOT
#

numpy/__init__.pyi lines 1462 to 1463

if sys.version_info >= (3, 12):
    def __buffer__(self, flags: int, /) -> memoryview: ...```
blazing dove
#

Are there any good convenience libraries that would allow the syntax require(foo).x.y such that if x could nominally be Bar | None it will be assumed to be non-null in the typing, such that .x evaluates to Bar and an exception is thrown during runtime if it is not?

tranquil turtle
#

should not be hard to make it yourself

undone saffron
#

I don't think you can make that yourself in a generic fashion

#

problem with it is you can't bind a type variable to "not none" and you cant negate None directly either

blazing dove
#

Yeah, it seems like it could be done in runtime fine, just not with correct typing

restive rapids
blazing dove
#

I guess it would be more of a hypothetical macro, where the only thing that happened was that it was asserted that the segments parts would not be none

undone saffron
#

formally? It's a projection of the attributes of the input type when the input type has non-None attributes, and an error for any attributes which are None, which would require either HKTs and type negation, or algebraic effects to describe

blazing dove
#

Seems like it would need to be a language feature maybe

undone saffron
#

This kind of structure is often better handled by defining the non-null structure you want (often with something like msgspec or pydantic because this is usually incoming structured data being turned into a python object) and attempting to validate that the original object has the attributes you want with the types you expect

#

then after that validation, just use it

blazing dove
#

In this case the data is already modeled in pydantic, but later there are cases where I know that certain paths must be available - so having the ability to do something like foo.x!.y or foo.x?.y would be very convenient

restive rapids
restive rapids
rough sluiceBOT
undone saffron
#

505 would be a nice solution for this kind of problem too

#

at the language level, you just know the resulting type might have short circuited to None

blazing dove
#

Thanks! Yeah, that's the one. Maybe another time 🙂

restive rapids
trim tangle
undone saffron
trim tangle
#

Yeah I mean from the runtime perspective. You get a None... why?

summer hinge
#

it's hard to understand which one of them is actually none

undone saffron
#

the whole point of this is you don't actually care which was none if you're using that

#

and presumably any good use of that is followed by handling None

# 
x = foo?.bar?.baz?.quack or default_x()
summer hinge
#

yep, it seems like the whole point of pep 505. wanna handle every is not None mannualy? just don't use ? then.

undone saffron
#

probably not actually using or but split across lines, but point is sometimes you just want the nested object if it exists, otherwise do something else

blazing dove
#

You could also have foo!.bar!.baz which would raise an exception on None so you could identify during runtime

oblique urchin
undone saffron
#

That works for if you want to ensure foo isnt None (And breaks in the presence of Any as you said), but not for wrapping all of the attributes of foo

stiff acorn
#

is there a way to type something like this?

# If limit is 1 return a single object
def something(limit: int = 1) -> str:


# If limit is >1 return a single object
def something(limit: int = 2) -> list[int]:


# Implementation
def something(limit: int = 2) -> int | list[int]:

    assert limit >= 1

    if limit == 1:
        return "a"
    else:
        return ["a"]*limit
trim tangle
#

You could theoretically do this:

from typing import overload, Literal

@overload
def something(limit: Literal[1]) -> str: ...

@overload
def something(limit: Literal[2, 3, 4, 5, 6] = ...) -> list[str]: ...

@overload
def something(limit: int) -> str | list[str]: ...

def something(limit: int = 2) -> str | list[str]:
    # implementation
``` but I would change the interface of this function. How it would change depends on how you're planning to use it
stiff acorn
#

it's a function that queries a site and returns all the results with a limit parameter to avoid http 429

rare scarab
#

APIs that auto unpack single length lists are the worst

trim tangle
#

Yep

stiff acorn
#

thanks, i'll do just that then

trim tangle
#

Or just have one function and always return a list, even if it has 0 or 1 elements

#

If you know that you want one thing, you can do [thing] = get_things(1)

rare scarab
#

You can always do this anyway. ```py
result, = something(limit=1)

#

^ will fail if it returns an empty list

stiff acorn
#

I wasn't certain how I want to make it but i guess that does give me some idea

#

thank you once again, appreciate the help

frank field
#

is there any way to indicate the range of a variable (i.e say you have an integer input that has to be between 0 and 255) with type hinting or anything similar to it?

#

i assume not

split mist
#

send me a site to do beginner programs and exercices

oblique urchin
frank field
#

interesting

#

honestly ive started type hinting mainly just for the little highlight that IDEs have, i dont think ive used type checkers at all

#

does this seem alright for typehinting btw?

def rgb_to_ycbcr(r:float,g:float,b:float) -> tuple[float,float,float]:
    """
    Converts a RGB color to a JPEG Y'CbCr color.
    https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion
    """
    y = (0.299*r) + (0.587*g) + (0.114*b)
    cb = 128 - (0.168736*r) - (0.331264*g) + (0.5*b)
    cr = 128 + (0.5*r) - (0.418688*g) - (0.081312*b)
    return y,cb,cr

def ycbcr_to_rgb(y:float,cb:float,cr:float) -> tuple[float,float,float]:
    """
    Converts a JPEG Y'CbCr color to a RGB color. 
    https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion
    """
    r = y + 1.402 * (cr-128)
    g = y - 0.344136 * (cb-128) - 0.714136 * (cr-128)
    b = y + 1.772 * (cb-128)
    return r,g,b
#

i guess i could hint the internal variables

oblique urchin
frank field
#

kk

chrome hinge
#

If you wanted to be fancy, you could do

RGBColor = NewType("RGBColor", tuple[float,float,float])
YCbCrColor = NewType("YCbCrColor", tuple[float,float,float])

def rgb_to_ycbcr(rgb: RGBColor) -> YCbCrColor:
frank field
#

oh neat

#

at some point i might just make a proper module for color conversions and have it all in classes but that's a neat one

restive rapids
tranquil turtle
#

i dont think it will give you any safety
it probably will cause more problems

tranquil ledge
#

It might work if you have a runtime type-checker actively validating input against the type annotations, e.g. typeguard or beartype or whatever, but yeah, a regular type-checker using just static analysis won't do anything with that.

covert ocean
#

tuple

viscid spire
covert ocean
#

oh so like they cant be changed what not?

dull lance
#

!e

a = (1, 2)
a[0] = 3
rough sluiceBOT
# dull lance !e ```py a = (1, 2) a[0] = 3 ```

:x: Your 3.12 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 2, in <module>
003 |     a[0] = 3
004 |     ~^^^
005 | TypeError: 'tuple' object does not support item assignment
dull lance
#

!e

a = (1, 2)
del a[0]
rough sluiceBOT
# dull lance !e ```py a = (1, 2) del a[0] ```

:x: Your 3.12 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 2, in <module>
003 |     del a[0]
004 |         ~^^^
005 | TypeError: 'tuple' object doesn't support item deletion
dull lance
#

!e

a_tuple = b_tuple = (1, 2)
a_tuple += (3, 4)
print("Tuples:", a_tuple, b_tuple)

a_list = b_list = [1, 2]
a_list += [3, 4]
print("Lists:", a_list, b_list)
rough sluiceBOT
tacit sparrow
#

Is there a way to make it work without defining TEST_VALUES = ("A", "B", "C") explicitly?

from typing import get_args, Literal, assert_type

TEST = Literal["A", "B", "C"]
TEST_VALUES = get_args(TEST)
# this would fix it
# TEST_VALUES = ("A", "B", "C")

def literal_compatible(a: str):
    if a not in TEST_VALUES:
        raise Exception
    # "assert_type" mismatch: expected "Literal['A', 'B', 'C']" but received "str"
    assert_type(a, TEST)
#

oh, I guess I can just

from typing import get_args, Literal, assert_type

TEST = Literal["A", "B", "C"]
TEST_VALUES: tuple[TEST, ...] = get_args(TEST)

def literal_compatible(a: str):
    if a not in TEST_VALUES:
        raise Exception
    assert_type(a, TEST)
spiral fjord
#

Why is a a str and not a TEST?

undone saffron
restive rapids
#

mhm, a newtype wrapper works better i guess, with a constructor that validates the value

#

i dont know how to do that correctly in python though, in haskell i would make a module which exports the type but not the constructor (they're separate in it, and the constructors that are declared with the type are like dataclasses, just using the thing you gave it), and the constructor that i export would do the validation thingy
maybe just

class UInt64:
  def __init__(self, value: int):
    if fits_in_u64(value):
      self.value = value
    else:
      raise ValueError

maybe do it in __new__

undone saffron
#

refinement types aren't really supported in python, and numerics being infinitely sized by default is an intentional design choice that dates back to pep237

#

you could use numpy's numeric types if you needed sizedness though

undone saffron
restive rapids
#

make UInt64.add_{wrap, saturate, error} obviously
i think thats the rust approach

undone saffron
#

Could also have different derived types with differing behavior there

#

there's also widening as an option, which zig allows (As long as the assignment is to a type it could fit in)

muted iron
#

Why is this not allowed? It works at runtime:

class Movie(TypedDict):
    name: str
    year: int

Movie([("name", "idk"), ("year", 2024)]) # misc - Expected keyword arguments, {...}, or dict(...) in TypedDict constructor
oblique urchin
#

Movie(name="idk", year=2024) should work though

muted iron
#

Yeah, so does passing a dict. I was just wondering if it's reserved for a future thing or something, or if I'm missing something obvious.

oblique urchin
#

I guess since you have to pass all the keys as literals there isn't much reason to use the pairs-of-tuples approach

thick stream
#

Why does pylint not like you hinting with functools.namedtuple but is fine with collections.namedtuple?
Non-working:

from functools import namedtuple
FileCoverage = namedtuple('FileCoverage', ['filename', 'vote', 'coverage', 'required_coverage'])
class ParseCoverageMessage:
   ...
    @classmethod
    def _get_warnings(cls, message: str) -> FileCoverage:

Vs collections.namedtuple working.

image.png image.png
oblique urchin
thick stream
rough sluiceBOT
#

Lib/functools.py line 18

from collections import namedtuple```
oblique urchin
hasty phoenix
#

I'd like to have a type marker for serialization, and the best type hint I've found is Runtime = Annotated[T, True] where T is a TypeVar. However in code the test field.type is Runtime is always false (field comes from dataclasses.fields()), and isinstance(field.type, Runtime) fails with TypeError. How may I test runtime that the type is the Runtime type?

hasty phoenix
#

The reason is that with from __future__ import annotations the .type attribute from fields contains str and not the actual type object

#

So now I need to figure out how to translate that str into actual objects

cunning plover
hasty phoenix
#

The problem where not the "container" type Annotated[], but the implications of using from __future__ import annotations which alters how types are stored in dataclasses. I'm simply comparing the string and it works.

tranquil turtle
rough sluiceBOT
#
typing.get\_type\_hints 

typing.get_type_hints(obj, globalns=None, localns=None, include_extras=False)```
Return a dictionary containing type hints for a function, method, module or class object.

This is often the same as `obj.__annotations__`, but this function makes the following changes to the annotations dictionary:
winged palm
#

Hey so I have something like so:

class SomeTestClass:
    TYPE_ATTR: type[SomeBaseClass]

    @classmethod
    @pytest.fixture
    def foo(cls: type['SomeTestClass']) -> SomeBaseClass:
        c = cls.TYPE_ATTR()
        yield c

and I'm trying to figure out how to type hint this fixture such that i knows what the type of cls.TYPE_ATTR is as the return type, is this possible?

near mural
winged palm
#

oh I don't need that, that was a separate problem actually, variable shadowing, but yes I think the bound typevar works. Thanks!

winged palm
restive rapids
winged palm
#

I guess I'm a little confused about what that gets me, when I theoretically need to set the class variable TYPE_ATTR anyway, to create an instance of it in foo

#

I don't think I could just create an instance of T? rather than needing to set that attribute

restive rapids
#

i also suppose you'd want TYPE_ATTR to be a classvar? or it is actually per instance

winged palm
#

its a classvar yeah

#

but its defined in the subclasses

#

this is essentially a baseclass for other pytest test classes

#

and the subclasses will define the TYPE_ATTR so that the foo fixture returns an instance of what they define

restive rapids
winged palm
#

ah interesting

restive rapids
# winged palm this is essentially a baseclass for other pytest test classes

so SomeTestClass is not meant to be used directly too?
i suppose it could have uh.. an abstract class method returning the type then, to play around the classvar limitation
that is,

from abc import ABC, abstractmethod

class SomeBaseClass:
    ...

class SomeTestClass[T: SomeBaseClass](ABC):
    @classmethod
    @abstractmethod
    def TYPE_ATTR(cls) -> type[T]:
        raise NotImplementedError
winged palm
#

correct yeah

#

hmm

rare scarab
restive rapids
#

yeah typecheckers tend to work with that better

winged palm
#

hmm I haven't looked too much into metaclasses before I'll need to do some diving

rare scarab
#

you just create a class class AMeta(type), then make another class with class A(metaclass=AMeta)

#

isinstance(A, AMeta) will be True

winged palm
#

if that changes any answers/advice

#

using python 3.11 too so I don't think I can use some of the syntax you've suggested

winged palm
#

hmm looking this up more, I don't think what I wan't is necessarily possible, but this seems fine for now.

rare scarab
#

It's sad that vscode doesn't fully support the 3.12 syntax yet.

oblique urchin
rare scarab
#

It's not that it fails to parse the new generic syntax. It just ignore it.

#

I read it was from an unmaintained upstream library they use for syntax highlighting.

oblique urchin
#

Oh that might be true. The type checker (pyright) definitely supports the syntax fully

rare scarab
#

Yeah. It's just weird how the generics are the only part not highlihgted

#

microsoft/pylance-release#5824

mighty lindenBOT
rare scarab
#

^ you've commented on it

oblique urchin
#

yeah what you said made me think that the new syntax was ignored in type checking, not syntax highlighting

rare scarab
#

¯_(ツ)_/¯

clear vigil
#

This method works fine with a string as the only argument but it sells my static type checker that it shouldn't be a string. How can I fix it? Its from a package so idk how it works but I just want the type checking to not falsely go off

    def from_conf(self, *args, **kwargs): # real signature unknown
        """
        Construct a sender from a :ref:`configuration string <sender_conf>`.
        
                The additional arguments are used to specify additional parameters
                which are not present in the configuration string.
        
                Note that any parameters already present in the configuration string
                cannot be overridden.
        """
        pass
#

Changing it to

def from_conf(self, *args: Any, **kwargs: Any) -> None:  # real signature unknown

Didnt fix it so I think its getting overridden by something

trim tangle
clear vigil
trim tangle
#

and what typechecker gives you the error?

clear vigil
#

Config.CONF is just a str

clear vigil
trim tangle
# clear vigil

Is the error also coming from pycharm? Or from a mypy or pyright plugin?

clear vigil
trim tangle
#

then it seems like a pycharm bug

clear vigil
#

args / kwargs are so annoying

#

like seriously, the function does nothing but pass, why even write comments if you're going to make me turn into a detective to track down what it actually does

trim tangle
clear vigil
clear vigil
#

So I found the actual function, written in cython:

    @staticmethod
    def from_conf(
            str conf_str,
            *,
            str bind_interface=None,
            str username=None,
            str password=None,
            str token=None,
            str token_x=None,
            str token_y=None,
            object auth_timeout=None,  # default: 15000 milliseconds
            object tls_verify=None,  # default: True
            object tls_ca=None,  # default: TlsCa.WebpkiRoots
            object tls_roots=None,
            object max_buf_size=None,  # 100 * 1024 * 1024 - 100MiB
            object retry_timeout=None,  # default: 10000 milliseconds
            object request_min_throughput=None, # default: 100 * 1024 - 100KiB/s
            object request_timeout=None,
            object auto_flush=None,  # Default True
            object auto_flush_rows=None,  # Default 75000 (HTTP) or 600 (TCP)
            object auto_flush_bytes=None,  # Default off
            object auto_flush_interval=None,  # Default 1000 milliseconds
            object init_buf_size=None,  # 64KiB
            object max_name_len=None):  # 127
tacit sparrow
#

Is there some workaround for bytearray and memoryview popping in when trying to process just bytes?

from typing import IO, Union, Any

def extract(ifc_file: Union[IO[bytes], IO[str]]):
    line = next(ifc_file)
    reveal_type(line) # bytes

    if isinstance(line, bytes):
        line = line.decode("utf-8")
    else:
        # Type of "line" is "bytearray | memoryview | str"
        reveal_type(line)

    reveal_type(line) # str | bytearray | memoryview

    # Cannot access attribute "startswith" for class "memoryview"
    a = line.startswith("test")
tacit sparrow
#

I guess assert isinstance(line, str) will do as a workaround but doesn't looks pretty

acoustic thicket
#

works fine in pyright

tacit sparrow
#

maybe it's some older version problem, I'll check it out

tacit sparrow
#

actually it does fail for pyright test.py too for some reason

# 
>pyright test.py
test.py
  test.py:5:17 - information: Type of "line" is "bytes | str"
  test.py:10:21 - information: Type of "line" is "bytearray | memoryview | str"
  test.py:12:17 - information: Type of "line" is "str | bytearray | memoryview"
  test.py:13:14 - error: Cannot access attribute "startswith" for class "memoryview"
    Attribute "startswith" is unknown (reportAttributeAccessIssue)
  test.py:13:25 - error: Argument of type "Literal['test']" cannot be assigned to parameter "prefix" of type "ReadableBuffer | tuple[ReadableBuffer, ...]" in function "startswith"
    Type "Literal['test']" is incompatible with type "ReadableBuffer | tuple[ReadableBuffer, ...]"
      "Literal['test']" is incompatible with protocol "Buffer"
        "__buffer__" is not present
      "Literal['test']" is incompatible with "tuple[ReadableBuffer, ...]" (reportArgumentType)
2 errors, 0 warnings, 3 informations
acoustic thicket
tacit sparrow
#

🤔

acoustic thicket
#

pylance v2024.6.1

#

type checking mode strict

#

if i set typechecking mode to off then i do get this

image.png
#

doesnt work for off, basic, standard but works with strict

#

no idea how that works lol

tacit sparrow
#

🤨

acoustic thicket
#

can you reproduce

tacit sparrow
#

it works fine in strict mode

#

but breaks in standard for me

acoustic thicket
#

hmm

tacit sparrow
#

no idea why you get those errors with typechecking = off

#

😄

#

actually can reproduce by disabling the "strict" in pyright playground

#

looks like a bug in pyright

acoustic thicket
#

yeah even this look pretty weird

image.png
#

this might be related

image.png
tacit sparrow
acoustic thicket
#

its true in strict

tacit sparrow
#

okay, I guess it's not a bug

trim tangle
#

It's not as bug, it's correctly executing the wrong design 🙂

cinder bone
spiral fjord
#

It does not seem to like the overload, works just fine if you remove tht.

pastel egret
#

You need to use a "callable protocol".

#

If you define a Protocol class with just a __call__ method, that's treated the same as Callable, except this way you can specify keyword only arguments, generics, overloads, things like that.

pastel egret
cinder bone
#

Ah okay thanks :]

cinder bone
# pastel egret ```py from typing import Protocol class MyFunc(Protocol): @overload def ...

So I tried taking this example a little farther and, I crashed mypy xD

from typing import TYPE_CHECKING, Any, TypeVar, Self, Protocol, cast, overload

V = TypeVar("V")

class SimpleAttributeFunc(Protocol[V]):
    # getter
    @overload
    def __call__(self, new: None = None, **kwargs: Any) -> V: ...

    # setter
    @overload
    def __call__(self, new: V, **kwargs: Any) -> Self: ...

class Test:
    @property
    def color(self) -> SimpleAttributeFunc[str]:
        @overload
        def foo(new: None = None, **kwargs: Any) -> str: ...

        @overload
        def foo(new: str, **kwargs: Any) -> Self: ...

        def foo(new: str | None = None, **kwargs):
            if new is None:
                return str(["hi"])
            else:
                print(f"Setting to {new} with {kwargs}")
                return self
        return foo

x: str = Test.color()
y: Test = Test.color("#FF00", recurse=True)

EDIT: I got it working https://gist.github.com/mypy-play/62fad45174312b2b1ec0c17e555d9ece

oblique urchin
cinder bone
oblique urchin
#

oops I missed the expandable section 🙂

#

sorry for that!

cinder bone
#

no worries :) It was a really long stack trace lol - I still had to cut a lot of it out to fit in GitHubs requirements for maximum length

stiff acorn
#

Is there a way for me to make my "own" Iterable type (list/set/tuple) which I can then use it likeMyIterable[int]

#

What I'm thinking is to make my own union of list/set/tuple and then use that for typing various iterables

trim tangle
#

Not sure how it will interact with Pydantic, maybe you need something else entirely

#

(you can't convert something into another type without knowing what that type is, so you'll need to write a converter somehow)

stiff acorn
#

don't need to convert it really, all im doing is for i in x so Iterable would suffice but Iterable is currently broken in pydantic

#

looks like this worked?

from typing import TypeVar, Union, reveal_type

T = TypeVar("T")

Iterable = Union[list[T], set[T], tuple[T, ...]]


def test(i: Iterable[str]) -> None:
    print(reveal_type(i))

test(["a", "b"])
test({"a", "b"})
test(("a", "b"))
#

although vscode added a blue underline so I'm doing something wrong?

image.png
# 
❯ mypy .\test.py --strict
test.py:16: note: Revealed type is "Union[builtins.list[builtins.str], builtins.set[builtins.str], builtins.tuple[builtins.str, ...]]"
Success: no issues found in 1 source file
trim tangle
#

Maybe you want Collection instead of Iterable?

#

Pydantic is a serialization library, it wouldn't make sense to use a lazy iterator inside a pydantic model

#

well, apparently pydantic thinks it does make sense and wraps the result with a lazy validator

#

why is that broken for you?

viscid spire
#

it's just saying there is some information you can inspect there with intellisense

#

when you hover over it it will show you the type in this case

stiff acorn
# trim tangle why is that broken for you?

Pydantic is also validation library, it validates the input based on the type hints. In this case it's turning my collections.abc.Iterable to an iterator hence the bug as acknowledged by pydantic devs in the issue. So I'm trying to replace collections.abc.Iterable with something that still accepts the common iterables and see if pydantic works with that. Basically re-inventing collections.abc.Iterable so I can use it with pydantic

trim tangle
#

If you want something re-usable, you need a Collection, not an Iterable

stiff acorn
#

I wasn't aware of Collection, lemme read up on it

trim tangle
rough sluiceBOT
#
collections.abc

Added in version 3.3: Formerly, this module was part of the collections module.

Source code: Lib/_collections_abc.py

This module provides abstract base classes that can be used to test whether a class provides a particular interface; for example, whether it is hashable or whether it is a mapping.

An issubclass() or isinstance() test for an interface works in one of three ways.

  1. A newly written class can inherit directly from one of the abstract base classes. The class must supply the required abstract methods. The remaining mixin methods come from inheritance and can be overridden if desired. Other methods may be added as needed:
stiff acorn
#

Slightly related, where do the commonly used list/set/tuple fall in? Are they sequences? iterables? collections?

#

I find these a bit confusing so bear with me here

trim tangle
#

So list and tuple are sequences, but set is only a collection

stiff acorn
#

ah that clears somethings up, thank you

#

is it a gurantee that I can do for i in x for every instance of Collection?

trim tangle
stiff acorn
#

ah that makes sense

#

thank you

blazing dove
#

Is there a way to configure Ruff / Pylance to give a clear warning on cases with match foo: case Bar() as bar: where the evaluated type for bar is Never, i.e. when there is some issue with the code such that a match is performed that can never occur?

ashen fable
#

I have a class inheriting dict: py class MyDict(dict): ... and a Typehint defined:py class MyDictTypeHint(TypedDict): key1: int key2: str How do I typehint MyDict to MyDictTypeHint? I tried inheriting from both:```py
class MyDict(dict, MyDictTypehint):
...

undone saffron
restive rapids
ashen fable
blazing dove
dire glen
#

Hey, what should I say here? because i can't put the class name

ENlTS9U.png
cinder bone
dire glen
#

i can't

#

even with S

cinder bone
#

typing.Self is the full form

#

so from typing import Self first

dire glen
#

thank you

cinder bone
#

Inheritance just feels like the wrong approach here.

dire glen
#

Is there a way to make type hint for a db? i already made some kind of tree to represend my sqlite db, it was for another thing, but could I use that to tell my ide what var i am working with ?```py
class DBType(Enum):
TEXT = "TEXT"
JSON = "TEXT" # JSON is stored as text
INTEGER = "INTEGER"
BOOL = "INTEGER" # 0 or 1 but stored as integer
ID = "INTEGER" # ID is stored as integer
DATETIME = "DATETIME"
LIST = "TEXT" # List is stored as text

@dataclass
class Row:
name: str
type: DBType

def __str__(self) -> str:
    return f"{self.name} {self.type.value}"

@dataclass
class Table:
name: str
rows: list[Row]

def __str__(self) -> str:
    return f"CREATE TABLE IF NOT EXISTS {self.name} ({', '.join(str(row) for row in self.rows)[:-2]})"
#

Cause now, i a type Row, but that is not very specific

rare scarab
#

Maybe look at sqlalchemy orm

#

!pip sqlalchemy

rough sluiceBOT
cinder bone
dire glen
#

this looks sick but it will also means i will have to rewrite it

dire glen
cinder bone
#

So in your case you could write like
list[Row[DBType.TEXT]] as a typehint

dire glen
#

Oh okay

#

Nice

#

But why did you called it D ?

cinder bone
#

It was a choice I made

#

I just used the first letter in your enum because I was lazy and didn't feel like thinking

dire glen
#

understandable

#

thanks

shadow jasper
#

does mypy have an api i can use to do type checking manually

trim tangle
shadow jasper
graceful nova
#

how do you hint that the argument should be a subclass?

trim tangle
#

if you mean an actual class and not an instance of a subclass:

class A: ...
class B(A): ...
class C(A): ...

def f(a: type[A]) -> None: ...

f(A)  # ok
f(B)  # ok
f(C)  # ok
graceful nova
trim tangle
#

in class methods, you should not annotate cls

graceful nova
trim tangle
#

just like you don't annotate self in normal methods

#

btw, you don't need to inherit from object

graceful nova
trim tangle
# 
from abc import ABC, abstractmethod

class MyClass(ABC):
    @classmethod
    @abstractmethod
    def sub_method(cls) -> str:
        raise NotImplementedError

    @classmethod
    def foo(cls) -> int:
        print(cls.sub_method())
        return 42
graceful nova
graceful nova
#

from a subclass

trim tangle
#

Yes, that's the point of abstract methods

#

!d abc

rough sluiceBOT
#
abc

Source code: Lib/abc.py

This module provides the infrastructure for defining abstract base classes (ABCs) in Python, as outlined in PEP 3119; see the PEP for why this was added to Python. (See also PEP 3141 and the numbers module regarding a type hierarchy for numbers based on ABCs.)

The collections module has some concrete classes that derive from ABCs; these can, of course, be further derived. In addition, the collections.abc submodule has some ABCs that can be used to test whether a class or instance provides a particular interface, for example, if it is hashable or if it is a mapping.

This module provides the metaclass ABCMeta for defining ABCs and a helper class ABC to alternatively define ABCs through inheritance:

Similar names: aiohttp.doc.abc
graceful nova
# trim tangle You mean override?

I created a parent class called TagAbc, and made an overload for an abstract method (dump) when I create a child of the TagAbc the overload is not shown because the method has been overwritted by the child, how do I keep the sig of the overloaded dump method?

trim tangle
#

Can you show the code?

graceful nova
#

yeah sure

graceful nova
#

I will send a minified version in sec

trim tangle
graceful nova
trim tangle
#

I think you will need to repeat the overload in the child class

#

I think you shouldn't have a single method that's overloaded. Have a single dump method that accepts a file-like object

graceful nova
trim tangle
graceful nova
#

hm

trim tangle
#

Overloads are a pain and you should avoid them if possible

graceful nova
#

I will go drink something and look at this again, I feel frustrated

trim tangle
#

why?

graceful nova
trim tangle
#

ah

#

taking a break does sound like a good option

graceful nova
#

one function to serialize and the other to write to a file (basically: dumps, dump)

# 
    def dump(self, data: io.BytesIO) -> int:
        return data.write(self.dumps())

    def dumps(self) -> bytes:
        return self._type_.pack(self.value)

that's much cleaner now

tacit sparrow
#

Is get_type_hints is the correct way to get annotations if I want to use them at runtime or there is some other caveat?

The problem with __annotations__ that with from __future__ import annotations values of annotations dict become strings instead of types.

from __future__ import annotations
import typing

class A:
    a: str

an = A.__annotations__['a']

# without future annotations:
# <class 'str'> <class 'type'>

# with future annotations:
# str <class 'str'>
print(an, type(an))

# always appears to be
# <class 'str'> <class 'type'>
an = typing.get_type_hints(A)['a']
print(an, type(an))
pastel egret
#

It is yes, since it will handle that for you, also doing a few other convenient things like unwrapping Annotated.

winged palm
#

Q: I have defined this TypeMeta dataclass as a generic for the default parameter. Is there some way to define this such that if I call TypeMeta(size=2) it won't complain that I haven't set a type? I only need to set T when default is set such as TypeMeta[int](default=4). Would making this a frozen dataclass help?

T = TypeVar("T")

@dataclass
class TypeMeta[T]:
    size: int = 1
    default: T | None = None
#

like some kind of default type if [T] isn't set when creating a class

lunar dune
oblique urchin
lunar dune
#

i'm that cool

winged palm
errant plinth
# 
# setup typing for editor autocompletions
from typing import TypeVar
from another_file import class_type1
from another_file import class_type2
# create the types
reference_class_type1 = TypeVar('reference_class_type1', bound=class_type1)
reference_class_type2 = TypeVar('reference_class_type2', bound=class_type2)

class another_type():
    # type these class variables as the class types
    class_type1_instance: reference_class_type1
    class_type2_instance: reference_class_type2
    # now . completion will show the methods of the types as needed
    # eg. when you type "class_type1_instance." at the "." the class methods appear

The above code gives myself editor completions correctly, it is what I want. I'm very new to typing though - is there a way to accomplish this same result with less code. I need the imports because the other classes are in other files, and I need the TypeVar's to bind the reference. Is there any way to combine those two items?

#

And I'm using Visual Studio Code with pylance.

#

And the class instance references for another_type are in the class variable scope, I'm using them to give units a reference to a unit manager. And setting them during initialization wherever needed.

oblique urchin
errant plinth
main cargo
#

I'm using pyusb, which creates attributes based on the "external" information dictated by the USB spec. For example, for USB interfaces, it does this:

_set_attr(
        desc,
        self,
        (
            'bLength',
            'bDescriptorType',
            'bInterfaceNumber',
            'bAlternateSetting',
            # ...

Is there a brief way in pyright, or in general, to say that "yes, the bInterfaceNumber field exists on usb.Interface objects"? So far all I can think of is to use typing.Protocol, but having to do that for every different kind of USB "thing" I use is a lot of boilerplate.

#

(oh, I also know about # pyright: ignore [reportAttributeAccessIssue], but that is perpetually getting broken by black or ruff's reflowing)

main cargo
#

unrelated question: how can I type check a hatch_build.py build hook? it uses build time dependencies, which pip installs into an ephemeral environment and removes

stray summit
#

anyone familiar with using callable protocols for method definitions - unlike Callable, there isn't yet a working hack for knowing the difference between method descriptor invoked or not (i'm hitting https://github.com/python/mypy/issues/16200)

GitHub
Issues · python/mypy

Optional static typing for Python. Contribute to python/mypy development by creating an account on GitHub.

#

correctly specifying the method descriptor is a utter pain

stray summit
#

i arrived at

class HasApplication(Protocol):
    @property
    def application(self) -> Application: ...


R = TypeVar("R")
P = ParamSpec("P")
TM = TypeVar("TM", covariant=True)
HAS_APPLICATION = TypeVar("HAS_APPLICATION", bound=HasApplication, contravariant=True)


class ApplicationEntityMethod(Protocol, Generic[HAS_APPLICATION, P, TM]):
    def __call__(protocol_self, self: HAS_APPLICATION, *k: P.args, **kw: P.kwargs) -> TM: ...

    @overload
    def __get__(
        self, instance: HAS_APPLICATION, owner: type[HAS_APPLICATION]
    ) -> Callable[P, TM]: ...

    @overload
    def __get__(self, instance: None, owner: type[HAS_APPLICATION]) -> Self: ...

    def __get__(
        self, instance: HAS_APPLICATION | None, owner: type[HasApplication]
    ) -> Callable[P, TM] | Self: ...

it was most frustrating to figure that one - i wish there was a builtin

cinder bone
#

how would I typehint something like list[BaseClass] where the list contains subclasses of BaseClass?

#

right now I'm thinking something like

T = TypeVar("T", bound=BaseClass)
class MyC(Generic[T]):
    attr: list[T]

but I'm not sure if that's a good idea nor if it will work

#

Cause in my case saying MyC[something] doesn't really make sense

undone saffron
#

probably like:

list[Base | SubA | SubB | ...]
#

If you just have list[Base], you're not allowed to do anything that would need one of the subclasses

viscid spire
# 
class A: ...


class B:
    def m(self): ...


def f(xs: list[A]):
    for x in xs:
        if isinstance(x, B):
            x.m()

this is fine

undone saffron
#

Yeah, that's fine.

#

the benefit to spelling out the specific expected subclasses sometimes comes up if you would exclude the base that only exists for shared behavior or ADT-like class use

shadow elk
#

don't quite understand the ins and outs of type hinting... what does this mean

image.png
#

H and V are string objects

soft comet
#

can someone help me. i want to make something but dont know what to type to make something in python

shadow elk
oblique urchin
viscid spire
#

because the str class is not subscriptable, which means it would have to be skipping evaluation in some way

grave fjord
shadow elk
#

it's invalid anyways

#

you can't use literals there

cinder bone
#

is there another (equivalent) data structure that's covariant (I'm guessing not)?

#

I guess I could always cast it lemon_thinking

undone saffron
#

fully equivalent? no, if you're already needing to do isinstance or only used shared behavior, you can just do list[Base], and then match out when you need to know specifics

#

collections.abc.Sequence is covariant, but that comes with potential problems by being covariant. If you can use it, it might be appropriate.

cinder bone
#

Well 99% of the time I don't need to know the specifics - I just need to be able to add something to the list
Like

def add(self, *subclasses: Base) -> None:
    self.items: list[Base]
    self.items.extend(subclasses)
cinder bone
undone saffron
#

yeah, that's why it can be covariant

cinder bone
#

or should I just type: ignore these?

undone saffron
#

I'd probably go with list[Base] here. you can see up here #type-hinting message an example of still being able to narrow when needed (you'd probably want case match instead for a large number of classes though)

cinder bone
#

wait how would that even work from a typecheck perspective, shouldn't a list[A] only contain As?

oblique urchin
cinder bone
#

w h a t

#

I thought lists were invariant

oblique urchin
#

yes. that means that a list[Sub] isn't assignable to list[Super]. It does not mean the list can't contain instances of Sub at runtime

cinder bone
#

Ohhh

#

interesting, thanks for clearing that up :)

cinder bone
#

Another question:
I have some code (that I can't really change) like this

self.play(Square().animate.shift(UP))

where .animate returns an _AnimationBuilder that's a proxy for the Square. In order to get autocomplete for the methods of Square, I have the typehint of the property as

@property
def animate(self) -> _AnimationBuilder | Self:  # Self for autocomplete
    ...

However, I only want the self.play call to accept _AnimationBuilders, which now isn't possible because of the union type of animate.
I also want it so that the users IDE would warn them if they do something like

self.play(Square())

so I can't just change the typehint of play to accept Square

Is there anything I can do? If not, what's the way you would suggest approaching typing this?

oblique urchin
cinder bone
oblique urchin
cinder bone
#

thanks ❤️

weak oriole
#

Having some issues with pyright. My editor is just filled with errors that don't affect run time, ie: these errors do not prevent code from running or failing. They're picked up by pyright. One example is a module not being exported. Here's a snippet that would result in that error

# foo.__init__.py
from foo.bar import Baz as baz

Then in another project that uses the package foo

# test.py
import foo

with foo.baz(...): # error here, "baz is not exported from module foo"
    # do stuff

I know that I can get rid of the error by using one of the following

# foo.__init__.py
from foo.bar import Baz

baz = Bazb

or

# foo.__init__.py
from foo.bar import Baz as baz

__all__ = [..., "baz", ...]
paper salmon
# weak oriole Having some issues with pyright. My editor is just filled with errors that don'...

was able to reproduce your reportPrivateImportUsage warning by installing the package into a .venv and deleting the source - looks like the rules for whether an import is considered private is documented here:
https://microsoft.github.io/pyright/#/typed-libraries?id=library-interface

  • Imported symbols are considered private by default. If they use the ... “from X import A as A” (a redundant symbol alias) ... symbol “A” is not private unless the name begins with an underscore.
    ...
  • A module can expose an __all__ symbol at the module level that provides a list of names that are considered part of the interface. ... All symbols included in the __all__ list are considered public even if the other rules above would otherwise indicate that they were private.
    pyright considered it a private member because you renamed the symbol, so you have to add it to __all__ to explicitly make it public
weak oriole
#

The re-export won’t apply for my example. It’s from foo.bar import Baz as baz. Notice the case.

weak oriole
#

I don’t see the renaming referenced in the link though.

paper salmon
weak oriole
#

I just don’t understand why my editor yells at me for all this whereas my teammates see zero errors.

#

They’re using PyCharm. Not sure what checkers / linters etc it uses. I assume it would use Pyright LSP but maybe not?

paper salmon
#

^ @weak oriole i believe pycharm uses an in-house type checker which supposedly isn't well known for its correctness

wild cypress
#

So my group works in an internal module that passes settings from some BaseSettings component.
We run into a case of needing to often transform Enums we define into their string name to do some magic later on.

We're a bit stuck on the proper type signature, and this is where we landed. Can anyone confirm if it's clear and good

from enum import Enum
from typing import Type, TypeVar
_T_Enum = TypeVar("_T_Enum", bound=Enum)
class BaseEnumSettings(BaseSettings, abc.ABC):
    def str_to_enum(self, attr: str, cls: Type[_T_Enum]) -> None:
        """Transform an attribute into an enum value, if it is a string.
        This assumes the attribute exists.
        Args:
            attr: Name of the attribute to update.
            cls: Enum class used to create the enum value.
        """
    ...
#

It feels like overkill, I think it could just be cls: Type[Enum] but my colleague disagrees, and we can't find any good resources with clear opinions

trim tangle
#

The correct way would be ```py
def str_to_enum(self, attr: str, cls: type[Enum]) -> None:

#

I don't know why type checkers don't show an error here, but here they do:


T = TypeVar("T")

def func(x: T) -> None:
# TypeVar "T" appears only once in generic function signature
#   Use "object" instead
    print(x)

class Foo(abc.ABC):
    @abc.abstractmethod
    def bar(self, value: T) -> None:  # same diagnostic
        print(value)
#

The purpose of a typevar is to link two types together:

T = TypeVar("T")

def choose(first: T, second: T) -> T:
    return random.choice((first, second))

If it's used only once in a signature (and it's not one of the type variables in a Generic), that's not right

wild cypress
#

Thank you, this makes a lot of sense tome

viscid spire
trim tangle
grand vault
#

I'm annotating a list of callbacks that are invoked with named arguments (yeah, I know, but I can't change that yet). Mypy complains about Unexpected keyword argument "arg1" [call-arg]. The annotations have to support Python 3.8 (for now). Any tips to how to best fix this? (I'm far from a typing expert.)

restive rapids
grand vault
trim tangle
#

/ is the opposite, it means that self can only be a positional argument

restive rapids
trim tangle
#

ah

grand vault
weak oriole
paper salmon
#

doesnt look like pylance offers a setting for that

weak oriole
#

It looks like pycharm has some deep learning model that handles the auto complete (in addition to dependency stuff and hieristics i'm sure)

#

full line completeion is the name?

#

all 3 pycharm, vscode and vim (my setup at least) are all using pyright for lsp

#

the difference being pycharm adds more completions... I'm actually not sure at all, I turned off the full line completion stuff so just w/e is default then is still adding private modules such as those not exported and _method / __methods

paper salmon
#

ah right, pyright comes with a more basic lsp too

#

hmm, it seems to autocomplete private modules for me on vsc

weak oriole
#

oh it does?

#

oh sry, it does for me too but it's not including that module I listed above... the from foo.bar import Baz as baz, baz is shown as an autocomplete option with PyCharm but not VSCode or Vim.

paper salmon
#

odd, it does for me

image.png
#

well i am using pylance rather than pyright's LSP

weak oriole
#

Is the left in the same directory?

#

When I’m in the same directory as my package it autocompletes the baz. If I’m in a different project that imports the package it won’t see the baz.

urban imp
#

hey chums, i've got a bit of a weird one for you. Is there a way for me to update a class's typehints from an inherited class in such a way that mypy will see them?

More concretely, i've got a class B that inherits from A. For reasons, i do some magic to make sure any of A's methods that are called from an instance of B do not actually call those A methods, just store that method call for later.

However, i've got an issue with MyPy complaining that methods on A which are type-hinted to return A are now actually returning B, so when i try to use B's methods on the returned object, MyPy thinks it's an A not a B. I'll reply with an example.

urban imp
#

is there a way for me to tell mypy that actually it's totally cool for me to call .cool_magic_trick()?

trim tangle
urban imp
#

there are currently about 20 methods that i'd need to update this way

trim tangle
#

If you don't control A, how does it know about B?

urban imp
trim tangle
urban imp
trim tangle
urban imp
#

yeah sorry, i fixed it.

trim tangle
#

If you are certain that the method always returns a B, then maybe add

b = B()
b2 = b.some_fun_func()
assert isinstance(b2, B)
b2.cool_magic_trick()
#

(and contribute a fix to the original library 🙂 )

urban imp
#

the extra wrinkle is that the code i'm writing for B is a library used by people other than myself, and MyPy would complain for all of them as well 😩

#

but yes i will submit a PR to Microsoft to fix their typehints in Playwright, haha

#

thanks for the suggestions and discussion @trim tangle !

restive rapids
trim tangle
#

That is also possible

urban imp
#

a friend is suggesting i generate a stubfile for my B class that just changes the values in there

#

does this sound like a tree worth barking up?

urban imp
urban imp
#

OK, new but slightly related—if i have a class C that inherits from A and B, but i'm overriding the typing for a method that only A has, how do i do that without mypy complaining?

The error mypy gives is override and seems to be complaining that one of the classes C inherits from does not have the method that i'm typing.

oblique urchin
urban imp
#

ah ha, i missed that this was a @property'd method, not a method. Problem fixed!

#

thanks for challenging my assumption @oblique urchin!

simple field
#

is there a generalised way to type hint 'iterable of strings other than str' nowadays?

undone saffron
#

Not a good one.

#

This is the best current option:

type CommonSeqStr = list[str] | tuple[str, ...]
type CommonIterableStr = Generator[str, Any, Any] | dict[str, Any] | set[str] | frozenset[str] | CommonSeqStr

there's ongoing current discussion about this on discourse: https://discuss.python.org/t/removing-sequence-str-as-base-class-of-str/56907

Discussions on Python.org
Removing `Sequence[str]` as base class of `str`

The typeshed builtins.pyi stub indicates that the str class derives from Sequence[str]. This is somewhat consistent with runtime behavior in that Sequence is a registered virtual base class for str (and hence isinstance(str, Sequence) evaluates to True), but it’s also somewhat inconsistent because Sequence is not a real base class of str and doe...

simple field
#

what are some of the bad ones? im still wrapping my head around TypeGuard, but was wondering if it's doable using that? my understanding is that it only works when the type guard is explicitly called inside a function, is that right?

simple field
undone saffron
#

since typeguard/typeis can't result in a denotable type for your user facing api either

simple field
wary pulsar
#

I feel like I read something like this yesterday, but why doesn't pyright like this?

def testfn(arg: list[int | float | bool]) -> None:
    pass

testfn([True, False])
wary pulsar
undone saffron
#

it should allow that because the below is fine:

def testfn(arg: list[int | float | bool]) -> None:
    pass

x: list[int | float | bool] = [True, False]
testfn(x)

but this would require pyright to do more complicated things. (it does do those, see below)

oblique urchin
#

(And pyright shows no errors for it)

wary pulsar
undone saffron
#

I should have checked the example, I took the error at face value and thought for some reason pyright had reduced where it uses bidirectional inference

wary pulsar
oblique urchin
wary pulsar
#

Thanks. Context is I'm working on all the possible ways to provide keys to __getitem__ and __setitem__ for netCDF variables. 🙂 It's kind of awful because you're allowed do something like this for a 3D variable:

var[0, 2.0, "-1"] = 2
muted iron
# 
for _ in range(2):
    v = 1
v  # Unbound | Literal[1]

Shouldn't type checkers know this will always be bound?

trim tangle
atomic idol
#

I want to type hint a param which has these requirements:

  1. param must be subclass of tkinter.Tk
  2. it has a property called app
    How do i do it?
restive rapids
# 
from typing import Protocol
from tkinter import Tk
class Thing(Tk, Protocol):
  app: ... # whatever

maybe?

oblique urchin
oblique urchin
atomic idol
#

so its better to use type: ignore then

trim tangle
atomic idol
#

I am writing chat app where each sub major component of ui require refrence to the app so i just pass app to the child and mypy then says that it do not has attribute called app

#

app is like tkinter.Tk + app property

# 
class App(tkinter.Tk):
  @property
  def app(self) -> Self: ...
trim tangle
#

Maybe you want some base class that inherits from Tk and has an app property, or something like thaT?

atomic idol
#

yah

trim tangle
#

Then the parameter annotation will just be App, or something like that

rose mist
#

Guys, how can I make pylance understand generic types? So, I have a generic container called Collection that basically allows the user to pass either list or ndarray. However, pylance's type checker doesn't interpret list as a Collection.

#

Here's my code for Collection:

from typing import (Iterator, overload, Protocol, TypeVar, Type,
                    TypeAlias, Self, runtime_checkable)


T = TypeVar("T")

@runtime_checkable
class Collection(Protocol[T]): # pragma: no cover
  def __len__(self) -> int:
    ...
  def __iter__(self) -> Iterator[T]:
    ...
  @overload
  def __getitem__(self, idx: int) -> T:
    ...
  @overload
  def __getitem__(self, idx: slice) -> Self:
    ...
  @overload
  def __setitem__(self, idx: int, value: T) -> None:
    ...
  @overload
  def __setitem__(self, idx: slice, value: Self) -> None:
    ...
  def __add__(self, other: Self) -> Self:
    ...
  def __mul__(self, other: int) -> Self:
    ...
#

Before you ask, there is reason for making this generic container. Yes, I have tried all of collections.abc containers. No, none of them was a good fit for my usecase. Yes, I am aware I can just do a union over list and ndarray and call it a day but it's not sth I consider good practice.

#

Here is the error I am getting.

def _controlled_qubit_gate(self,
                               gate: Literal["I", "X", "Y", "Z", "H", "S", "T", "RX", "RY", "RZ"],
                               control_indices: int | Collection[int],
                               target_indices: int | Collection[int],
                               angle: float=0) -> None:
        control_indices = [control_indices] if isinstance(control_indices, int) else control_indices # HERE
        target_indices = [target_indices] if isinstance(target_indices, int) else target_indices # And HERE

        # Define the gate mapping for the non-parameterized controlled gates
        gate_mapping = {
            "X": qml.PauliX(0).matrix(),
            "Y": qml.PauliY(0).matrix(),
            "Z": qml.PauliZ(0).matrix(),
            "H": qml.Hadamard(0).matrix(),
            "S": qml.S(0).matrix(),
            "T": qml.T(0).matrix(),
            "RX": qml.RX(angle, wires=0).matrix(),
            "RY": qml.RY(angle, wires=0).matrix(),
            "RZ": qml.RZ(angle, wires=0).matrix(),
        }

        # Apply the controlled gate controlled by all control indices to each target index
        for target_index in target_indices:
            self.circuit.append(
                qml.ControlledQubitUnitary(
                    gate_mapping[gate],
                    control_wires=control_indices,
                    wires=target_index
                )
            )

And this is the error I am getting for the marked lines:

Expression of type "list[int] | Collection[int]" is incompatible with declared type "int | Collection[int]"PylancereportAssignmentType
#

Meaning it doesn't understand that list itself should be interpretted as Collection.

#

I would deeply appreciate some guidance regarding this.

trim tangle
rose mist
trim tangle
#

if the user has just a single int, they can put square brackets around it

rose mist
#

It's not the int.

trim tangle
#

Ah I see

trim tangle
#

I think pylance does not like it when you redefine an argument with a different type

# 
def f(x: int) -> None:
    x = str(x)
rose mist
#

I think my approach whilst starting on the right track is quite flawed. Firstly because if the user for instance passes a numpy array of ints, it's not actually int. It's np.int32 and so on. So, then you'd have to make a new type alias for int itself. Feels like a weird thing to do.

Am I going about the generalization the wrong way?

rose mist
#

Casting is a very standard thing.

trim tangle
rose mist
#

So, what should I do now?

trim tangle
#

You can rename the variable to control_indices2 or something more creative

rose mist
#

Wait, so now I have to rename my vars?

trim tangle
#

The argument name is not very accurate then. 42 is not indi_ces_

rose mist
#

There should be a better way, no?

trim tangle
rose mist
trim tangle
#

The part about an object being both int and a Collection[int] is not relevant

#

As a library user/maintainer, I'd much prefer a simpler interface, even if it means I have to write [42] instead of 42

rose mist
#

Also, I wanted to be smart about this and not do a ton of overloads.

trim tangle
#

[42] is not harder to do than 42

rose mist
#

I mean...

#

When you think about an index, you think about an int, not [42].

trim tangle
#

but the parameter is indi_ces_

rose mist
#

I don't know a better way of saying index/indices.

trim tangle
#

The harder thing, in my view, is understanding a very flexible function interface. Later you'll add more cases later and end up with a monster like this
https://github.com/aio-libs/aiohttp/blob/master/aiohttp/typedefs.py#L41-L49
If a function accepts a dict[str, BaseCookie[str]], then you immediately know how to call it. If it accepts LooseCookies... you'll need to read the type alias definition and figure out what you could have

rough sluiceBOT
#

aiohttp/typedefs.py lines 41 to 49

LooseCookiesMappings = Mapping[str, Union[str, "BaseCookie[str]", "Morsel[Any]"]]
LooseCookiesIterables = Iterable[
    Tuple[str, Union[str, "BaseCookie[str]", "Morsel[Any]"]]
]
LooseCookies = Union[
    LooseCookiesMappings,
    LooseCookiesIterables,
    "BaseCookie[str]",
]```
rose mist
# 
Argument of type "list[int]" cannot be assigned to parameter "qubit_indices" of type "int | Collection[int]" in function "Identity"
#

Even with just list it doesn't work.

trim tangle
#

Can you show the code?

rose mist
#

Sure.

#

!paste

rough sluiceBOT
#
Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

rose mist
#

Would you like me to push this real quick and check the full codebase?

trim tangle
# rose mist

you need to make the parameters in __getitem__ and other dunders positional-only, probably

rose mist
#

Is there a rule in pylance for retyping a variable?

#

I can just turn that off.

trim tangle
# rose mist Why?

The protocol says: it should be legal to call the_thing.__getitem__(idx=0)

rose mist
#

It's erasing the problem, which I hate more than anything...

rose mist
#

I allow for both int and slice.

trim tangle
#

But this call won't work if the_thing is a list, its __getitem__ only accepts a positional argument

rose mist
#

Is it maybe it needing T = TypeVar("T")?

trim tangle
#

hm?

trim tangle
rose mist
#

I believe I tested my Collection before and it would pass for both list and ndarray.

trim tangle
#

How did you test it?

rose mist
#

Alisa helped me with it.

tranquil turtle
trim tangle
rose mist
#

I think I can narrow my question a bit more. How can I have pylance interpret list[int] as a Collection[int]?

#

Alternatively, and I hate to say this, I could make Collection a type alias for list and ndarray? For now at least.

#

Would that be just

Collection: TypeAlias = List[T] | # NDArray type? I tried NDArray but it didn't accept it. Also tried np.ndarray, same issue
undone saffron
trim tangle
rose mist
oblique urchin
#

i think the problem is list.getitem doesn't return Self for a slice

#

(on a phone so can't check)

grave fjord
rough sluiceBOT
#

stdlib/builtins.pyi line 1026

def __getitem__(self, s: slice, /) -> list[_T]: ...```
rose mist
#

Like so?

T = TypeVar("T")

@runtime_checkable
class Collection(Protocol[T]): # pragma: no cover
  def __len__(self) -> int:
    ...
  def __iter__(self) -> Iterator[T]:
    ...
  @overload
  def __getitem__(self, idx: int, /) -> T:
    ...
  @overload
  def __getitem__(self, idx: slice, /) -> Self:
    ...
  @overload
  def __setitem__(self, idx: int, value: T, /) -> None:
    ...
  @overload
  def __setitem__(self, idx: slice, value: Self, /) -> None:
    ...
  def __add__(self, other: Self, /) -> Self:
    ...
  def __mul__(self, other: int, /) -> Self:
    ...
#

It removed the errors, but I don't understand why.

#

If you're wondering why on earth I added / to all of the methods, the errors didn't go away with just __getitem__.

rose mist
grave fjord
rose mist
#

Ohh I missed that, my apologies.

rose mist
#

It's a list, and __getitem__ with slice is returning a list.

undone saffron
#

with a protocol, you're allowed to only specify the parts of the protocol you personally use and care about, being more permissive that other behaviors exist too

#

if you dont index with a slice, you don't need to specify the interface you expect when that happens

rose mist
undone saffron
#

ah, then yeah, you've got more to specify here

rose mist
undone saffron
#

if slice was genenric, you could model only that case.

rose mist
#

Still this error remains:

Argument of type "list[int] | Collection[int]" cannot be assigned to parameter "qargs" of type "Sequence[QubitSpecifier] | None" in function "append"PylancereportArgumentType

For this line

# Apply the controlled gate controlled by all control indices to each target index
for target_index in target_indices:
    self.circuit.append(gate_mapping[gate], control_indices[:] + [target_index])

Which is odd given this method takes lists and ndarrays, so why not Collection?

#

A Collection is a Sequence.

undone saffron
#

other way around

#

a Sequence is a collection

rose mist
#

I mean a Collection can be treated as a Sequence but a Sequence can't be treated as a Collection.

oblique urchin
rose mist
rose mist
undone saffron
#

there's a lot of missing capabilties people want when it comes to structural typing, you might be able to do:

type alias = Sequence[int] | MutableSequence[int] | numpy.typing.NDArray[Any]

might also be able to just use numpy.typing.ArrayLike

rose mist
#

!e

import numpy as np
from numpy.typing import ArrayLike

a = [1, 2, 3]
b = np.array(a)

print(isinstance(a, ArrayLike))
print(isinstance(b, ArrayLike))
rough sluiceBOT
# rose mist !e ```py import numpy as np from numpy.typing import ArrayLike a = [1, 2, 3] b ...

:x: Your 3.12 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 7, in <module>
003 |     print(isinstance(a, ArrayLike))
004 |           ^^^^^^^^^^^^^^^^^^^^^^^^
005 |   File "/lang/python/default/lib/python3.12/typing.py", line 1564, in __instancecheck__
006 |     return self.__subclasscheck__(type(obj))
007 |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
008 |   File "/lang/python/default/lib/python3.12/typing.py", line 1568, in __subclasscheck__
009 |     if issubclass(cls, arg):
010 |        ^^^^^^^^^^^^^^^^^^^^
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/4FB4VPYE3767OKJC6RLOU3YAAM

rose mist
# 
TypeError: Subscripted generics cannot be used with class and instance checks
undone saffron
#

it's not a runtime checkable option here

rose mist
#

Right.

#

What's the proper way of doing what I'm trying to do? Doing union of a bunch of types feels weird.

#

I feel duck typing is the way to go, but as you can see it's still not working.

undone saffron
#

If you need it to be runtime checkable, you may want to check out https://github.com/beartype/numerary/

I personally would not reccomend runtime checking for this for a multitude of reasons involing some currently existing soundness holes, but there are some libraries that exist to try and make numeric types work well with typing

GitHub
GitHub - beartype/numerary: A pure-Python codified rant aspiring to...

A pure-Python codified rant aspiring to a world where numbers and types can work together. - beartype/numerary

rose mist
#

I feel if I want to do runtime checking I can just do with my Collection.

#

I honestly still don't understand why it's not interpreting a list as a Collection.

trim tangle
rose mist
#

Am I misunderstanding it?

trim tangle
# rose mist Am I misunderstanding it?

If you have this:

class A:
    def foo(self) -> Self:
        ...

class B(A):
    ...

it means that B().foo() returns B, but here: ```py
class A:
def foo(self) -> "A":
...

class B(A):
...
``` B().foo() returns A

#

If you subclass a plain list and get a slice out of an instance of your subclass, you will get a plain list, not an instance of your subclass

rose mist
#

Hmm

#

So, I can't use the generic container with what you're saying, since if I set it to list, then it's gonna have issues with NDArray, and vice versa.

#

In my head, I understood it as list return a subset list, and ndarray returns a subset ndarray.

#

Hence Self.

trim tangle
rose mist
trim tangle
trim tangle
rose mist
rose mist
#

So like this?

T = TypeVar("T")

@runtime_checkable
class Collection(Protocol[T]): # pragma: no cover
  def __len__(self) -> int:
    ...
  def __iter__(self) -> Iterator[T]:
    ...
  @overload
  def __getitem__(self, idx: int, /) -> T:
    ...
  @overload
  def __getitem__(self, idx: slice, /) -> Collection[T]:
    ...
  @overload
  def __setitem__(self, idx: int, value: T, /) -> None:
    ...
  @overload
  def __setitem__(self, idx: slice, value: Collection[T], /) -> None:
    ...
  def __add__(self, other: Collection[T], /) -> Collection[T]:
    ...
  def __mul__(self, other: int, /) -> Collection[T]:
    ...
trim tangle
#

Yes

rose mist
#

Hmmm

trim tangle
#

Actually, __add__ might require Self

rose mist
#

I am really sorry for being slow.

rose mist
#

Isn't it following the same logic?

trim tangle
#

You can only add a list to a list, not an arbitrary collection to a list

#

So maybe ```py
def add(self, other: Self, /) -> Collection[T]:

rose mist
#

But, uh, hmmm...

undone saffron
#

If you don't use list addition, I'd leave it out. binops are not easy to type properly, and in this particular case, it's actually impossible to fully express in the type system

rose mist
trim tangle
#

But how do you add two things if you don't know if both of them are lists?

#

Do you have an example of where you use it?

undone saffron
#

lists also add differently from numpy arrays, if you are accepting both of those 🤔

trim tangle
#

oh yeah

rose mist
trim tangle
#

the method signature would fit, but not the meaning

undone saffron
#

like, straight up different results here, numpy arrays add element wise, lists create a new list that's one concatenated with the other

rose mist
# 
for target_index in target_indices:
            self.circuit.append(gate_mapping[gate], control_indices[:] + [target_index])
#

Full code

def _controlled_qubit_gate(self,
                               gate: Literal["I", "X", "Y", "Z", "H", "S", "T", "RX", "RY", "RZ"],
                               control_indices: int | Collection[int],
                               target_indices: int | Collection[int],
                               angle: float=0) -> None:
        control_indices = [control_indices] if isinstance(control_indices, int) else control_indices
        target_indices = [target_indices] if isinstance(target_indices, int) else target_indices

        # Define the gate mapping for the non-parameterized controlled gates
        gate_mapping = {
            "X": XGate().control(len(control_indices)),
            "Y": YGate().control(len(control_indices)),
            "Z": ZGate().control(len(control_indices)),
            "H": HGate().control(len(control_indices)),
            "S": SGate().control(len(control_indices)),
            "T": TGate().control(len(control_indices)),
            "RX": RXGate(angle).control(len(control_indices)),
            "RY": RYGate(angle).control(len(control_indices)),
            "RZ": RZGate(angle).control(len(control_indices))
        }

        # Apply the controlled gate controlled by all control indices to each target index
        for target_index in target_indices:
            self.circuit.append(gate_mapping[gate], control_indices[:] + [target_index])
trim tangle
rose mist
undone saffron
#

then you can't accept numpy arrays

trim tangle
#

!e

import numpy as np
xs = np.arange(6).reshape(3, 2)
print(xs)
print(xs + [42])
rough sluiceBOT
rose mist
#

I see.

trim tangle
#

Actually, why does numpy allow this 🤔

rose mist
#

Not for container representation.

trim tangle
#

Yeah I mean, adding a 3x2 with a 1x1

undone saffron
#

broadcasting addition is fine, I actually have more of an issue with python having list addtion

rose mist
#

Try with 1x4.

trim tangle
#

ah, it treats a 1x1 as a scalar

undone saffron
#

which saves you for needing to do .item on certain reductions

rose mist
undone saffron
#

not if you expect addition to be concatenation, no

rose mist
#

Should I make a class to represent indices?

trim tangle
#

Everything would be so much simpler if you accepted a single concrete type 😛

rose mist
#

And define dunders there?

rose mist
#

Although, if I can't pass numpy directly then there's no point anyways, I could just use sth from collections.abc.

trim tangle
#

You could accept an ndarray if you indent to work with those

rose mist
#

Issue with collections.abc was it would never fit both.

undone saffron
#

you could also just not use + for this and isntead do the concatenation/alternative indexing yourself, based on what types you have, but you're increasing the amount of work you're doing

rose mist
trim tangle
rose mist
#

Should I just make a class called IndexSet and just convert list and ndarray there to a IndexSet instance and define dunders for it?

undone saffron
#

rather than +, you can do : [*control_indices, target_index]

#

if you really want to retain the capability you have on that line

rose mist
rose mist
trim tangle
#

Actually, + on list is pretty stinky, considering that a + b is already expressible as [*a, *b]
Also it allows people to do atrocities like sum(lists, start=[])

undone saffron
#

yeah, I'm not a fan of + being concatenate, but we'd need a python4 to fix that and some other things, let alone agreement on what the things justifying a python4 would even be.... it wont happen

rose mist
#

I can't change python, but I can make a class Hehehe.

#

It'd be generic enough, but also a concrete class.

#

Actually no, then people would have to use my container to make lists.

trim tangle
#

I'm not sure what you're even using this Collection for

rose mist
rose mist
#

I just wanted to allow people to pass list or ndarray.

#

Or anything that behaves like that.

#

Let me push the changes and make it public, one moment...

trim tangle
#

You don't have to use the same protocol every time you want something that smells like a list. In this function you only really need a collections.abc.Collection, since you need to iterate over it and get its len. So use that for the parameter

rose mist
trim tangle
#

Not for this method

rose mist
#

Other methods will call these methods, and they need the additional methods. I tried it before, it didn't work because it's quite interconnected, so I need the smallest common denominator.

#

And that needs those dunders.

trim tangle
#

Is there more code in this method?

rose mist
#

Would you like to see the full codebase?

trim tangle
#

sure

rose mist
#

Ok, one moment...

trim tangle
rose mist
#

Just gonna run pytest to ensure I didn't break anything, and I'll push to the repo and make it public and send the link here.

rose mist
trim tangle
#

But if you some specific needs, it's alright to accept a concrete type. You don't have a protocol for Literal["I", "X", ...] or int or float, right?

#

What you could do is accept any collection/iterable in your class's __init__ (assuming this is where you get them from), and then convert them to list for use by the class

rose mist
#

Can you kindly clone the repo and let me know so I can make it private again please?

trim tangle
#

Why do you want to make it private?

rose mist
rose mist
#

And that I don't like to put unfinished code on my organization repo.

trim tangle
rose mist
trim tangle
#

Well, big projects are never really done

rose mist
#

True, but still haven't reached the MVP stage.

#

You should see the backlog of tasks on my SCRUM board.

#

I still haven't added proper backend support, still adding additional support for other packages, fixing a bug with Mottonen, and figuring out the Bra-Ket interface.

#

It's currently a very tiny percentage of what I aim for as the alpha release.

#

By the time I'm done, it'll likely be the new go-to SDK for gate-based quantum computing.

#

Did you clone it?

#

Two things you'll need. One, open it in WSL remote window (I use vscode). This is because cuda-quantum only works on Ubuntu. Two, use python 3.11.9, that's the only one that's compatible for making the venv.

tranquil ledge
#

Apologies if this already came up during the convo, but are the places that you’d like to use your protocol exposed to users, or is this all internal?

rose mist
rose mist
#

But right now as explained above, I have not done this, so it needs fixing, or rather complete rewriting.

tranquil ledge
#

Gotcha.

rose mist
#

@trim tangle May I ask if you cloned it?

undone saffron
# rose mist Sinbad can you please clone it as well?

I could, but I'm unlikely to spend more time on this in terms of specific reccomendations, the general ones I'd give here:

  1. Start with determining what interfaces you need (how you use it)
  2. determine if making it generic actually allows more types
  3. determine if expressing what is neccessary is possible in a way that allows more than just a union of a few types

type hint appropriately from there.

#

there's an implicit 0. there of just start with specific types you will use it with before generalizing it for others

rose mist
#

Understood, thank you very much.

#

Can I make it private now?

trim tangle
#

if you want to

rose mist
#

Did you clone it?

trim tangle
#

I think I've looked at the relevant bits

rose mist
#

Oh I see.

#

So, what do you think?

trim tangle
# rose mist So, what do you think?

I would choose one of these options:

  1. Accept a concrete type (like list)
  2. Accept a very general type like Iterable at interface boundaries, and then turn it into a list for actual use (if you need to mutate it and such. actually mutating a parameter is pretty stinky, so don't do that)
  3. Accept a very general type like Collection or Sequence and then only do operations on it that you know you can do. You can do [*collection, new_element] instead of using + and list(collection) * 3 instead of using * on the object directly
rose mist
trim tangle
#

I forgor whether I have

#

Can you paste it?

rose mist
#

Of course, uno momento...

#

!paste

rough sluiceBOT
#
Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

rose mist
#

Sorry, had to delete the methods at the end (you don't need them) to make it fit.

#

So, have a look at these two methods:

  • .vertical_reverse()
  • .horizontal_reverse()
    I believe these are instances of where I mutate sth.
#

So, the indices are changed (which are of type Collection).

trim tangle
#

Yes, you mutate some internal state of the object. The user shouldn't care whether it's an ndarray or a list or some quantum magic

rose mist
#

Or the angles (which can be of type Collection[float),

rose mist
#

There's more methods than I can remember to be frank, spread across the codebase. I've noticed that they need all of the ones I mentioned before to work, so I can't go with anything from collections.abc for a generic interface. That would mean it would only really let list or MutableSequence for instance to pass.

trim tangle
rose mist
#

You can see this in gatemethod which is a static method at the top.

#

All the nasty changes, checks, etc., are inside this static method. However, I don't change the type from ndarray to list, or from list to ndarray, so if you pass sth that doesn't support all of the dunders I mentioned, it'll break then.

#

I need __len__, __getitem__, and __iter__ and even __add__ in some cases for gate methods themselves. Afterwards when I call sth like .vertical_reverse() I need __setitem__.

#

With .horizontal_reverse() I need __mul__.

#

So, overall I need all of those dunders for the entirety of the codebase to run correctly.

trim tangle
#

Mutating parameters is generally a bad idea. It's very surprising behaviour, especially if it happens far into the future. It's like quantum entanglement
If you want to mutate some data, you should save a copy of it, not the original one

#

And in that case you can accept any iterable/sequence/collection and just turn it into a list before saving it

rose mist
rose mist
trim tangle
#

You're already converting range to a list

rose mist
# trim tangle Yes, for example

Wouldn't that make my gate methods become list only too (type hint I mean) given how decorators affect the signature of the decorated?

rose mist
trim tangle
rose mist
#

Instead of int | Collection[int] it would be list[int].

#

So if someone hovers over the method to see what it is they'll think it has to be a list.

#

Which basically makes all of this just useless.

trim tangle
rose mist
#

At that point, I should just change my Collection to MutableSequence, and kiss ndarray support goodbye.

trim tangle
rose mist
#

Other way is to do overloads, like what TKET does (qiskit uses generics as far as I can tell) which is veryyyy messy to read.

trim tangle
rose mist
rose mist
#

FWIW it's just a generic type for mutable containers.

#

You can see everything now, I only need the dunders I mentioned.

#

Preferably I'd like to support as many interfaces as possible.

trim tangle
#

You could use a generic type all the way until you actually save it. And then the thing you save will always be converted to a list

rose mist
# trim tangle Yes, that's unfortunate. There's no way to express argument conversions

So, choices are:

  • Make conversion to list internally (this effectively causes the gate method documentation to be purely list[int] for indices which is sth I'm trying to avoid with inclusion of int to begin with)
  • Change to int | MutableSequence[int] but ndarray can't be supported anymore.
  • Perform the conversion within the gate methods, which seems messy and would appear across all subclasses of Circuit, but in turn would allow a flexible type hint with the mentioned generic.
#

Third one seems like the best out of the bunch, huh...

trim tangle
#

Third one is an option, yes

rose mist
#

It would allow me to keep int | Collection[int], pass ndarray if I want, and perform the casting to list internally.

#

At the cost of a few honestly dirty lines.

trim tangle
#

I don't think accepting a list from a user and then mutating it later is a good idea. It's very surprising when someone else touches your data in the future. (or if the user mutates the list, and now things inside your Circuit class break)

rose mist
#

Actually, hmm...pylance would technically still scream at me since it doesn't interpret list nor ndarray as a Collection.

rose mist
trim tangle
#

But why not circuit.X((0, 1))?

rose mist
trim tangle
#

well my point was, you don't need to require it to be mutable, if you want to mutate something, make a copy

rose mist
#

Technically it'd be allowed.

rose mist
#

Or make a copy like you said and set that to my var internally.

#

Wait...

#

F*ck...

#

If I make the casting internally, the data recorded is still what the user passes (I'm recording the signature and param values that user passes).

#

Which means I have to do it in gatemethod staticmethod one which would again make the type hint all wrong.

#

Goddamit.

#

user passes values -> values go through decorator first, where decorator has to convert them to list -> which forces the gate methods themselves to be type hinted as lists only as well in turn.

Alternatively
user passes values -> values go through decorator first, where decorator makes a copy and records that -> this allows the gate methods to be int or collection int, but would need another copy and cast inside the method definition

#

So, second one would need to repeat the conversion and in return would allow me to keep the signatures like I want it (int or collection int, or whatever generic container, but allowing int).

#

Seems like an ok deal?

#

It would be a justified duplication as it pays back with a simpler user interface.

#

What do you think?

#

Am I getting colder?

#

Ok, so I think this would make some sense:

  1. Type hint the indices as int | Collection[int]. This needs some fixing since pylance doesn't accept list as a Collection.
  2. Copy the indices value in the decorator, and convert to list and record that for self.circuit_log, but let the non-converted value go to the decorated method.
  3. Redo the conversion inside the decorated method, and always write the dunder uses assuming a list is being used.
rose mist
#

Oh my god, I killed him with my stupidity!

viscid spire
#

no error here

image.png
grave fjord
viscid spire
#

wut

#

I have a feeling I'm not gonna like this...

grave fjord
viscid spire
#

rlymad

rose mist
# viscid spire rlymad

I wanted to provide a generic container compatible with both list and ndarray. Nothing from collections.abc fit both, so I had to make my own.

#

Issue now is that I neglected to remember that ndarray and list have different dunders, so I'd have to cast to list internally, and was showing fix error my ideas.

#

I'd appreciate feedback on that.

rose mist
#

This is how qiskit has type hinted it.

image.png
#

I'm going to see what the type of Qubit is.

#

This is the src for Qubit and QuantumRegister (the latter is usually how you would express a container of qubits).


class Qubit(Bit):
    """Implement a quantum bit."""

    __slots__ = ()

    def __init__(self, register=None, index=None):
        """Creates a qubit.

        Args:
            register (QuantumRegister): Optional. A quantum register containing the bit.
            index (int): Optional. The index of the bit in its containing register.

        Raises:
            CircuitError: if the provided register is not a valid :class:`QuantumRegister`
        """

        if register is None or isinstance(register, QuantumRegister):
            super().__init__(register, index)
        else:
            raise CircuitError(
                "Qubit needs a QuantumRegister and %s was provided" % type(register).__name__
            )


class QuantumRegister(Register):
    """Implement a quantum register."""

    # Counter for the number of instances in this class.
    instances_counter = itertools.count()
    # Prefix to use for auto naming.
    prefix = "q"
    bit_type = Qubit
#

@trim tangle So, qiskit has done it like you said (a concrete class). But it's their custom concrete class.

#

So, pylance screams at me when I pass ndarray to qiskit (saying NDArray is not a match for QubitSpecifier, however, it works if I run it).

#

So, not sure how to think about that.

#

Either pylance doesn't understand that it needs to look at the interface of the container passed (here it's ndarray) to assess the situation, or it's just being nit picky.

#

Likely the prior.

#

TKET just uses Sequence with lots of overloaded options.

(method)
def add_gate(
    Op: Op,
    args: Sequence[int],
    **kwargs: Any
) -> Circuit: ...

def add_gate(
    Op: Op,
    args: Sequence[UnitID],
    **kwargs: Any
) -> Circuit: ...

def add_gate(
    type: OpType,
    args: Sequence[int],
    **kwargs: Any
) -> Circuit: ...

def add_gate(
    type: OpType,
    args: Sequence[UnitID],
    **kwargs: Any
) -> Circuit: ...

def add_gate(
    type: OpType,
    angle: Expr | float,
    args: Sequence[int],
    **kwargs: Any
) -> Circuit: ...

def add_gate(
    type: OpType,
    angle: Expr | float,
    args: Sequence[UnitID],
    **kwargs: Any
) -> Circuit: ...

def add_gate(
    type: OpType,
    angles: Sequence[Expr | float],
    args: Sequence[int],
    **kwargs: Any
) -> Circuit: ...

def add_gate(
    type: OpType,
    angles: Sequence[Expr | float],
    args: Sequence[UnitID],
    **kwargs: Any
) -> Circuit: ...
#

So the indices are either Sequence[int], or Sequence[UnitID].

rose mist
#

So, I think this could be a good solution. In my decorator, I record the params right? And only the recorded params need to be iterable, not the actual params of the decorated.

What I can do is to convert any container to a list and record that, but only pass a Sequence (from collections.abc) to the decorated.

#

What do you guys think?

#

Although, I don't know how to ensure what's being sent to the decorated (from the decorator part) would be a Sequence. If I indiscriminately cast containers to list, then isn't pylance going to scream at me saying it wants a list?

#

Only the param sent to self.circuit_log needs to always be a list. The param sent to the decorated just needs to be a Sequence (I would need __len__, __getitem__, and __iter__ as far as I can tell).

viscid spire
rose mist
vivid ore
#

Does anybody know why mypy rejects this?

from numbers import Real

def foo(a: Real, b: Real, c: float) -> Real:
    return c * (a - b)

saying:

main.py:4: error: Incompatible return value type (got "float", expected "Real")  [return-value]
main.py:4: error: Unsupported operand types for * ("float" and "_ComplexLike")  [operator]
#

Why is float not compatible with Real?
Why is the subtraction of two Reals a _ComplexLike?
Why can’t a _ComplexLike be multiplied by a float?

stiff acorn
#

is there something wrong here?

image.png
# 
from typing import reveal_type

def greet(name: str | bytes, encoding: str = "utf-8") -> None:
    name = name.decode(encoding) if isinstance(name, bytes) else name
    reveal_type(name)
    print(f"Hello {name}!")
#

I just want to accept either bytes or string and convert it to string internally

tranquil ledge
#

bytes gets expanded by type checkers into bytes | bytearray | memoryview when in parameter annotations (and possibly elsewhere), so your code only accounts for one of those three possibilities.

stiff acorn
#

okay, thank you

#

Another question, what's the difference between bound and constraints in a TypeVar

#

I want a TypeVar that allows only two types, i.e, str or None

oblique urchin
tranquil ledge
oblique urchin
tranquil ledge
# oblique urchin It's type checker behavior and doesn't depend on the Python version. I think we ...

Fair enough. Makes me a bit curious, because I imagine there's some implicit dependence on python version if only to avoid immediately invalidating a ton of <3.12 code with bytes annotations that depends on this promotion since it hasn't transitioned to Buffer yet. 3.11 being EOL would provide justification since Buffer will always available then, even without typing_extensions. Is there any explicit coordination between type-checkers for this kind of thing, or could e.g. mypy deprecate this promotion a month from now and pyright a year from now?

oblique urchin
tranquil ledge
oblique urchin
#

Well it's more that you can't really "force" type checkers

#

In PEP 688 I simply wrote that the special case should be removed

tranquil ledge
#

Right, "force" doesn't convey what I wanted it to there. "encourage" is more like it, I suppose, for the sake of potentially transitioning the user bases at once.

trim tangle
stiff acorn
barren pendant
#

Having issues with code I'm creating a number guessing game project to get more familiar I've added the quit() command so if the user inputs a string it should say "please pick a number next time" and then quit I'm getting error code NameError: name 'quit' is not defined. Any thoughts on what I'm doing wrong? Running on Spyder

#

Lemme know if this needs to be put elsewhere as well

trim tangle
barren pendant
#

I will do so tomorrow when I have more brain power, thank you

tranquil turtle
#

i have a generic class A[X, Y]
is there a way to make A[X] equivalent to A[X, X] ?

tranquil ledge
tranquil turtle
#

!pep 696

rough sluiceBOT
tranquil turtle
#

X here is a typevar
i want A[int] to be the same as A[int, int] and A[str] to be the same as A[str, str]

how do i do that?
would class A[X, Y = X]: ... work?

#

pep 696 is very vague about what can be a default type of typevar

soft matrix
#

Is it?

undone saffron
tranquil turtle
#

i see
this is the most cryptic text i have seen in a while

undone saffron
#

seems clear to me 😐

#

anyhow, for your case:

from typing_extensions import TypeVar  # pre 3.13 backport for default= in TypeVar

X = TypeVar("X")
Y = TypeVar("Y", default=X)

class A(Generic[X, Y]):
    ...
tranquil turtle
# 
class A[X, Y=X]: ...

?

oblique urchin
tranquil turtle
#

nice!
thank you both

cinder bone
#

Is there a way to type hint this, maybe with overloads+generics on __init__? Or am I stuck with the Callable[..., object]?

@dataclass(frozen=True, slots=True, eq=False)
class Updater:
    _updater: Callable[..., object]
    dt: bool = False

    def __call__(self, mob: OpenGLMobject, dt: float) -> None:
        if self.dt:
            self._updater(mob, dt)
        else:
            self._updater(mob)
tranquil ledge
#

AFAIK, there’s no way currently other than overloads to represent a “relationship” between two sets of types, which is what it looks like you’re going for.

trim tangle
cinder bone
#

I don't, I was just protyping and wanted a nice __repr__ lol

trim tangle
#

If you don't want the dt, you can ignore it in the callback

cinder bone
#

The problem is that it needs to be a class because it'll store lots of other data

#

(in this case like whether the function is going to modify the colors, etc. so that it can be optimized)

#

Also forcing a float parameter is not semvar minor happy :(

trim tangle
#

ah

#

If you're in a deep pickle, maybe```py
class Updater:
@overload
def init(self, Callable[[OpenGLMobject, float], object], dt: Literal[True]) -> None: ...
@overload
def init(self, Callable[[OpenGLMobject], object], dt: Literal[False] = False) -> None: ...

cinder bone
#

Thanks!

cinder bone
#

Another question: if I wanted something like

Callable[[Mobject], object]

where the first parameter is any subclass of Mobject, how would I do it? I don't know why the above works because parameters are (to my understanding) contravariant, but at this point I'm not really sure how variance works anymore 🙃

trim tangle
#

Variance doesn't matter here, because you can always do animal: Animal = cat and then do fn(animal)

#

Variance matter when you need to figure out whether one of (Callable[[Organism], None], Callable[[Dog], None]) is assignable to fn. Callable[[Organism], None] does work: if it works for any organism, it must work for animals (which is only a smart part of that). However, Callable[[Dog], None] doesn't work, because Callable[[Animal], None] needs to be able to work with cats, birds etc., not just dogs

cinder bone
#

So in general variance is only for assigning to something?

trim tangle
cinder bone
#

yeah conceptually

trim tangle
# cinder bone yeah conceptually

Variance describes how subtyping works for a generic type. For example, list is invariant, so even if A is a subtype of B, list[A] and list[B] aren't subtypes of each other. tuple is covariant, so if A is a subtype of B, then tuple[A, ...] is a subtype of tuple[B, ...]. And functions are contravariant in the parameters but covariant in the return type, so if C is also a subtype of D, then Callable[[B], C] is a subtype of Callable[[A], D]

oblique kindle
#

legit thought that was working

oblique kindle
rare scarab
#

Use TypeAdapter

quasi spindle
# 
from typing import Any, Generic, Protocol, TypeVar


_ST = TypeVar("_ST", bound=Any)
_ST_co = TypeVar("_ST_co", bound=Any, covariant=True)

_DT = TypeVar("_DT", bound=Any)
_DT_co = TypeVar("_DT_co", bound=Any, covariant=True)


class ArrayProtocol(Protocol[_ST_co, _DT_co]):
    def count(self, value: Any, /) -> int: ...


class Array(Generic[_ST, _DT]):
    def __init__(self, shape: _ST, dtype: _DT) -> None:
        self._shape = shape
        self._dtype = dtype

    def count(self, value: Any, /) -> int:
        return 2

    @property
    def shape(self) -> _ST:
        return self._shape


def test_pass_typevars(a: ArrayProtocol[_ST, _DT]) -> ArrayProtocol[_ST, _DT]:
    return a


arr = Array(shape=(3,), dtype=int)
reveal_type(arr)  # information: Type of "arr" is "Array[tuple[Literal[3]], type[int]]"

# array: ArrayProtocol[tuple[int, ...], int] # Works correctly uncommented
array = test_pass_typevars(arr)
reveal_type(array)  # information: Type of "array" is "ArrayProtocol[Unknown, Unknown]"

I struggle to understand why test_pass_typevars does not pass through the typevars from arr, with pyright I get Unknown, and Any with mypy. Any ideas why?

trim tangle
#

ArrayProtocol does not use any of its typevars, so anything with a count method with the appropriate signature satisfies ArrayProtocol[???, ???]

quasi spindle
# 
from typing import Any, Generic, Protocol, TypeVar


_ST = TypeVar("_ST", bound=Any)
_ST_co = TypeVar("_ST_co", bound=Any, covariant=True)

_DT = TypeVar("_DT", bound=Any)
_DT_co = TypeVar("_DT_co", bound=Any, covariant=True)


class ArrayProtocol(Protocol[_ST_co, _DT_co]):
    @property
    def dtype(self) -> _DT_co: ...

    @property
    def shape(self) -> _ST_co: ...


class Array(Generic[_ST, _DT]):
    def __init__(self, shape: _ST, dtype: _DT) -> None:
        self._shape = shape
        self._dtype = dtype

    @property
    def shape(self) -> _ST:
        return self._shape

    @property
    def dtype(self) -> _DT:
        return self._dtype

    def count(self, value: Any, /) -> int:
        return 2


def test_pass_typevars(a: ArrayProtocol[_ST, _DT]) -> ArrayProtocol[_ST, _DT]:
    return a


arr = Array(shape=(3,), dtype=int)
reveal_type(arr)  # information: Type of "arr" is "Array[tuple[Literal[3]], type[int]]"

array = test_pass_typevars(a=arr)
reveal_type(array)  #  information: Type of "array" is "ArrayProtocol[tuple[Literal[3]], type[int]]"

aha, forcing the protocol to use the typevars does indeed do the trick, thanks @trim tangle !

trim tangle
quasi spindle
#

Yeah, the more advanced version will have a proper bound eventually.

trim tangle
restive rapids
#

| Type annotations | Аннотации тип**а** |
нот аннотации тип**ов**? https://fastapi.tiangolo.com/ru/python-types/ uses that for example, and most articles on types in russian language.
perhaps same with type variables but im not sure

tranquil turtle
trim tangle
#

Type annotation in English is more like "Типоаннотация", it refers to both cases

#

Typanmerkung

restive rapids
trim tangle
tranquil turtle
#

if you share invite to your private server, we can discuss there without cluttering this channel)

trim tangle
#

)

#

Maybe we should discuss this in the PR

restive rapids
#

github pr messaging is very.. not live

#

| positional-only | чисто позиционный | lol
"чисто" is kinda jargon, "только" (only) makes more sense

trim tangle
#

that is true

trim tangle
muted iron
# 
from dataclasses import dataclass
from typing import Protocol

@dataclass(frozen=True)
class D:
    x: int

class P(Protocol):
    x: int

def f(a: P) -> None:
    pass

f(D(1))

Argument of type "D" cannot be assigned to parameter "a" of type "P" in function "f"
  "D" is incompatible with protocol "P"
    "x" is writable in protocolPylancereportArgumentType

What do? Make x a property?

#

Remove frozen?

#

The equivalent of f in the actual code takes a mutable D1 or an immutable/frozen D2 (see above). It doesn't mutate anything.

restive rapids
frank field
#

is (vartype,vartype) syntax valid in any case, like maybe in older versions or smth? noticed it in a module someone else made; my vscode doesn't like it

image.png
restive rapids
oblique urchin
#

The runtime lets you put whatever you want in annotations

restive rapids
#

basedmypy also allows it i think, but it allows.. a lot of weird stuff. although it wouldnt be valid here either as the amount of values doesnt match

frank field
#

that confused me as well

restive rapids
frank field
#

huh odd the Tuple[var] syntax is used in other bits of this code

oblique urchin
frank field
#

yeah thats me basically lol (though i didnt write this)
i mainly just do it for the ide highlighting

#

btw why does typing have things for List and Dict but you can just do tuple[var]

restive rapids
frank field
#

ah

#

oh mfw theres also one for tuple
i guess i just never checked to see if there was one lmao

rare scarab
#

yup. list[T], dict[K, V], set[T], tuple[T, ...]

outer beacon
#

is there a way to have the type checker enforce that an attribute in an abstract class must be overwritten?

lunar dune
outer beacon
#

in the example below, you’ll see an error if you try to assign attribute foo to the wrong type. but if you leave it off entirely, no error is shown

class AbstractFoo:
    foo: str


class Foo(AbstractFoo):
    foo = 5  # shows an error

class Bar(AbstractFoo):
    bar = 5  # no error for not assigning foo
outer beacon
oblique urchin
#

I think pyre may check for this

lunar dune
#

I think attributes on Protocol classes that are not initialised are also treated as abstract by at least some type checkers

reef vortex
#

Guys why i getting this mypy error even I have type hinted it

class FileHandler(logging.FileHandler):
    _lastEntry: datetime.datetime = datetime.datetime.today()

    def __init__(self, *, ext: str, folder: Path | str = "logs") -> None:
        self.folder = Path(folder)
        self.ext = ext
        self.folder.mkdir(exist_ok=True)
        super().__init__(
            self.folder / f"{datetime.datetime.today().strftime('%Y-%m-%d')}-{ext}.log",
            encoding="utf-8",
        )
        self.setFormatter(Formatter(name=ext))

    def emit(self, record: logging.LogRecord) -> None:
        if self._last_entry.date() != datetime.datetime.today().date():
            self._last_entry = datetime.datetime.today()
            self.close()
            self.baseFilename = (
                self.folder / f"{self._last_entry.strftime('%Y-%m-%d')}-{self.ext}.log"
            ).as_posix()
            self.stream = self._open()
        super().emit(record)

src/utils/logger.py:68: error: Cannot determine type of "_last_entry"  [has-type]
oblique urchin
rustic gull
#

this is a pytorch function

def _foreach_sub(
    self: Tuple[Tensor, ...] | List[Tensor],
    other: Tuple[Tensor, ...] | List[Tensor],
    *,
    alpha: Number | _complex = 1
) -> Tuple[Tensor, ...]: ...

how to annotate a Callable to accept functions like this??

#

I say Callable[[TensorSequence, TensorSequence, PyNumber], TensorSequence] but it screams at me because alpha is keyword only

rustic gull
# 
class TorchForeachAlphaFn(Protocol):
    def __call__(self, __self: TensorSequence, __other: TensorSequence, *, alpha: PyNumber) -> TensorSequence: ...
rustic gull
# 
Type parameter "_T@list" is invariant, but "Parameter" is not the same as "Tensor"
      Consider switching from "list" to "Sequence" which is covariant
#

why is list invariant

#

this is even dumber how to fix

      Type parameter "_T@list" is invariant, but "Parameter" is not the same as "Tensor | Parameter"
restive rapids
# rustic gull why is list invariant

mutable things tend to be invariant, imagine you have a list[int], and you could pass it to something that expects a list[str | int] -> it could add strings to your list of ints

rustic gull
#

because pytorch has List[Tensor] | Tuple[Tensor], so if I say Sequence[Tensor], it says Sequence is incompatible