#type-hinting

1 messages · Page 44 of 1

trim tangle
#

this channel is also appropriate, as the channel description suggests

hasty jungle
#

not strictly related to typing, but I'm in doubt:

do variables have types in Python ? (from a formal specification perspective, at the language level). What is the actual formal definition of variable ?

when one annotated a variable with a type, how does it work in the perspective of the formal language definition?

spiral fjord
hasty jungle
#

yeah that I get

fringe junco
#

Hey

#

idk if this is to be sent here but anyways, can anyone help me with type hints to the source code of my programming language?

#

it is in Python, hence im here

trim tangle
rare scarab
#

Ever thought t-strings would make a good type annotation?

# 
x: t"{int}" = "4"
y: t"prefix_{str}" = "prefix_xxx"
fringe junco
trim tangle
fringe junco
# trim tangle > Improving type inference for lists or expressions what do you mean by this?

Right now, the infer_type function in Jam only detects very basic literals:

  • integers (42)
  • floats (3.14)
  • booleans (true / false)
  • strings ("hello"
  • lists ([...]) without checking what’s inside
  • variables already in symbol_table

By “improving type inference for lists,” I mean making Jam figure out the types of elements inside lists. For example, [1, "hi", true] should ideally recognize that the list contains an int, a str, and a bool.

trim tangle
#

Oh, you meant that you want help with type handling in your language, not in the Python source code

fringe junco
#

Yep exactly. I am talking about type handling inside Jam not Python code itself.

#

Basically, building a more accurate type system for Jam itself.

trim tangle
#

I don't think warning about assigning a value of a different type at runtime makes sense as a model

#

I think you should decide what the typing rules for the language are first. Is it a dynamically typed language? Statically typed? Something in between?

restive rapids
#

whatever analysis you'll want to do, you should start with making an AST, because handling all of it as strings is going to be horrible

fringe junco
trim tangle
#

wouldn't it help better if the error was caught without running the code?

fringe junco
trim tangle
#

I don't think beginners are scared by error messages about types (maybe they're scared by bad error messages). They're especially helpful when you don't have the debugging skills of an experienced programmer

restive rapids
fringe junco
trim tangle
#

In a statically typed language, that situation cannot happen at runtime

fringe junco
trim tangle
#

But in a dynamically typed language, that's not supposed to be an error. It's fine to do this in Python ```py
x = 1
x = "foo"
...
name = get_name_bytes()
name = name.decode("utf-8")

fringe junco
restive rapids
#

what about static warnings? static analysis doesnt have to produce errors
if you do it at runtime, you'll need something like, track the type of the first element, then each time it tries to get added an element - check if its type matches that. you'll need a pretty big runtime for stuff like that

fringe junco
fringe junco
#

But I see that doing it at runtime can get heavy, especially for large or nested lists. Using AST-based static analysis to produce friendly warnings before running the code could achieve the same guidance without the runtime overhead.

#

How would you recommend approaching this with ASTs?

restive rapids
#

bidirectional type checking is pretty easy to implement and you can produce errors early with it (=> they can be small and simple, instead of hindley-milner kinda solving-away some stuff but ending up with a crazy error later)

fringe junco
#

The goal is to warn learners gently without blocking execution or making the runtime heavy.

fringe junco
#

but I’d like to try AST-based static warnings instead.

#

Could you give a hint on how to apply bidirectional checking for lists and expressions, especially for beginner-friendly mixed types?

restive rapids
#

i mean
you just have inference rules for things where you can infer their type, like, number literals, string literals, variables that are already given a type in this context
and you have check rules for things where you cant always exactly know but you can check whether its of some type, e.g. an empty list literal being list[T] for any T

for inference of non-empty list literals you could for example infer the type of the first element, and check the other elements against that, though for stuff like xs = [] you'll want an explicit annotation

it will also be easy to implement functions with that (given that your functions have annotations for the parameter types, otherwise you'll want to design the language around it to make hindley-milner style constraint solving viable),
in a call to f with a function type [Parameters] -> Return, just pairwise check the arguments against the parameter types, and infer the call result as the Return type

fringe junco
#

I see how this could extend nicely to functions once Jam supports parameter annotations.

#

would this work well for beginner-friendly mixed-type lists, or should I always warn about mismatches?

restive rapids
fringe junco
#

I’m thinking whether Jam should allow mixed-type lists with gentle warnings or just enforce strict types. What do you think—warnings or strict only?

restive rapids
#

you can support union types if you think beginners can make use of them, but then you'll also want some feature like isinstance that would play with the typesystem by narrowing the type in the context of say, an if, so that it could be used safely

honestly the language feels very weird, the special syntax for map with over xs, the add x and y into z while functions use usual expressions (n * 2)
feels like a bunch of random ideas thrown together

you should spend some time designing it, think of what would make things actually simpler for beginners and what kind of beginners specifically (children? or people that already know stuff but not programing specifically?). its definitely not replacing z = x + y with add x and y into z.
you can take a look into existing beginner/education oriented languages like https://pyret.org/ , https://www.hedy.org/
imo, a lot of "beginner orientedness" is not necessarily in the language feature set itself, as much as it is in its error messages, learning resources and the programming environment (e.g. thonny's debugger is amazing)

you can join https://discord.com/invite/programming-language-development-530598289813536771 and talk to people in it about language design,
and I think it would be great to do ask programmers (both already somewhat experienced and beginners) what things they struggled with when they were learning, and how they think that could be improved

Pyret
indigo halo
#

How can mypy complain that return self.slot1.is_ready and self.slot2.is_ready would return Any? Isn't that by definition a bool?

pastel egret
#

No, Any is anything. and doesn't return a bool, it returns one or the other of its operands.

indigo halo
#

oh, that is news to me, but I guess it makes sense.

#

I use foo or "default" a lot, same thing I guess

#

that said,is_ready is a property declared to return bool, so and should always return bool in this case, no?

# 
    @property
    def is_ready(self) -> bool:
        return self.type in (SlotType.ENTRY, SlotType.BYE)
trim tangle
#

or maybe ```py
reveal_type(self)
reveal_type(self.slot1)
reveal_type(self.slot1.is_ready)
reveal_type(self.slot2)
reveal_type(self.slot2.is_ready)

indigo halo
#

I now need to run and get kids but I will later

trim tangle
#

that's a long process, at least 9 months...

indigo halo
#

i really just did lol and now the kids want to know why

# 
        def _slot(self, pm: PlayerMatch, idx: Literal[0] | Literal[1]) -> Slot:
            if self._slots is None:
                return Slot(content=pm.entry or Unknown())

            return self._slots[idx]

        slot1 = property(lambda s: s._slot(s.pm1, 0))
        slot2 = property(lambda s: s._slot(s.pm2, 1))

        @property
        def is_ready(self) -> bool:
            reveal_type(self)     ● Pyright: Type of "self" is "Self@Match"
            reveal_type(self.slot1)     ● Pyright: Type of "self.slot1" is "Any"
            reveal_type(self.slot1.is_ready)     ● Pyright: Type of "self.slot1.is_ready" is "Any"
            reveal_type(self.slot2)     ● Pyright: Type of "self.slot2" is "Any"
            reveal_type(self.slot2.is_ready)     ● Pyright: Type of "self.slot2.is_ready" is "Any"
#

it's the lambda!

#

and functools.partialmethod cannot be assigned to property fget ☹️

#

slot1 = property(cast(Callable[["Match"], Slot], lambda s: s._slot(0))) also yields Any

#

grrrr

#

no, it's the property

# 
  ▎     def _slot1(self) -> Slot:
  ▎         return self._slot(0)
  ▎
  ▎     def _slot2(self) -> Slot:
  ▎         return self._slot(1)
  ▎
  ▎     slot1 = property(_slot1)
  ▎     slot2 = property(_slot2)
  ▎
        @property
        def is_ready(self) -> bool:
 ▎         reveal_type(self)     ● Pyright: Type of "self" is "Self@Match"
 ▎         reveal_type(self.slot1)     ● Pyright: Type of "self.slot1" is "Any"
#

slot1 = cast(Slot, property(_slot1)) works

#

kinda makes sense given how property works, and how lambda neutres all types

trim tangle
#

show what mypy thinks of those reveal_types

indigo halo
trim tangle
#

The Slot class I mean

indigo halo
trim tangle
#

hold on, is it working for you now?

#

I think I jumped over a few messages, looks like it works

#

actually no, wait

indigo halo
#

yeah, I cast the result of proprty

trim tangle
#

ugh, @property seems to be special-cased by type checkers

#

🥴

trim tangle
indigo halo
#

confirmed

hearty shell
#

Something I just noticed today while looking at functools.singledispatch (for fun, personally I wouldn't really use it), is that it does not interact at all with typing.overload. Is that really the case or am I missing something? playground

mypy Playground

The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.

trim tangle
oblique urchin
#

mypy does have special-case support for it

wooden kayak
hardy oyster
#

Hello, guys! I am building a small library to work with YAML configuratiins files: https://github.com/shadowy-pycoder/pyya Basically, it allows to merge production and sample YAML configs, dynamically validate types with Pydantic, and other stuff related to YAML formatting. Since it uses munchify under the hood to convert dict to dynamic attribute style object, it lacks type hints, autosuggestiond and completions: everything is dynamic and created at runtime. In the last version, however, i added dynamic generation of stub files based on samole config. This approach mostly works, but requires creating global variable, for example, config beforehand. So if stub contains config: Config, you need to create config = init_config() to associate it with stub file and get completion. That is not very convinient, and creates dependency on naming. What else can i do to make things more generic? Thank you

GitHub
GitHub - shadowy-pycoder/pyya: Convert YAML configuration files to ...

Convert YAML configuration files to Python objects - shadowy-pycoder/pyya

trim tangle
indigo halo
#

given a factory-like function:

def myfun[T: MyBaseClass](cls: type[T] = MyBaseClass) -> T: ...

why would mypy complain?

error: Incompatible default for argument "cls" (default has type "type[MyBaseClass]", argument has type "type[T]") [assignment]

trim tangle
#

There's currently no way to influence T other than passing the parameter, but the logic is still the same

restive rapids
indigo halo
#

pyright "allows" it, and it makes sense. Unless I specify cls, T==MyBaseClass

trim tangle
#

Yes, pyright allows it. Mypy doesn't

indigo halo
#

if I specify cls, it has to be a type derived from MyBaseClass

restive rapids
indigo halo
trim tangle
#

Not sure if the type system allows this, but it can present issues if you're making a library

#

if it's not a function exported from a library, then pyright's behaviour is very handy

soft matrix
#

can you not fix it with a typevar default?

trim tangle
#

in this case: py def f[T](x: T, y: T = 42) -> None: ... even if you say f[T = int], there needs to be some way to signal that the default is not applicable for all Ts, and currently there isn't

trim tangle
soft matrix
trim tangle
#

for a brief moment I forgot you were the author

feral wharf
#

I tried that a while back and was confused why it wasn’t valid, and then realised it isn't built-in, yet blobpain

trim tangle
#

Found a new trick for exposing a class but not its constructor (and also making it private): ```py
class _Foo:
def init(self, ...):
# supposed to be private to this module

def method(self, x: int) -> str:
    ...

type Foo = _Foo

soft matrix
#

I might plead with the devs to make these kwargs cause that's the only way I know to do it

spiral fjord
#

small thing I noticed is you got a float | int in there

soft matrix
#

I think it might be cause you can do 26.1 and 261 and to draw more attention to that?

#

why they allowed that is beyond me though

restive rapids
soft matrix
#

the return type is currently a big union which depends on the mode argument

spiral fjord
#

You can make overloads for each mode

soft matrix
#

yeah that's what I'm thinking but man it sucks

tranquil ledge
#

Assuming we could make any syntax or runtime construct, got any ideas what would make that easier to express? I'm currently lacking imagination and don't know what a better way would look like, but I'm sure others have spitballed answers to this problem before.

# Rough idea?

import enum

class FluentMode(enum.Enum): ...

def launch_fluent[
    ModeType: FluentMode,
    ReturnType: (
      Meshing | PureMeshing | Solver | SolverIcing
      corresponding to ModeType  # Create a 1-to-1 correspondence with the members of FluentMode?
    )
](
    *args: object,
    mode: ModeType,
    **kwargs: object,
) -> ReturnType:
    ...
restive rapids
# 
def rtype(mode: FluentMode) -> type:
  match mode:
    case FluentMode.MESHING:
      return Meshing
    ...
  
def launch_fluent(mode: FluentMode) -> rtype(mode):
  ...

would be really cool though, dependent typing

soft matrix
#

dependent or mapped types would make this so much nicer

#

i do also think it needs some kind of typed mapping to make the abundance of errors less bad when you even go to use it

image.png
#

I think I need to do some pyi hacks to make these work at type time

unreal urchin
#

is this possible?

def check_authorization[**P, R](func: Callable[Concatenate[Any, sanic.Request, P], Coroutine[Any, Any, R]] | Callable[Concatenate[sanic.Request, P], Coroutine[Any, Any, R]]):
    @functools.wraps(func)
    async def wrapper(arg1: Any, *args: Concatenate[sanic.Request, P].args | P.args, **kwargs: P.kwargs) -> R:
        return await func(arg1, *args, **kwargs)

    return wrapper

i'm trying to make a decorator that would work for both a method and a function which takes a sanic.Request as the first argument
*args: Concatenate[sanic.Request, P].args | P.args is what's failing.

rare scarab
#

You don't need to concatenate *args. Just add the request arg.

# 
async def wrapper(self: Any, request: sanic.Request=..., *args: P.args, **kwargs: P.kwargs) -> R: ...
#

=... means its optional

#

I'd recommend making 2 wrapper functions. one for method and one for function.

unreal urchin
#

imo the wrapper's signature has to be (arg1, *args, **kwargs) -> R

#

so i just need to know how to type *args proplerly

rare scarab
#

Introduce another typevar. Maybe typevartuple of (Any, Request) | (Request,)

unreal urchin
rare scarab
#

You can unpack it into the *args

unreal urchin
rare scarab
#

Which is why a typevar should be used.

#

Just tested, typevartuple doesn't support bounds

summer berry
#

Why does this fail on mypy? https://mypy-play.net/?mypy=1.18.2&python=3.11&gist=6e03609385c9b451e208003e9fe28c5a

Pyright has no complaints:
https://pyright-play.net/?pythonVersion=3.11&strict=true&code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAqiApgGoCGIAUFQMYA25Azk1AIIBcVUPUCzTGgwFQAQl158BNKgSgBeQiQogAFACIC6gDRQARmACuKACby2UAD5iAlDJPFgUWiGLkYxVQGtGLDrBIAbQIAXRsoAFoAPkIJXlcYQxAUKB8BVRsgA

mypy Playground

The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.

#

I can work around it with a cast (which Pyright then complains is redundant)

restive rapids
rare scarab
#

Otherwise klass: Callable[[], T] will work

unreal urchin
unreal urchin
rare scarab
summer berry
rare scarab
#

Do you want A or B including subclasses?

summer berry
#

Yes

#

Oh actually I misread

#

I think that will work since I don't need to pass a union as the input

#

One of the SO answers says that if I don't use bound=Union, then subclasses aren't allowed. But I tried it and it passes

rare scarab
#

Consider using the Callable factory approach.

summer berry
#

Okay that works

#

Should this typevar be covariant

rare scarab
#

when in doubt, let the type checker tell you

summer berry
#

Oh right it infers variance if unset?

rare scarab
#

or it will tell you the typevar should be covariant, contra, etc

unreal urchin
trim tangle
stiff acorn
#

Desperately want 3.12 to be EOL so I can use the new syntax

#

Unfortunately that's quite far away

soft matrix
#

is there a way to extend builtins.py's types in a typechecker?

rare scarab
#

currently 3.12.

stiff acorn
#

sounds nice

stiff acorn
severe owl
stiff acorn
severe owl
#

Neat, looks like a nice improvement

stiff acorn
#

Surprisingly, vs code still doesn't syntax highlight PEP 695

#

multi billion dollar corporation vs 1 PEP

restive rapids
# stiff acorn Surprisingly, vs code still doesn't syntax highlight PEP 695

https://github.com/microsoft/vscode/blob/main/extensions/python/syntaxes/MagicPython.tmLanguage.json#L2-L6
-> https://github.com/MagicStack/MagicPython/pull/270
3 years no changes 💀

GitHub
add rudimentary support for PEP 695 by asottile · Pull Request #27...
GitHub
vscode/extensions/python/syntaxes/MagicPython.tmLanguage.json at ma...

Visual Studio Code. Contribute to microsoft/vscode development by creating an account on GitHub.

stiff acorn
#

Tragic

feral wharf
#

is this intended?

code

# code
V = TypeVar('V', bound='BaseView', covariant=True)

if TYPE_CHECKING:
    from typing_extensions import TypeVar

    ItemT = TypeVar('ItemT', bound='Item', covariant=True, default=Item)
else:
    ItemT = TypeVar('ItemT', bound='Item', covariant=True)


class Label(Item[V], Generic[V, ItemT]):
  ...

error

Traceback (most recent call last):
  File "x\test.py", line 5, in <module>
    class MyModal(discord.ui.Modal):
    ...<21 lines>...
            reveal_type(self.text.component)
  File "x\test.py", line 6, in MyModal
    fruit: discord.ui.Label['MyModal'] = discord.ui.Label(
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "x\Python\Python313\Lib\typing.py", line 432, in inner
    return func(*args, **kwds)
  File "x\Python\Python313\Lib\typing.py", line 1242, in _generic_class_getitem
    args = prepare(cls, args)
TypeError: Too few arguments for <class 'discord.ui.label.Label'>; actual 1, expected at least 2
#

I feel like it shouldn't throw an error on runtime?

#

when it works as intended at type checking

trim tangle
#

and yes, it intended behavior to error if the wrong number of type parameters is supplied

#

There's a good chance you're already depending on typing-extensions, so might as well depend on it

feral wharf
#

oh that makes sense

trim tangle
#

actually no, aiohttp doesn't require typing_extensions anymore

#

but if your library intends to use annotations for runtime effects, I think it's totally fair to depend on typing-extensions

feral wharf
#

it does not unfortunately

soft matrix
#

just throw the entire thing in quotes then

#

fruit: 'discord.ui.Label[MyModal]' = discord.ui.Label(

#

or future annotations

#

or patch the typevar at runtime 😎

feral wharf
#

I don't think the maintainer of discord.py will appreciate that KEKW

trim tangle
#

Any idea how overloads were supposed to interact with ParamSpec? Neither typing.python.org nor the paramspec PEP mention this

#

code on the basedpyright playground: link

#

pyright says:

Type of "f1" is "Overload[(x: int) -> (list[Literal['aaa']] | None), (x: str) -> (list[Literal['aaa']] | None), (x: int) -> (bytes | None), (x: str) -> (bytes | None), (x: int) -> (str | None), (x: str) -> (str | None), (x: int) -> (list[Literal['bbb']] | None), (x: str) -> (list[Literal['bbb']] | None)]"
Type of "f2" is "Overload[(fizz: None, buzz: None) -> (int | None), (fizz: None, buzz: None) -> (str | None), (fizz: bytes, buzz: bytes) -> (int | None), (fizz: bytes, buzz: bytes) -> (str | None), (fizz: str, buzz: str) -> (int | None), (fizz: str, buzz: str) -> (str | None), (fizz: int) -> (int | None), (fizz: int) -> (str | None)]"

apparently it "zips" the parameter signatures of one overloaded with the return types of another overloaded function, which makes no sense
mypy says:

Revealed type is "def (x: builtins.int) -> builtins.list[Literal['aaa']] | None"
Revealed type is "def (fizz: None, buzz: None) -> builtins.int | None"
``` which is similar but just the first lines
oblique urchin
#

the behavior here I think makes sense, your example is quite complex with two different overloads that get their return types swapped

trim tangle
#

The functions work by filling up each other's queue on call

oblique urchin
#

right, so you return a function that takes whatever arguments the one function takes and returns whatever the other returns

#

isn't that what pyright's inferred return type says?

trim tangle
#

The correct return type for the first function would be ```py
type BUnion = list[Literal["aaa"]] | list[Literal["aaa"]] | bytes | str

Overload[(x: int) -> BUnion, (x: str) -> BUnion]```

#

Perhaps there's too much code, here's a simplified version ```py
def cross[**P, **U, A, B](_fn1: Callable[P, A], _fn2: Callable[U, B]) -> tuple[Callable[P, B | None], Callable[U, A | None]]:
...

@overload
def foo(x: int) -> int: ...
@overload
def foo(x: str) -> str: ...
def foo(x: int | str) -> int | str: return 69

@overload
def bar(a: int, b: int, c: int) -> range: ...
@overload
def bar(a: int) -> list[int]: ...
def bar(*args: object, **kw: object) -> object: ...

f1, f2 = cross(foo, bar)
reveal_type(f1)

Overload[(x: int) -> (range | None), (x: str) -> (range | None), (x: int) -> (list[int] | None), (x: str) -> (list[int] | None)]
``` the argument signature (x: int) from foo is linked to the first return signature of bar because they are at the same index in the respective overload lists. If you swap the overloads in bar (which obviously defines the same signature) it will then say (x: int) -> (list[int] | None)

#

there's a second overload involving (x: int) but it is unreachable because there's already an overload with (x: int)

oblique urchin
trim tangle
# 
reveal_type(f1(42))
``` this shows `range | None` in pyright
oblique urchin
#

might be that pyright still doesn't fully implement the overload spec here

#

oh wait no, ordering does matter here

#

yeah, in this case the later overload is fully obviated

trim tangle
#

I wonder how they solved it in typescript

hearty shell
#

If I wanted to internally enforce a semantic invariant on a mutable type, would that be a sound usecase for Allow subclassing without supertyping ? I had a read through and a lot of the examples were deemed unsound but also the conversation also mixed feasibility and practical concerns. Just if you have thought about that before and reached a conclusion, not asking for someone else to re-read the discussion ^^

indigo halo
#

I have this:

class BaseModel[T: TPModel](ComparableMixin, ReprMixin, StrMixin, PydanticBaseModel): ...

class Court(BaseModel[TPCourt]): ...

and this yields:

tptools/court.py:14: in <module>
    class Court(BaseModel[TPCourt]):
                ^^^^^^^^^^^^^^^^^^
.direnv/python-3.13/lib/python3.13/site-packages/pydantic/main.py:860: in __class_getitem__
    raise TypeError(f'{cls} cannot be parametrized because it does not inherit from typing.Generic')
E   TypeError: <class 'tptools.basemodel.BaseModel'> cannot be parametrized because it does not inherit from typing.Generic

who is confused about the new syntax here?

trim tangle
#

Or is it your own BaseModel?

#

wait, it's your own BaseModel

#

can you print BaseModel.mro()?

indigo halo
#

yes, sorry, my own base model

# 
[<class 'tptools.basemodel.BaseModel'>, <class 'tptools.mixins.comparable.ComparableMixin'>, <class 'tptools.mixins.repr.ReprMixin'>, <class 'tptools.mixins.str.StrMixin'>, <class 'pydantic.main.BaseModel'>, <class 'typing.Generic'>, <class 'object'>]
trim tangle
rough sluiceBOT
#

pydantic/main.py lines 859 to 860

if not cls.__pydantic_generic_metadata__['parameters'] and Generic not in cls.__bases__:
    raise TypeError(f'{cls} is not a generic class')```
trim tangle
#

Sounds like it expects that you explicitly include typing.Generic in the bases when you define BaseModel

indigo halo
#

but that doesn't work together with new syntax.

#

anyway, i'll give this a shot. Thank you!

trim tangle
#

there should be a Generic in there

oblique urchin
#

the new syntax should inject Generic into the __bases__

# 
... 
>>> C.__bases__
(<class 'typing.Generic'>,)
trim tangle
#

yeah

indigo halo
# 
(<class 'tptools.mixins.comparable.ComparableMixin'>, <class 'tptools.mixins.repr.ReprMixin'>, <class 'tptools.mixins.str.StrMixin'>, <class 'pydantic.main.BaseModel'>, <class 'typing.Generic'>)
#

so yes

#

i have to run but I will look into this tomorrow.

#

thanks folks

ocean dust
#

What ppl use these days for type checking. Mypy? Ty? Or something else?

trim tangle
#

If you're starting a new project, I would recommend basedpyright

ocean dust
trim tangle
#

Unfortunately, pyright is maintained by Microsoft. Microsoft decided to keep some nice features out of pyright and put them into Pylance, which is a closed-source VSCode extension. pyright got forked into basedpyright, with some pylance features added into it along with a few other improvements.
basedpyright also provides a sensible PyPI package. Pyright is written in TypeScript so you need to do have node.js to make it work (which the PyPI package bundles)

#

Hopefully ty gets production-ready in the near future and we'll all switch to it...

ocean dust
#

So for production code is mypy ? Do I get it right?

trim tangle
ocean dust
#

It's an existing project but we haven't type checked so far. We use type hints. But there may be errors 🙂 as we didn't check...

trim tangle
meager slate
#

Basedpyright uses strict type checking by default so you need to configure it to your liking from the very beginning

trim tangle
#

yeah, make sure to check the docs for all the configuration options

ocean dust
#

We probably want to start from basic. I enabled basic py right checks in VSCode and it shows a lot of errors 🙂

trim tangle
#

Yeah, whenever you add a new linter it's going to scream

ocean dust
#

Zed (editor) uses basedpyright as the primary language server. So it may be the new standard by the looks of it.

#

Until ty is out of beta that is 🤣

trim tangle
#

marimo also uses it

indigo halo
#

So how do I get __parameters__ into this generic base class?

#

lemme make a test case

#

I cannot of course ☹️

#

I'

#

ll push the code when I am done with the refactor and then can provide a link

#

found the problem. I was overriding __init_subclass__ in one of the ancestors and not calling super() in it.

stiff acorn
#

Yes

# 
def f(s: Iterable[str]) -> None:
    pass

f("string")
f(["a", "b"])

This will type check without any errors

oblique urchin
#

yes

feral wharf
#

list[T] | tuple[T, ...] I think would be the solution for that

  • generator and idk but yes
trim tangle
#

speaking of...

# 
from collections.abc import Iterable

type Tensor = float | Iterable[Tensor]

foo: Tensor = "banana"
``` mypy says `str` is a Tensor, pyright says it's not
#

I have to agree with mypy here

restive rapids
trim tangle
#

oh right, I remember now

rare scarab
restive rapids
scarlet dirge
#

Is picking any type hinting option that doesn't involve importing something (unless necessary) from typing module a better option?
For example,
int | None > Optional[int]
int | str > Union[int, str]
type Something = x | y | z > Something: TypeAlias = x | y | z
list[str] > List[str]
and so on

#

Unless I also have to consider backward compatibility (I don't care)

trim tangle
spiral fjord
#

Some of the ones on the right are deprecated iirc

trim tangle
#

just List

#

it doesn't raise a deprecation warning because... reasons

scarlet dirge
#

Yes that's why, whatever code ends up generated by LLMs use right ones and I see stack overflow answers mentioning left ones are introduced in newer versions and it is preferable to use that

#

But I'm not sure where to find guidelines on type hinting so I wanted a rule of thumb sort of

trim tangle
#

It's basically the wild west, you can do whatever you want, technically...

scarlet dirge
languid aspen
trim tangle
#

You can use pyupgrade or some rules from Ruff to upgrade to a newer spelling

scarlet dirge
#

List example is fine, but for others, both works but left one is slightly more preferred is what I understood

trim tangle
#

it is typically more concise and has the same meaning

trim tangle
scarlet dirge
indigo halo
#

if I have a type-parametrised class, can I somehow get at the actual type at runtime?

class Foo[T]:
    @classmethod
    make_T(cls) -> T:
        return T()
#

that obviously won't work. But what can I use instead of T()?

soft matrix
#

you can't really I was working on a proposal for this ages ago

#

the best you could do is your own GenericAlias which overwrites class_getitem and sets the type param or frame hacks

#

this

#

oh wait no wrong one

#

though idk how out of sync it is with my local version

restive rapids
#

honestly i dont like it
required type parameters should be regular parameters, with a type[...] type
adding runtime magic specifically for storing explicitly provided typevars is weird

soft matrix
restive rapids
soft matrix
#

well yes but its not hard to imagine a case which does

restive rapids
stiff acorn
soft matrix
restive rapids
#

my point is that the proposal only works with required type parameters, like, ones you explicitly provide in your code at every call site (as python itself is obviously not going to do type inference), and in those cases you can use a regular parameter just as well
you're just changing the syntax from providing them as regular ones to []
there's no duplication in usage, only in declarations, as you need both a typevar and a parameter that binds it

def f[T]() -> F[T]:
  return e(T)
f[t]
# ->
def f[T](t: type[T]) -> F[T]:
  return e(t)
f(t)
indigo halo
#

I am unsure if I am on a garden path in the wrong direction regarding the specialisation of a type hierarchy I am designing. The base class Match is type-parametrised with e.g. the type of the tournament entry, i.e. class Match[EntryT: Entry](…): .... So far, so good.

Now, I have a use-case in which I need a specialised Match, let's call it SpecMatch. More concretely, I need to provide a specific Pydantic model_serializer for SpecMatch, and a SpecMatch also uses EntryT=SpecEntry. So I thought: class SpecMatch(Match[SpecEntry): .... And while this works at runtime, mypy rightfully says:

… "SpecMatch" must be a subtype of "Match[Entry]"

which is is obviously not.

I guess I could inject the serialization logic via Pydantic contexts, but I was wondering if there isn't another way for me to achieve two things at once:

  1. Define the base class type parameter to be a specialised class for instances of the child class;
  2. Override some behaviour of the base class in the specialised class.

Does this make sense?

#

I made a test case that highlights the problem I am having:

from typing import Any
from pydantic import BaseModel, model_serializer


class DataType(BaseModel):
    x: int

    @model_serializer(mode="plain")
    def _model_serializer(self) -> dict[str, Any]:
        return {"x": self.x}


class SpecDataType(DataType):
    @model_serializer(mode="plain")
    def _model_serializer(self) -> dict[str, Any]:
        return {"x": 2 * self.x}


class MyBase[DataT: DataType = DataType](BaseModel):
    data: DataT

    @model_serializer(mode="plain")
    def _model_serializer(self) -> dict[str, Any]:
        return {"data": self.data, "desc": "just x"}


class MySpec(MyBase[SpecDataType]):
    @model_serializer(mode="plain")
    def _model_serializer(self) -> dict[str, Any]:
        return {"data": self.data, "desc": "specialised X (should be double)"}


class App[BaseT: MyBase = MyBase](BaseModel):
    model: BaseT


class SpecApp(App[MySpec]): ...

Pyright says:

Diagnostics:
1. Type "MySpec" cannot be assigned to type variable "BaseT@App"
     Type "MySpec" is not assignable to upper bound "MyBase[DataType]" for type variable
   "BaseT@App"
       "MySpec" is not assignable to "MyBase[DataType]"
         Type parameter "DataT@MyBase" is invariant, but "SpecDataType" is not the same as
   "DataType" [reportInvalidTypeArguments]

and mypy:

Type argument "MySpec" of "App" must be a subtype of "MyBase[DataType]" [type-var]

craggy sage
pastel egret
#

That seems problematic - if it’s wrong, how do you specify that it should be mutable? The only other place I can think of where a name has meaning is with star-imports and underscored private names. But there you can just explicitly import or set __all__.

craggy sage
#

Hm, wouldn't one use type ignores for such cases? 🤔 Note this is not runtime behaviour, only the static checking.

inner sedge
#

how is this channel a thing? 👀

rare scarab
#

because type hinting is a complex topic and may be unfamiliar to many python developers

indigo halo
#

This doesn't seem right:

@pytest.fixture
def draw1(tpdraw1: TPDraw) -> Draw:
    return Draw.from_tp_model(tpdraw1)

where Draw inherits from Base and Base.from_tp_model returns typing.Self

Mypy says:

error: Incompatible return value type (got "Self", expected "Draw") [return-value]

Isn't that the entire point of Self?

brazen jolt
#

that's odd, I can't really reproduce it, are you sure you imported the right Self? If yes, can you try to put together a minimal repro?

indigo halo
#

from typing import TYPE_CHECKING, Any, Never, Self, overload

# 
class BaseModel[T: TPModel | None = None](
    ComparableMixin, ReprMixin, StrMixin, PydanticBaseModel
):
    @singledispatchmethod
    @classmethod
    def from_tp_model(cls, tpmodel: T) -> Never | Self:
        if tpmodel is None:  # pragma: nocover
            # thanks to singledispatch, we should never reach this
            raise NotImplementedError("No TPModel to instantiate from")
        return cls.model_validate(tpmodel.model_dump())

    @from_tp_model.register
    @classmethod
    def _(cls, _: None) -> Never:
        raise NotImplementedError("No TPModel to instantiate from")
soft matrix
#

I also thought there was an example in here where they needed to pass the type and the value for some reason

terse sky
# 
class Bar:
    x: int | None


def glug(b: Bar):
    ...

def foo(b: Bar) -> int:
    assert b.x is not None
    glug(b)
    return 5 + b.x
#

this has probably been asked before, maybe even me a long time ago

#

but I'm surprised to see this type checks?

#

what's the reasoning?

restive rapids
# terse sky what's the reasoning?

"practicality beats purity", i guess, or just not wanting to deal with the complexity and not producing unwanted false type errors
"purity"-wise for this to be safe we'd need to know that glug(b) cant set b.x to None
the "simple" thing would be to discharge all narrows when calling something which can affect mutable state

terse sky
#

I guess I just don't understand why this is more practical, I guess

#

it seems like a very real thing, that could happen in real codebases

#

yes, I know. that's why I'm saying this shouldn't type check.

feral atlas
#

Another similar example is

  assert some_global is not None
  glug() # might change global
  return 5 + some_global
trim tangle
#

I also think it is silly that it's allowed

feral atlas
#

As I recall typescript also allows it, flow doesn't.

trim tangle
#

it should be reserved for indexes of tuples and attributes of known immutables (like frozen dataclasses and NamedTuples)

restive rapids
trim tangle
#

On the other hand, if someone changed the value of this object while you weren't looking, how do you know other decisions that you're making are still valid?

#

e.g.: ```py
@dataclass
class User:
name: str
email: str | None
is_admin: bool

def my_func(u: User):
if u.email is not None and u.is_admin:
# do something long
assert "@" in u.email
``` u.email not being a string anymore is a possibility, but so is u.is_admin not being true anymore

feral atlas
#

You could make the assertion at the beginning of the function, do some valid thing based on the inferred type, then accidentally do an invalid thing at the end of the function

#

Where you should really assert again

trim tangle
#

ah, you mean when looking at the object operated on within the function

#

yeah, that's possible

terse sky
terse sky
trim tangle
#

true

terse sky
#

It's weird how loosey goosey python is with this

#

it honestly doesn't seem like there's any big negative to doing this correctly

#

i mean of course now it would be a big backwards compatibility break

#

but originally

feral atlas
#

at least this doesn't seem to be allowed:

    assert b.x is not None
    def ret() -> int:
        return b.x + 5
trim tangle
#

iirc the pyright maintainer marked this as "as designed"

#

(i.e.: won't fix)

jolly cipher
feral atlas
trim tangle
feral atlas
trim tangle
#

I don't think so

feral atlas
#

Sure, that's the status quo

trim tangle
#

What's the appropriate term for annotations that are used for typing analysis purposes? Is it "type hints" or "type annotations" or something else?

feral atlas
#

The spec calls these "type annotations"

trim tangle
#

and there's also typing.get_type_hints

#

from what I gathered the two are synonyms?

#

or maybe there's a subtle distinction?

feral atlas
#

Are the terms defined anywhere?

trim tangle
#

"annotation" on its own is a more generic term for the thing that was added to functions back in Python 3.0. But "type annotation" seems to be a synonym for "type hint"

#

tbh "type hint" might be a better term because it doesn't always appear in an "annotation"

#

a type hint can appear in the first argument to typing.cast, in a generic class parameter (e.g. list[int]()) or in a type alias

feral atlas
#

Isn't what appears in a cast just a "type"?

trim tangle
#

hm, yes, it does define it to be an annotation

oblique urchin
#

"type hint" gets used informally but not sure we've ever really defined it

chrome plover
#

Help me 5 minn plsss

chrome plover
trim tangle
trim tangle
#

Is there still no way to make this invalid? ```py
min(1, "a")

polar aurora
#

I personally can't think of a way that doesn't wrap it in something that has a runtime cost, but I'm not an expert on this so I'd be delighted to be wrong.

#

It's possible in Python that this is external spec language territory

fiery canyon
#

Do y'all think these overloads are an overkill or rather are a good thing to have

image.png
soft matrix
#

They seem good to me

trim tangle
#

Each event class could store its code/number/whatever in a class variable

fiery canyon
trim tangle
#

Is it a library that has to maintain backwards compatibility? Or is it an internal thing?

fiery canyon
trim tangle
#

yes

fiery canyon
trim tangle
#

wdym?

fiery canyon
#

By backwards compatibility you mean what

trim tangle
#

If my code works with library version v1.0, but not with v2.0, it means that version 2.0 is not backwards compatible with 1.0

#

Libraries usually try to avoid breaking changes, they want users to be able to update to the next version of the library with no breakages

fiery canyon
hasty phoenix
#

Is there a way to specify inheritance dynamically in a way that static type annotators are able to follow:

if HAVE_PANDAS:
    MixinCls = AdditionalPandasMixin
else:
    MixinCls = NoPandasMixin
class A(MixinCls):
    pass

However, the type checker isn't able to follow this. It's ok that the type checker follows one of those paths.

restive rapids
hasty phoenix
#

Nice. Let me try

hasty phoenix
#

So I think it might only be able to use this if it can be resolved statically

restive rapids
hasty phoenix
#

This works both runtime and with static type checker

try:
    import pandas as pd
    _HAVE_PANDAS = True
except ImportError:
    _HAVE_PANDAS = False

if TYPE_CHECKING:
    HAVE_PANDAS: Final[bool] = True
else:
    HAVE_PANDAS: Final[bool] = _HAVE_PANDAS
#

It works! Thank you

#

Not sure this is typing related or something else: In a class with __getattr__() implemented, is there some way to specify to VS Code not to navigate to it, but rather to directly to the property/function?

#

Can I hide the __getattr__() method for the type checker?

#

Found this discussion: https://discuss.python.org/t/should-structural-subtyping-consider-getattr/85062

Discussions on Python.org
Should structural subtyping consider `__getattr__`?

Consider the following code snippet: from typing import Protocol class P(Protocol): x: int def f(p: P) -> None: ... class C: def getattr(self, name: str) -> int: ... f(C()) # Should this be an error or not? Different type checkers have diverging behaviors today: Mypy and Pyre1 thinks that this is fine, and Pyright reports a...

#

Solution:

class A:
    if not TYPE_CHECKING:
        def __getattr__(self, name):
            ...

(I didn't even know that if operators where permitted on class level. Learned something new.)

restive rapids
spiral fjord
#

top level await when

feral wharf
#

Its really annoying that it thinks everything exists when the class implements getattr

hoary fiber
hasty phoenix
# 
class Mixin(FakeA):
    """ Needs to be mixed in with A """
    def f(self):
        print(f"{self.a=}")

class A(Mixin):
    """ Very large class with lots of methods """
    a = 42

Is there a way to tell the type checker that Mixin has all of A available? Which is emulated with the FakeA class in this case? The best would be to let FakeA be a Protocol, but A is very big so FakeA becomes very repetitive. Runtime-wise Mixin without FakeA works with no problem, so this is just about how to specify it to the type checker.

spiral fjord
#

Mypy recommends doing it like this

dreamy ridge
#

Hi! I hope this is the right place to ask.

I am importing a library that assigns most of the class methods dynamically at runtime using (I believe) decorators. Besides the fact that this is making type checking pretty impossible, my IDE can't "see" the dynamically assigned methods and so I am getting piles of "method not found" errors.

Is there a best practice for handling this? I've starting writing my own pyi file and it works, but it's a bit time-consuming. I'm coming back to Python after a while so while I'm aware of tools like stubgen I'm not sure if that's the most modern approach

marsh gorge
#

Hello I am attempting to write a parser combinator library in python with the goal of passing mypy strict mode type checking. I have a funny error:
tests/test_sexpr.py:43: error: Argument 2 to DelimitedBy" has incompatible type "Repeat[str, list[Sexpr] | Atom | String | Int]"; expected "Parser[str, list[Sexpr] | Atom | String | Int]" [arg-type]

The interesting part is that Repeat is a Parser, it inherits and conforms to the Parser protocol

rough sluiceBOT
marsh gorge
#

the type error happens near the bottom of the test file

trim tangle
marsh gorge
#

not really but i have heard of it

trim tangle
#

In your case, Parser[I, O] is invariant in O, but you should probably make it covariant

marsh gorge
#

is there a way to do that with the new generics syntax or do i need a TypeVar

#

ah the docs mention this i think

trim tangle
#

New-style type variables infer variance. However, the core issue is that Parser cannot be covariant right now

#

(because of the signatures of Parser's methods)

#

E.g. if you have this: ```py
class Foo1T:
def bar(self, x: T) -> T:
...

class Foo2T:
def bar(self) -> list[T]:
...
``` Foo1 and Foo2 cannot be covariant

marsh gorge
#

interesting

trim tangle
#

do you see why that's the case in this example?

marsh gorge
#

its because the signatures are different right?

#

and the signatures are not compatible

trim tangle
#

I'm not talking about the relation between Foo1 and Foo2

marsh gorge
#

oh

trim tangle
#

Foo being covariant means that if A is a subtype of B, then Foo[A] is a subtype of Foo[B]

#

An example of a type that's not covariant is list. list[int] is not a subtype of list[int | str] (or vice versa)

marsh gorge
#

okay i think i can see that

#

its a little confusing but ill read more about it

#

thank you btw

trim tangle
marsh gorge
#

yes but i was pretty confused by it

trim tangle
#

I'm working on a tutorial on generics right now and I'll have to explain variance somehow

marsh gorge
#

i would be interested in seeing the tutorial for sure. im familiar with rust and c++ generics but somehow ive never had to worry about it there, or at least not that i was aware of

#

except with rust lifetimes thats one area it pops up

trim tangle
#

Rust doesn't have subtyping in the same way Python has

marsh gorge
#

ah yup that would explain it

fiery canyon
marsh gorge
#

thank you for helping with this btw

trim tangle
trim tangle
#

What are AnyEvent and HandlerFuncDecorator?

fiery canyon
#
  1. The _AnyEvent class
  2. An event object, subclassing _AnyEvent
  3. "AnyEvent" TypeVar
  4. HandlerFuncDecorator
  5. Wrong type annotation. I need to indicate it should be a class var of an event object.
  6. "Frontend"
image.png image.png image.png image.png image.png image.png
trim tangle
#

can you post it as text

#

are you using the old TypeVar syntax for compatibility with python <3.12?

fiery canyon
#

Uh no i thought thats the right usage

#

Im using py 3.10 though

trim tangle
fiery canyon
#

type being what

trim tangle
#

it's just the built-in type

#

type[Foo] means the class Foo or any of its subclasses

#

e.g. ```py
t1: type[int] = int
t2: type[int] = bool

fiery canyon
#

Oh nice I see

#

Is there a problem with my other syntax

trim tangle
#

TypeVar and TypeAlias were replaced with nicer built-in syntax in Python 3.12 ```py
S = TypeVar("S", bound=int)

def foo(s: S) -> list[S]:
...

Foo: TypeAlias = Callable[[S], list[S]]

->

def foo[S: int](s: S) -> list[S]:
...

type Foo[S: int] = Callable[[S], list[S]]

#

also, I'd recommend using stricter type checker settings, it would highlight issues like forgetting to pass parameters to a generic type

fiery canyon
#

hmm I'll see, kinda wanna get used to Standard first

#

Thanks

#

I'm planning to switch to Py 3.13 when it comes out

#

For now I use 3.10.6

trim tangle
#

Python 3.14 is already out

fiery canyon
#

Damn I'm not caught up

marsh gorge
# 

I = TypeVar("I")
O = TypeVar("O", contravariant=True)
O2 = TypeVar("O2", contravariant=True)


class Parser(Protocol):

    def parse(self, input: I) -> tuple[I, O]:
        pass

    def __add__(self, other: Parser) -> Add:
        return Add(self, other)


@dataclass
class Add(Parser):
    a: Parser
    b: Parser

    def parse(self, input: I) -> tuple[I, tuple[O, O2]]:
        rest, a = self.a.parse(input)
        rest, b = self.b.parse(rest)
        return (rest, (a, b))```
Now it doesnt like that the output for `Add` is different from the output of `Parser` even though the output is generic?
#

i suspect im confused about how to use the old style type parameters

trim tangle
restive rapids
#

if you did want to, then Parser should declare its type parameters by "inheriting" from typing.Generic, because currently the type variables are bound in methods
and then Add would have them too

fiery canyon
trim tangle
fiery canyon
#

That's cursed

marsh gorge
#

at least without assigning to a local var with an underscore

restive rapids
marsh gorge
trim tangle
#

you probably meant to use covariant

marsh gorge
#

oh do i have it backwards

#

jeez im sorry for being so slow at this

restive rapids
#

when you use the new pep695 typevars, the variance will be inferred (generally to what you'd want)

#

i would probally use ABC instead of protocol here, since you'll inherit from it anyways to get the __add__

# 
from abc import ABC, abstractmethod
from dataclasses import dataclass

class Parser[I, O](ABC):
    @abstractmethod
    def parse(self, input: I) -> tuple[I, O]:
        ...
    def __add__[O2](self, other: "Parser[I, O2]") -> "Add[I, O, O2]":
        return Add(self, other)

@dataclass
class Add[I, O1, O2](Parser[I, tuple[O1, O2]]):
    a: Parser[I, O1]
    b: Parser[I, O2]

    def parse(self, input: I) -> tuple[I, tuple[O1, O2]]:
        rest, a = self.a.parse(input)
        rest, b = self.b.parse(rest)
        return (rest, (a, b))
trim tangle
# marsh gorge i didnt see a way to change the variance with the new style parameters maybe i o...

Here is why list[int] is not a subtype of list[int | str]:

def do_things(xs: list[int | str]) -> None:
    do_things.append("banana")

things: list[int] = [1, 2, 3]
do_things(things)
#^ if this was allowed, then things` would contain a string now
``` essentially, variance depends on the methods a class has. ```py
class Foo[T]:  # we want Foo to be covariant
    def m1(self) -> T: ...   # ok, T is in the output position
    def m2(self) -> tuple[T, ...]: ...   # ok, tuple[_, ...] is covariant 
```  Why is this okay? Let's assume that `Sub` is a subtype of `Super` and compare the signatures.
```py
# Foo with T = Sub 
    def m1(self) -> Sub: ...
    def m2(self) -> tuple[Sub, ...]: ...
# Foo with T = Super
    def m1(self) -> Super: ...
    def m2(self) -> tuple[Super, ...]: ...
``` `() -> Sub` is a subtype of `() -> Super`, and `() -> tuple[Sub, ...]` is a subtype of `() -> tuple[Super, ...]`.

Here's a method that would make our class not covariant:
```py
class Foo[T]:  # we want Foo to be covariant
    def m3(self, arg: T) -> T: ...
``` Why? Let's see an example: ```py
def func(x: Foo[int | str]) -> None:
    print(x.m3(1))
    print(x.m3("banana"))

foo_int: Foo[int] # imagine that we got it from somewhere
# foo_int.m3 expects an `int`.

func(foo_int)  # But in this function, x.m3 must expect `int | str`
marsh gorge
#

@restive rapids the reason i need to change the variance is because of this parser i think (if i understand correctly)

@dataclass
class Repeat[I, O](Parser[I, list[O]]):
    parser: Parser[I, O]
    key: slice[int, int, None] | int

  def parse(self, input: I) -> Tuple[I, list[O]]```
restive rapids
#

i'd just not use a list return type, use collections.abc.Sequence instead
its not particularly important that it returns specifically a list (which is mutable -> invariant)

marsh gorge
#

interesting

#

can you still index into the seq?

restive rapids
#

yes

trim tangle
marsh gorge
#

so even with Sequence as the return type of Repeat im getting the same error?

marsh gorge
trim tangle
#

If you have just one method that e.g. returns list[T], the class cannot be covariant anymore

#

Variance isn't a toggle that you choose for each class, it's just a consequence of what methods a class has

marsh gorge
#

so what is the correct fix here for me? I was originally going to use the old style type vars to control the variance, but if thats not needed im not sure how to actually do it

restive rapids
# 
from abc import ABC
from collections.abc import Sequence
from dataclasses import dataclass

class Parser[I, O](ABC):
    def parse(self, input: I) -> tuple[I, O]:
        ...
    def __add__[O2](self, other: "Parser[I, O2]") -> "Add[I, O, O2]":
        return Add(self, other)

@dataclass
class Add[I, O1, O2](Parser[I, tuple[O1, O2]]):
    a: Parser[I, O1]
    b: Parser[I, O2]

    def parse(self, input: I) -> tuple[I, tuple[O1, O2]]:
        rest, a = self.a.parse(input)
        rest, b = self.b.parse(rest)
        return (rest, (a, b))

#
@dataclass
class Repeat[I, O](Parser[I, Sequence[O]]):
    parser: Parser[I, O]
    key: slice[int, int, None] | int
    def parse(self, input: I) -> tuple[I, Sequence[O]]:
        ...

is just fine

marsh gorge
#

okay lets try that

#

still getting this error
ests/test_sexpr.py:44: error: Argument 2 to "DelimitedBy" has incompatible type "Repeat[str, Sequence[Sexpr] | Atom | String | Int]"; expected "Parser[str, Sequence[Sexpr] | Atom | String | Int]" [arg-type]

rough sluiceBOT
trim tangle
#

the new variance inference is infuriating because it doesn't give a way to debug why you don't have the variance you want

marsh gorge
#

oh they used collections.abc Sequence

#

i wonder if thats why mine isnt working

#

no that still didnt work hmm

trim tangle
# marsh gorge no that still didnt work hmm

here's a janky way to debug this:

class Super: pass
class Subc(Super): pass


def f(r1: Parser[str, Subc]) -> None:
    _r2: Parser[str, Super] = r1
``` you should see an error on the assignment to the effect of

Type "Parser[str, Subc]" is not assignable to declared type "Parser[str, Super]"
"Parser[str, Subc]" is not assignable to "Parser[str, Super]"
Type parameter "O@Parser" is invariant, but "Subc" is not the same as "Super"

then you comment out methods on `Parser` until the error goes away... that way you know the last method you commented was problematic
#

For example: with just parse and map, there was no issue. When you add __add__, the type becomes invariant

#

That must mean that Add[I, O, O2] is not covariant in O

#

Ah... that's because dataclasses don't support having covariant attributes even when it's frozen

restive rapids
#

i must say that it would be less code and less fighting with the typesystem if you didnt use parser combinators
s-exprs are easy to parse with a couple of loops and conditions

marsh gorge
#

i plan on parsing gentoo atoms and stuff with it

trim tangle
#

It works but it's a very 🥴 debugging technique

marsh gorge
#

would just setting the dataclass fields to _a work?

#

little bit less typing

trim tangle
#

No, the issue is dataclass itself. Consider this:

@dataclass
class Box[T]:
    value: T
``` if you have a `box: Box[Shape]`, you can both read `box.x` and write `box.s`. So this would be a problem: ```py
def twiddle_with_box(box: Box[Shape]):
    box.value = Rectangle()

circle_box: Box[Circle]
set_twiddle_with_box(circle_box)
#

There's @dataclass(frozen=True), but for various reasons it still keeps type variables invariant

trim tangle
# marsh gorge little bit less typing

Actually, here's a better suggestion: replace concrete types in Parser's methods with Parser ```py
class ParserI, O:
@abstractmethod
def parse(self, input: I) -> tuple[I, O]:
raise NotImplementedError

def map[O2](self, f: Callable[[O], O2]) -> Parser[I, O2]:
    return Map(self, f)

def __add__[O2](self, other: Parser[I, O2]) -> Parser[I, tuple[O, O2]]:
    return Add(self, other)

def __sub__(self, other: Parser[I, object]) -> Parser[I, O]:
    return Sub(self, other)

def __or__[O2](self, other: Parser[I, O2]) -> Parser[I, O | O2]:
    return Or(self, other)

def __getitem__(self, key: int | slice[int, int, None]) -> Parser[I, Sequence[O]]:
    return Repeat(self, key)
#

now you can keep Add and such invariant

marsh gorge
#

so __getitem__ and __add__ both dont seem to like that

#

does that mean those are the issue?

trim tangle
#

Can you post the new code?

restive rapids
#

you'll also need to change Sexpr from list[Sexpr] to Sequence[Sexpr]
also Sub is wrong, it requires the thing that wont even be used as an output to match the output type with the other thing

marsh gorge
#

wait its my fault it doesnt like them

trim tangle
#

Oh yeah you need to change Sub to this ```py
@dataclass
class Sub[I, O](Parser[I, O]):
a: Parser[I, O]
b: Parser[I, object]

def parse(self, input: I) -> tuple[I, O]:
    try:
        self.b.parse(input)
        raise ParserError(input)
    except ParserError:
        return self.a.parse(input)
#

my bad

restive rapids
#

not sure why you'd want this though
is this a negative assertion? like, that b does not appear?
i'd expect you to eat b and then parse a

marsh gorge
#

brb one moment

trim tangle
#

yeah, like (?!=) in regex

restive rapids
#

generic errors also dont really work, because signatures dont show what exceptions a function raises
when you except ParserError, you wont know which output you got, and you cant isinstance with a typevar since they're not reified

marsh gorge
#

i could easily fix that and require a string error

old anchor
#

Does anyone know if there's a way properly type a function like this using TypeVarTuple?

def make_instances(types: tuple[type, ...]) -> tuple:
    return tuple(t() for t in types)
``` I was kinda hoping something like this would just work
```python
def make_instances[*Ts](types: tuple[type[*Ts]]) -> tuple[*Ts]:
    return tuple(t() for t in types)
``` but it appears that it does not. 
I've searched the PEP and docs, and they do indeed not mention this type of usage, which makes me a little sad 😦 
Am *I* missing something here, or is it the feature I want that simply does not exist (yet)?
trim tangle
#

TypeVarTuple is not very useful sadly

#

You can do something like this: ```py
class Banana[*Ret]:
def init(self: Banana[*tuple[()]]) -> None: ...
def add[T](self, t: type[T]) -> Banana[*Ret, T]: ...
def run(self) -> tuple[*Ret]: ...

i, s, b = Banana().add(int).add(str).add(bool).run()

i: int, s: str, b: bool

restive rapids
#

only relevant discussion i could find is https://discuss.python.org/t/pre-pep-considerations-and-feedback-type-transformations-on-variadic-generics/50605

Discussions on Python.org
Pre-PEP Considerations and Feedback: Type Transformations on Variad...

Preface Before I begin, please note that this is the first time that I have considered and researched the process of proposing a PEP. The purpose of this post is to get some feedback from the community on a PEP I plan to propose. The format of this post does not necessarily represent the final format of the PEP, and, based upon other PEPs I have...

trim tangle
# old anchor Does anyone know if there's a way properly type a function like this using `Type...

If you really want this, you can add overloads:

@overload
def make_instances() -> tuple[()]: ...
@overload
def make_instances[A](_a: type[A]) -> tuple[A]: ...
@overload
def make_instances[A, B](_a: type[A], _b: type[B]) -> tuple[A, B]: ...
@overload
def make_instances[A, B, C](_a: type[A], _b: type[B], _c: type[C]) -> tuple[A, B, C]: ...
@overload
def make_instances[A, B, C, D](_a: type[A], _b: type[B], _c: type[C], _d: type[D]) -> tuple[A, B, C, D]: ...

def make_instances(*args: type[object]) -> tuple[object, ...]:
    # implementation
``` that's how the built-in `zip` and `map` are defined
old anchor
old anchor
trim tangle
#

This is great if you're paid by the line of code... or by the hour

#

the overloads I mean

trim tangle
#

Can someone share some experience with teaching or learning generic functions/classes in Python? What was particularly confusing? (besides variance)

marsh gorge
#

so ive rewritten some stuff and im getting a new error:

sexpr: Parser[str, Sexpr] = DelimitedBy(
    Lit("("), Lazy(lambda: sexpr)[0:-1], Lit(")")
) | atom.map(map_atom)```
```tests/test_sexpr.py:42: error: Incompatible types in assignment (expression has type "Parser[str, Sequence[Sequence[Sexpr] | Atom | String | Int]]", variable has type "Parser[str, Sequence[Sexpr] | Sexpr]")  [assignment]
tests/test_sexpr.py:44: error: Argument 1 to "map" of "Parser" has incompatible type "Callable[[str], Sexpr]"; expected "Callable[[str], Sequence[Sequence[Sexpr] | Atom | String | Int]]"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)```
#

gah

#

doing this in rust was trivial idk why python is fighting me so much. maybe its a user error

marsh gorge
#

i understand why its complaining i think but im not entirely sure how to solve it

marsh gorge
#

i might have fixed it

# 
from pyparse import Parser, ParserError, DelimitedBy, Lit, Word, Lazy
from string import ascii_letters as letters, digits
from dataclasses import dataclass
from collections.abc import Sequence


@dataclass
class List:
    list: Sequence["Sexpr"]


@dataclass
class Atom:
    atom: str


@dataclass
class String:
    string: str


@dataclass
class Int:
    integer: int


type Sexpr = List | Atom | String | Int

alphanums = letters + digits
atom = Word(letters, alphanums)
string = DelimitedBy(Lit('"'), Word(alphanums, alphanums), Lit('"'))
integer = Word(digits, digits)


def map_atom(s: str) -> Sexpr:
    return Atom(s)


def map_string(s: str) -> Sexpr:
    return String(s)


def map_int(s: str) -> Sexpr:
    return Int(int(s))


def map_list(s: Sequence[Sexpr]) -> Sexpr:
    return List(s)


sexpr: Parser[str, Sexpr] = DelimitedBy(
    Lit("("), Lazy(lambda: sexpr)[0:-1].map(map_list), Lit(")")
) | atom.map(map_atom)
#

im not super sure why it cares if i wrap the Sequence in a class?

marsh gorge
#

I have a new problem:
I have these two parsers

@dataclass
class SeparatedList[I, O, O2](Parser[I, Sequence[O]]):
    _parser: Parser[I, O]
    _delimiter: Parser[I, O2]

    def parse(self, input: I) -> tuple[I, Sequence[O]]: ...

@dataclass
class Whitespace(Parser[str, str]):

    def parse(self, input: str) -> tuple[str, str]: ...

And im using it like this

sexpr: Parser[str, Sexpr] = (
    DelimitedBy(
        Lit("("),
        SeparatedList(Lazy(lambda: sexpr), Whitespace).map(map_list),
        Lit(")"),
    )
    | atom.map(map_atom)
    | string.map(map_string)
    | integer.map(map_int)
)

Im getting a funky error though

Found 1 error in 1 file (checked 1 source file)
#

why does it want a Never type here?

polar aurora
#

I'm missing how Whitespace qualifies as a delimiter in your definitions? Maybe I'm just being dumb today?

marsh gorge
#

the delimiter takes anything that is a Parser[I, O2]

#

and Whitespace is a Parser[str, str] right

#

there is a Parser class that they all inherit from and the idea is that you can put parsers inside of other parsers

class Parser[I, O](ABC):

    @abstractmethod
    def parse(self, input: I) -> tuple[I, O]:
        pass```
polar aurora
#

Hmm. Maybe I'm just not fully read-in on the Python type syntax, but how do your two 'names', _parser, and _delimiter, get differentiated in your constructor call?

#

Those look keyword-y to me?

#

That's the part that is throwing me.

marsh gorge
#

would it help if i pasted the whole file

polar aurora
#

It would probably help more if somebody less-ignorant answered haha

#

You might be doing it totally right and I'm seeing the wrong hint

marsh gorge
#

no its okay this is pretty esoteric i think

rough sluiceBOT
polar aurora
#

Hmm, this doesn't look wildly crazy to me, what am I missing...

marsh gorge
#

i come from rust where spewing generics everywhere is super common, i dont see much python code that is highly generic like this but maybe i dont look at enough python

#

trying to write rust in python is probably one of my issues

polar aurora
#

I don't hate it honestly, let me see if I can figure out why it's angry

#

This looks a lot like a Ruby compiler's parser we created once

#

Oh I think I see it

#

SeparatedList wants two instances and you're passing Whitespace itself, right?

#

At least I think that's what the error is poorly trying to indicate?

marsh gorge
#

oh

#

haha

#

yeah that fixed it smh thank you

#

what a silly error

polar aurora
#

the error for that should be WAY better

#

I guess that's probably hard in the current architecture

#

because there's not a great direct relationship between class Foo and an instance of foo, like there is the other way around

#

a foo obviously has a type that points right at Foo, etc

#

but there's no way to say "hey what does your constructor construct exactly, and would that fit here?"

#

maybe there is?

feral wharf
#

I don't get what or if I am doing something wrong here...

It doesn't seem to like variables:

pages = ["str", discord.Embed()]
v2_pages = ["str", discord.ui.TextDisplay[Any]("str")]

pag = BaseClassPaginator(pages, components_v2=False) # <- 
# Argument of type "list[str | Embed]" cannot be assigned to parameter "pages" of type "list[BoundPage]" in function "__new__"
#  "list[str | Embed]" is not assignable to "list[BoundPage]"
#    Type parameter "_T@list" is invariant, but "str | Embed" is not the same as "BoundPage"
#    Consider switching from "list" to "Sequence" which is covariant PylancereportArgumentType

# No overloads for "__new__" match the provided arguments PylancereportCallIssue
# core.py(169, 13): Overload 2 is the closest match
v2_pag = BaseClassPaginator(v2_pages, components_v2=True) # <- same thing except closes match is overload 1 instead of 2

# now these are fine too
pag = BaseClassPaginator(["str", discord.Embed()], components_v2=False)
v2_pag = BaseClassPaginator(["str", discord.ui.TextDisplay[Any]("str")], components_v2=True)

# these both working as expected both times
reveal_type(pag.pages)
reveal_type(v2_pag.pages)

the class: https://mystb.in/0dad0503d7d8083931

polar aurora
#

I've never used this API, but shouldn't this be discord.ui.TextDisplay[Any]("str")()?

#

but if it works fine in the second case maybe not? dunno

feral wharf
#

wdym? ui.TextDisplay[T]("string") is the same as ui.TextDisplay("string") runtime wise

polar aurora
#

I mean isn't TextDisplay("string") giving you back a string-typed TextDisplay that you then need to instantiate?

#

Or is it not a factory thingy?

indigo halo
#

Does anyone know of another project using pyproject.toml that has a directory structure like this, and which installs assets/icon.png to a sensible location on the target system, from where I can find it from within mymodule/main.py?

├── assets
│   └── icon.png
├── mymodule
│   ├── __init__.py
│   └── main.py
└── pyproject.toml
feral wharf
feral wharf
indigo halo
#

oh i am sorry, wrong channel.

#

i am an idiot

trim tangle
feral wharf
#

yeah that works, but why

trim tangle
feral wharf
#

oh fml

trim tangle
#

A better solution would be accepting Sequence[PageT] or Iterable[PageT] instead of list[PageT]. list is almost never the best annotation for a parameter

feral wharf
#

I was using Sequeue before

#

but pages="string" is also valid in that case

#

and users can pass nested lists

#

so that's also annoying to handle

trim tangle
#

could do list[PageT] | tuple[PageT, ...]

feral wharf
#

but that's ugly lol

trim tangle
#

I would probably just accept Iterable/Sequence and then raise a runtime error if the argument is a srting

#

If this is a library, there's a 100% chance someone else will run into the variance issue

#

and unfortunately type checkers don't show you a tutorial on variance when one does run into an issue, so it can be difficult to resolve (besides a # type: ignore)

feral wharf
#

fair

trim tangle
#

btw, did you know about variance before? I'm in the process of making a tutorial, so I want to gather some data on which explanations work and which don't

feral wharf
trim tangle
pastel egret
#

Only sorta, there's covariance and contravariance, but you can have both (bivarance) or neither (invariance).

feral wharf
bleak imp
#

I recently found Jelle’s unsoundness library, and have been looking through it, very interesting. One thing confuses me though. In typeis_narrow_list it says “Narrowing to a list with TypeIs is almost always unsound, because it is not possible to know the type of a list in the type system at runtime.”
The thing that I’m wondering, isn’t this only true for lists with 0 elements? If a list has 1 or more elements, wouldn’t the runtime type of the list at least be list[union of all the types of the list elements]? And if there were a hypothetical “non-empty list” type, would you be able to safely narrow it with TypeIs?

trim tangle
#

e.g.: ```py
def is_int_list(x: object) -> TypeIs[list[int]]:
"""Check if x is a list of integers."""
return isinstance(x, list) and all(isinstance(i, int) for i in x)

def foo(x: Sequence[object]):
if is_int_list(x):
x.append(42)

lst: list[bool] = [False, False, True]
foo(lst, x)

now there's a 42 in my list of bools

bleak imp
#

Hm, so then wouldn’t that apply to any generic type, not just int?

trim tangle
#

?

trim tangle
bleak imp
#

Oops, I meant list, not int

trim tangle
#

Generic types are very diverse and they can be pretty much anything

#

for a tuple, it should be fine, e.g. ```py
def is_int_tuple(x: object) -> TypeIs[tuple[int, ...]]:
...

#

(or other immutable collections)

bleak imp
#

I see, that makes sense, thanks!

trim tangle
#

Covariant frozen dataclasses, huh

#

This doesn't work with pyright on python 3.13 anymore: ```py
@dataclass(frozen=True)
class Box[T]:
label: str
value: T

def test_box_covariant(box: Box[int]) -> None:
_0: Box[int | str] = box # error: Box[int] is not assignable to Box[int | str], Box is invariant

mossy rain
rough sluiceBOT
#

optype/_core/_just.py line 144

class JustInt(Protocol, metaclass=_JustMeta, just=int):  # type: ignore[misc]  # pyright: ignore[reportGeneralTypeIssues]```
trim tangle
mossy rain
#

well i mean replacing the TypeIs[list[int]] with TypeIs[list[JustInt]], as int implies subclasses of int are accepted too, even though they should not be (e.g. bool)

trim tangle
#

That'd still be an issue, 0 and 1 are JustInts at runtime

mossy rain
#

Well, a list[Literal[anything_int]] should pass, just the int subclasses shouldn't, right?

trim tangle
mossy rain
#

oh, right

lunar dune
trim tangle
#

I guess __replace__ breaks type checker expectations in other ways too...

# 
@dataclass(frozen=True)
class Box[T]:
    name: str | None
    value: T

@dataclass(frozen=True)
class StrictIntBox[int](Box[int]):
    name: str
``` `StrictIntBox` is-a `Box[int]`, so it's supposed to accept `__replace__(name=None, value="aaa") -> Box[str]`
#

Oh and while you're here @lunar dune
According to this message: https://discuss.python.org/t/clarifying-the-typing-spec-regarding-polymorphic-protocols/60255/5 this is not a spec-compliant way to define that I'm returning a generic identity function: py def get_identity[T](name: str) -> Callable[[T], T]: ... and instead I'm supposed to make a protocol with a __call__[T](self, arg: T, /) -> T method.

Is that still the case? I can't find this in the specification. But these "rescoping" functions are used on the typing.python.org site
https://typing.python.org/en/latest/reference/generics.html#decorator-factories
https://typing.python.org/en/latest/spec/dataclasses.html#id2

lunar dune
#

I'm always here... Lurking in the shadows...

lunar dune
#

It's a bit unfortunate IMO that PEP-695 makes this kind of decorator harder to write, but it is what it is. Better Callable syntax might help quite a bit here

trim tangle
#

thanks for the explanation 👍

lunar dune
#

Np

#

And it's definitely a footgun

trim tangle
#

I ran into an issue where pyright supports using the non-spec way with dataclass_transform, but doesn't support the official way

#

will probably file an issue

lunar dune
trim tangle
#

I was confused when this worked, because the actual scoping works out like get_identity(name: str) -> Callable[forall T. [T], T]

#

(and not what I expected, which is: T is used 0 times in the parameter types, so it will be unsolved when you just call get_identity("aaa"))

lone veldt
trim tangle
hollow canopy
trim tangle
#

yeah, blue color typically means a note

#

warnings are yellow and errors are red

hollow canopy
#

so this is part of a little refactor and the GradFunction gets called with an old siganture in many places currently. Could that lead to this note?

trim tangle
#

yes, you are probably getting errors at the old call sites

hollow canopy
#

that'd explain why I get a whole list of notes. The note itself doesn't make any sense to me but let's see, will finish this and see if that solves it. thanks so far!

hollow canopy
# 
from typing import Tuple, TYPE_CHECKING

if TYPE_CHECKING:
    import jax


class Context:
    """
    Context for storing values needed in backward pass.

    Use save_for_backward() for JAX arrays.
    Store non-array values (axis, keepdims, etc.) as direct attributes.

    Example:
        ctx = Context()
        ctx.save_for_backward(input_data)
        ctx.axis = 0
        ctx.keepdims = True
    """
    def __init__(self) -> None:
        self.saved_values: Tuple[jax.Array, ...] = ()

    def save_for_backward(self, *values: jax.Array) -> None:
        self.saved_values = values

Do I really have to import jax in runtime for the type hinting? I thought on runtime the type hints are ignored?

spiral fjord
#

I think future import will fix this on old python

rare scarab
#

^ lazy evaluation of annotations was added in py3.14

hollow canopy
#

on another note, I have this context object which purpose is to be a "flexible data container" (or whatever you wanna call it)

from __future__ import annotations
from typing import Tuple, TYPE_CHECKING

if TYPE_CHECKING:
    import jax


class Context:
    """
    Context for storing values needed in backward pass.

    Use save_for_backward() for JAX arrays.
    Store non-array values (axis, keepdims, etc.) as direct attributes.

    Example:
        ctx = Context()
        ctx.save_for_backward(input_data)
        ctx.axis = 0
        ctx.keepdims = True
    """

    def __init__(self) -> None:
        self.saved_values: Tuple[jax.Array, ...] = ()

    def save_for_backward(self, *values: jax.Array) -> None:
        self.saved_values = values

So sometimes we add random new proeprties to the object. Mypy doesn't like that. I don't wanna add a # type: ignore[attr-defined] to every new property, can't I define that on the Context class itself? "Don't throw an error if we add a new property" kinda thing?

#

ah apparently I can add:


    if TYPE_CHECKING:
        def __getattr__(self, attribute: str) -> Any: ...
        def __setattr__(self, attribute: str, value: object) -> None: ...

to my Context class. Hard for me to judge if this is some hacky thing or not though.

rare scarab
#

You should consider typing your context attributes.

hollow canopy
rare scarab
#

Extend Contxt and add your type hints

#

You could do what click does and have context be class based.

hollow canopy
#

no idea what click is and what you mean by extend context - these are "dynamic attributes", also class based meaning?

rare scarab
# 
    def find_object(self, object_type: type[V]) -> V | None:
        """Finds the closest object of a given type."""
        node: Context | None = self

        while node is not None:
            if isinstance(node.obj, object_type):
                return node.obj

            node = node.parent

        return None

    def ensure_object(self, object_type: type[V]) -> V:
        """Like :meth:`find_object` but sets the innermost object to a
        new instance of `object_type` if it does not exist.
        """
        rv = self.find_object(object_type)
        if rv is None:
            self.obj = rv = object_type()
        return rv
#

Though it is limited to one object per context

hollow canopy
#

ok thanks, not sure what I should learn from that example though.

rare scarab
#

You'd use it like this. ```py
@dataclass
class Settings:
input_shape: tuple[int, ...]
axis: int
keepdims: tuple[int, ...]
input_size: int

obj = ctx.ensure_obj(Settings)
obj.input_shape = self.data.shape
obj.axis = axis
obj.keepdims = keepdims
obj.input_size = self.data.size

#

completely typesafe

hollow canopy
#

ah nope that is way too overengineered and probably slow.

rare scarab
#

assuming some types of course.

#

Then just use a dict.

#

dataclasses are only slow if you're creating thousands of them.

#

and you can add slots=True to make it more memory efficient if you do make lots of them.

#

or just omit the dataclass decorator entirely

#

Your context class might look like this. ```py
class Context:
obj: Any = None

def ensure_obj[T](self, obj_type: type[T]) -> T:
if not isinstance(self.obj, obj_type):
self.obj = obj_type()
return self.obj

hollow canopy
#

the reason for Context is simply to have a unified signature for my backward functions:

    def __add__(self, other_node) -> Node:
        if not isinstance(other_node, (int, float, jax.Array, Node)):
            raise TypeError(
                f"Unsupported type for addition with Node. Can't add {type(other_node)} to Node."
            )

        if isinstance(other_node, jax.Array):
            other_node = Node(other_node)

        if isinstance(other_node, (int, float)):
            other_node = Node(jnp.array(other_node))

        data: jax.Array = self.data + other_node.data

        ctx = Context()
        ctx.save_for_backward(self.data, other_node.data)

        result = Node(
            data=data,
            dtype=data.dtype,
            device=data.device,
            requires_grad=self.requires_grad or other_node.requires_grad,
        )

        result.grad_fn = GradFunction(
            backward_fn=add_backward,
            ctx=ctx,
            parents=[self, other_node],
        )

        return result

depending on the math. function you implement, the params are different. hence you just add them to ctx - I did think of using Context::params: dict though.

But I have 100ks or millions of those in the long term, so I'll have to rethink it anyway. Atm I mainly try to understand the problem haha and geta first version working and then go back and design it properly.

Thanks for the inputs!

hollow canopy
#

on another note:

def binary_cross_entropy_loss(logits: Node, targets: Node):
    """
    Binary cross-entropy loss for 2-class classification.
    L = (1/N) * sum(-y_i * log(p_i))

    Args:
        logits: (batch_size, 2) - raw scores for 2 classes
        targets: (batch_size,) - class indices (0 or 1)
    """
    # Convert class indices to one-hot encoding
    targets_one_hot = jax.nn.one_hot(targets.data, num_classes=2)
    probs: Node = logits.softmax(dim=1)

    per_sample_loss = (1.0) * (targets_one_hot * probs).sum(axis=1)
    loss = per_sample_loss.mean()
    ...

here we have target_one_hot * probs i.e. jax.Array * Node. Now Node::__mul__ returns Node, same with Node::sum() and the multiplication with 1. So the whole expression should end up as a Node.

If I do: per_sample_loss: Node = (1.0) * (targets_one_hot * probs).sum(axis=1) I get a type error though because apparently MyPy can't figure that out.

error: Incompatible types in assignment (expression has type "Array", variable has type "Node")

  1. How can I help MyPy figure that out?
  2. Is it okay to not have a type hin for per_sample_loss?
#

ah actually I might be in the wrong here.

# 
(Pdb) foo = (targets_one_hot * probs)
(Pdb) whatis foo
<class 'semiflow.node.Node'>
(Pdb) foo1 = (targets_one_hot * probs).sum(axis=1)
(Pdb) whatis foo1
<class 'semiflow.node.Node'>
(Pdb) foo2 = (1.0) * (targets_one_hot * probs).sum(axis=1)
(Pdb) whatis foo2
<class 'semiflow.node.Node'>

Vscode thinks .sum() is jax.Array::sum() hence my littel confusion but no, should all be dandy. hmm.

fierce ridge
#

i think the way it works is, python checks the lhs __mul__ first, and if that doesn't exist then it checks the rhs __rmul__

#

so if Array has no Array.__mul__ then python looks for Node.__rmul__ in that example, not Node.__mul__

hollow canopy
# fierce ridge the rhs needs `__rmul__`

I misremembered then but in any case both are implemented and evaluate to Node as you see in the pdb output. Which mypy doesn't seem to recognize. or maybe even my LSP in general (no idea who mypy etc is all exactly implemented in vscode)

fierce ridge
hollow canopy
#

ah wait maybe I see an issue

hollow canopy
fierce ridge
hollow canopy
#

Array is jax.Array - no idea why it keeps displaying it as Array

fierce ridge
#

it just doesn't say the fully qualified name

#

(which imo it should)

#

can you check this one step at a time, in mypy specifically?

n1 = targets_one_hot * probs
assert_type(n1, Node)

n2 = n1.sum(axis=1)
assert_type(n2, Node)

n3 = 1.0 * n2
assert_type(n3, Node)

per_sample_loss = n3
hollow canopy
#

__mul__ nor __rmul__ had proper types in their singature - I just never saw it. But that might explain why it doesn't realize that.

fierce ridge
#

yeah that's the tricky part: if you're type checking your program, you assume that all the type hints of libraries etc are correct. otherwise everything goes off

hollow canopy
# 
    def __mul__(self, other_node: Node | int | float | jax.Array) -> Node:

    def __rmul__(self, other_node: Node | int | float | jax.Array) -> Node:
hollow canopy
#

ah fuck only inside vscode.

$ mypy ./semiflow/nn/loss/bce_loss.py
semiflow/nn/loss/bce_loss.py:24: error: Incompatible types in assignment (expression has type "Array", variable has type "Node")  [assignment]
Found 1 error in 1 file (checked 1 source file)

.< 😄

hollow canopy
#

I solved it by just turning around the multiplication making it invoke __mul__. Not sure why it can't resolved the rmul case though but more importantly hwo one would handle that

fierce ridge
polar aurora
#

Sometimes I think this should be called #type-haunting

trim tangle
#

that's spooky

polar aurora
#

But in the model Python's got, we kinda do 'haunt' our implementations with type names 🙂

#

Wisdom of our ancestors.

feral wharf
#

So is the future annotations import not necessary anymore when using the if TYPE_CHECKING: statement in 3.14, or did I understand that wrong?

trim tangle
#

(or rather, if your minimum supported version is 3.14 or later)

feral wharf
#

So, that's a yes?

trim tangle
#

yes

feral wharf
#

So why did that error out?

frail mesa
#

if a vscode workspace has multiple python interpreters/virtual environments, how do i make it load all of them rather than one, so i can see all the types in all projects without changing interpreter manually.

trim tangle
#

TYPE_CHECKING is False at runtime, no matter the Python version

feral wharf
#

Ah

#

And the fix for that would be to use the new annotations lib?

trim tangle
#

(without the TYPE_CHECKING guard)

feral wharf
#

What if it's a circular import

trim tangle
#

Why is cogs.utils.context importing this module?

feral wharf
#

Good question

#

So deferred imports impl isn't that exciting after all blobpain

trim tangle
#

3.14 doesn't change how imports function at all

#

I would definitely just fix the circular dependency

rare scarab
#

import changes will probably come in 3.15

trim tangle
#

yeah, lazy imports

rare scarab
#

opt in of course

trim tangle
#

though even that won't automagically fix all circular import issues

fiery canyon
#

Relative imports without needing a package stonks

rare scarab
#

import foo vs from . import foo

#

if you don't have a package, your import is top level anyway

fiery canyon
#

Odd cuz there was just a place where it didn't work normally for me

hollow canopy
#

I don't see why the "type hinter" (not too familiar with the lingo yet) wouldn't see that tho.

fierce ridge
hollow canopy
#

I do have two other questions though:

  1. How do people actually use mypy? Do you just code, run it at the end, make you commit, maybe have some git action whatever run it again and that's it? Or do people like configure something that everytime you run python it executes?
  2. Can I do something like "The Node type isn't allowed in this file at all"?
fierce ridge
#

I don't know about #2, but I treat it as part of my test suite

frail mesa
#

does python have something like typescript Pick<>

#

i have a dataclass and i need to pick some of the properties of it

#

and create a new dataclass based on it

marsh gorge
#

how would i go about making an Input protocol for my parser? I want the parser to be generic over str and bytes and maybe even a list of T

# 
class Input(Protocol):

    def __eq__(self, other: object) -> bool:
        pass

    def __getitem__(self, other: object) -> Self:
        pass

    def __str__(self) -> str:
        pass```
#

i have this but it doesnt pass the type checker when i use a str as parser input

#

it would be nice if mypy would tell me what and why it doesnt pass lol

#

or what it expects

trim tangle
#

Input describes an object that must be able to handle any object in __getitem__, but str can only handle integers and slices

marsh gorge
#

how did you get that output

trim tangle
marsh gorge
#

lhuh i dont get that

#

when i run it locally i mean

trim tangle
#

What version of mypy do you have? And can you show the code?

marsh gorge
rough sluiceBOT
marsh gorge
#

its kind of a big file now

trim tangle
#

Ah, now it says this main.py:263: error: Type argument "str" of "Parser" must be a subtype of "Input" [type-var] main.py:266: error: Argument 1 to "parse" of "TakeWhile" has incompatible type "str"; expected "Never" [arg-type] main.py:270: error: Type argument "str" of "Parser" must be a subtype of "Input" [type-var] main.py:277: error: Value of type variable "I" of "ParserError" cannot be "str" [type-var] but doesn't explain why str is not a subtype of Input

#

I remember why I use pyright now lol

marsh gorge
#

thank you for all of the help you and others have provided me with so far

trim tangle
#

Why do you want to constrain Parser though?

marsh gorge
trim tangle
#

yes

marsh gorge
#

yeah that would be better

trim tangle
#

or maybe this protocol never comes up and you just have separate parsers for e.g. str and bytes

marsh gorge
#

in theory i wanted to be able to parse any arbitrary sequence of things, so you could do a lexer pass and then parse "tokens" rather than str if that makes sense

#

although the main use case will be for parsing str or bytes lol

marsh gorge
#

whats up with this error?
pyparse/__init__.py:266: error: Argument 1 to "parse" of "TakeWhile" has incompatible type "str"; expected "Never" [arg-type]

#

I suspect it has to do with my protocol

trim tangle
marsh gorge
#

this is the TakeWhile iterator

@dataclass
class TakeWhile[I: Input](Parser[I, I]):
    _f: Callable[[I], bool]

    def parse(self, input: I) -> tuple[I, I]:
        i = 0

        while self._f(input[i]):
            i += 1

        return input[i:], input[:i]```
trim tangle
#

Can you paste all the code perhaps

marsh gorge
#

sure

rough sluiceBOT
marsh gorge
#

i couldnt get to pass mypy when i pulled the constrained off of Parser, but ill fix that later

trim tangle
#

I think mypy can't infer that the TakeWhile instance is a TakeWhile[str]

marsh gorge
#

ah

#

yeah annotating it fixed it

#

thats weird

#

how did you figure that out btw?

trim tangle
#

that's uhhh an educated guess

#

If you're used to Haskell or Rust or other languages with extensive bidirectional inference, python's type checkers are not like that

#

it exists sometimes but it's very limited

marsh gorge
#

yup im most used to rust, and porting my rust parser library over to python roughly

trim tangle
#

Why are you rewriting it from Rust to Python?

#

I thought it's the other way around nowadays

rare scarab
#

^ just expose your rust bindings to python

trim tangle
#

that might be difficult if you're relying on traits

marsh gorge
#

haha mostly for fun and learning, but i also prefer writing python in general

rare scarab
#

A parser is one of those things you'd want to be written in rust.

trim tangle
#

Have you tried pyright or basedpyright instead of mypy?

marsh gorge
#

no but i definitely can

rare scarab
#

ty is also available, though it's in alpha

#

(python type checker written in rust, same guys that made uv and ruff)

summer lark
#

Is there a way to programmatically enhance the annotations of a function so that type checkers like mypy can see the enhanced annotations? I was spurred by https://www.peterbe.com/plog/in-python-you-have-to-specify-the-type-and-not-rely-on-inference to see whether a decorator could enhance type annotations of a function. The answer is yes, but: at runtime the changes are visible in __annotations__ but not to type checkers (at least, not to mypy)... so it's of no help for the OP's use case.

Also if someone knows a more specific discussion of why Python typing chose not to infer an argument type as the default value's type, I bet the blogger would not mind a citation in the comments. I don't have enough familiarity with the history of typing PEPs to guess where it might have been said.

from typing import TYPE_CHECKING

def annotate_defaults(fn):
    """Use default arguments to enhance a function's type annotations

    For any argument with a default that is not None,
    and which is not annotated with a type, update the argument's type
    annotation to match the default value."""

    code = fn.__code__
    annotations = fn.__annotations__
    argspec = inspect.getfullargspec(fn)
    default_offset = len(argspec.args) - len(argspec.defaults)
    for i, a in enumerate(argspec.args):
        if a in annotations: continue
        if i < default_offset: continue
        defarg = argspec.defaults[i-default_offset]
        if defarg is not None:
            annotations[a] = type(defarg)

    return fn

@annotate_defaults
def f(a:int|None, b=False, c=None, d=3): ...

# At runtime, inspect sees the "enhanced" type annotation
print(inspect.getfullargspec(f))

# sadly, when mypy-checking, it prints "Any", because mypy doesn't execute the
# annotate_defaults, it just looks at its [unspecified] return type.
if TYPE_CHECKING:
    reveal_type(f)
In Python, you have to specify the type and not rely on inference -...

Unlike TypeScript, if you give a variable a default, which has a type, that variable is implied to always have the type of the default. That's not the case with mypy and ty.

restive rapids
trim tangle
summer lark
trim tangle
#

One of the core ideas behind mypy initially was the gradual adoption of mypy into a codebase. So you can take a program consisting of 100_000 lines of untyped Python code and add types to it 1k lines at a time, or 10 lines at a time, or just one parameter at a time. This is why, if you have a completely unannotated function like this: py def do_things(): x = 42 print(x.upper()) mypy will not complain about it -- because the function doesn't have annotations, it assumes that you just haven't gotten to annotate this function as part of a larger gradually typed program (and there's a --check-untyped-defs flag to disable this)

summer lark
#

I've mostly worked with small codebases and jumped right to mypy --strict checking but that sounds like a very good property to have. (as a total aside I need to choose whether to stick with mypy or jump on the pyrefly or ty bandwagons)

#

I can see where annotating with the type of a default value would work contrary to that philosophy

trim tangle
summer lark
#

I'm aware of pyright, didn't know about basedpyright yet.

trim tangle
#

I like pyright's inference rules better. It seems like it was developed by essentially copy-pasting the TypeScript compiler

#

It also provides a language server, so it can be used with editors like VSCode/vim/emacs/helix/Sublime Text etc.

#

(I'm using "go to definition" a lot so that's helpful)

summer lark
#

I've set up one of my trivial projects (2185 SLOC in src+tests) so I have a make target for mypy, pyrefly & pyright. I found pyright "slower enough" (about 2s to run, vs as low as 0.097s for pyrefly) that I was not enthusiastic about it.

trim tangle
#

Yeah, ty and pyrefly are going to blow mypy/pyright ouf of the water in terms of performance

summer lark
#

and I've never taken the time to try to integrate one into my editor (neovim) successfully.

trim tangle
summer lark
#

I am old fashioned so I gravitate to things that I can use with :make and using :cfile to visit the site of problems. WFM, YMMV, etc.

trim tangle
#

pyright should be reasonably fast in the "hot" state (i.e. when it populated its cache)

summer lark
#

I generated about a half dozen pyrefly issues as soon as I tried it on this project, which was previously mypy --strict clean

#

(they were at PyOhio in an open space session & encouraged me, otherwise I wouldn't go file a half dozen issues on a project I just started using 🤣 )

trim tangle
summer lark
#

Yeah I can see why the details of even having a cache would be easier in the language server case than in the CLI case. And why "kids these days" care more about performance in the former case.

#

ty also comes in at about 90ms checking my 2k lines

trim tangle
#

However, according to pyright's docs...

Pyright was designed with performance in mind. It is not unusual for pyright to be 3x to 5x faster than mypy when type checking large code bases. Some of its design decisions were motivated by this goal.
It is also written in TypeScript for some reason.

#

very strange choice if performance is important

#

maybe it is faster in watch mode, i dunno

spiral fjord
summer lark
#

If I remove .mypy_cache it takes 1.6s instead of well under 1s. so it does have some effective caching of the CLI.

trim tangle
brazen jolt
trim tangle
#

I imagine some people use Pylance's editor integration while running mypy in CI (I used to do that)

drowsy glen
trim tangle
#

Stringified annotations used to be planned to become the default in 3.10, but then the whole idea got scrapped, and now we have deferred annotations using descriptors

#

!pep 649

rough sluiceBOT
drowsy glen
#

Well, sorry I was wrong, but as far as I know the minimum is 3.11 because type annotations are no longer evaluated by default

trim tangle
#

No, versions 3.10 through 3.13 all use the same model as python 3.5 by default, evaluating annotations eagerly

drowsy glen
#

Ahh I understand, it would be more like eager evaluation by default

trim tangle
# 
$ python --version
Python 3.13.5
$ cat foo.py 
def f(x: missing):
    pass
$ python foo.py 
Traceback (most recent call last):
  File "/tmp/scratch/foo.py", line 1, in <module>
    def f(x: missing):
             ^^^^^^^
NameError: name 'missing' is not defined
drowsy glen
#

In that case use from __future__ import annotations or strings

#

okok

void drum
#

@drowsy glen are you can help me ?

frail mesa
#

does python have something like typescript Pick<>
i have a dataclass and i need to pick some of the properties of it
and create a new dataclass based on it

trim tangle
frail mesa
#

what do i do then

trim tangle
#

write the dataclass manually

soft matrix
#

I don't really think that'd ever be possible

#

Data class construction at type time sounds very niche

trim tangle
#

not if we have 4 type checkers trying to agree on how it's spelled 😔

trim tangle
# 
# doesn't work
type CoolPoint = TypedDict[{"x": int, "y": int, "label": str}]
type NormalPoint = Pick[CoolPoint, Literal["x", "y"]]
# same as:
type NormalPoint = TypedDict[{"x": int, "y": int}]
trim tangle
#

btw @soft matrix PEP718 question: Suppose that I have this py class Crate: def __init__[B](self, *bananas: B, *, certify: Callable[[B], AnsiBanana]): self._bananas: list[AnsiBanana] = [certify(b) for b in bananas] can I bind the B parameter when calling the __init__?

mossy rain
trim tangle
mossy rain
#

true

trim tangle
#

in this example it doesn't have parameters, but it could ```py
class Crate[A: AnsiBanana]:
def init[B](self, *bananas: B, *, certify: Callable[[B], A]):
self._bananas: list[A] = [certify(b) for b in bananas]

mossy rain
#

then not i suppose, or the parametiization could make B, via the thing __class_getitem__ returns

trim tangle
#

I guess it's a really esoteric edge case, I don't really care if it's supported or not
Just a curiosity

soft matrix
mossy rain
#

or perhaps typecheckers should error on definitions of __init__ / __new__ in classes, which are generic

empty mason
#

whats is this "[B]" what does this do?

mossy rain
trim tangle
#

no, this exists now without pep 718

soft matrix
trim tangle
#

Not even sure what page to link

mossy rain
empty mason
#

ok. thanks

trim tangle
mossy rain
mossy rain
trim tangle
#

It's not useless

mossy rain
#

Well if you cant access it?

empty mason
#

Hey guys, I'm Brazilian and I'm learning to program in Python. I'm going to start communicating more with you. I want to make friends, and with that, I'll also learn more about programming in Python.

soft matrix
empty mason
#

I don't know any English but I'll try to communicate ks

mossy rain
#

It would only show that a and b have the same type for some def __init__[T](self, a: T, b: T) -> None: ..., which can be done without making __init__ generic

trim tangle
# mossy rain Well if you cant access it?

Right now, type variables are inferred based on the arguments. For example: ```py
def make_pair[T](obj: T) -> tuple[T, T]:
return (obj, obj)

pair = make_pair(42)
reveal_type(pair) # tuple[int, int]
``` the same would happen with that __init__

mossy rain
trim tangle
rough sluiceBOT
#

discord/ext/lava/_utilities.py lines 37 to 43

class DeferredMessage:

    def __init__[**P](self, callable: Callable[P, str], *args: P.args, **kwargs: P.kwargs) -> None:
        self.callable: functools.partial[str] = functools.partial(callable, *args, **kwargs)

    def __str__(self) -> str:
        return f"{self.callable()}"```
trim tangle
#

(in this case there's never a reason to provide P manually though)

rough sluiceBOT
#

src/sentry/utils/query.py lines 101 to 103

def __init__[M: Model](
    self,
    queryset: QuerySet[M, V],```
soft matrix
trim tangle
soft matrix
#

Oh that's the whole class

#

Hmm

#

Any idea how you'd want it to look syntactically?

trim tangle
# trim tangle Because the generics are not used to specify anything about the instance of the ...

Semantically it's equivalent to ```py
class MyClass:
def init[<Vars>](self, <non-generic args>, <generic args depending on Vars>)

c = MyClass(<non-generic args>, <generic args>)
#->

type _Inter = <doesnt depend on Vars>

class MyClass:
def init(self, <non-generic args>, intermediate: _Inter)

def _helper[<Vars>](<non-generic args>, <generic args depending on Vars>):
intermediate: _Inter = "compute a tuple of values using all the args"
return MyClass(<non-generic args>, intermediate)

c = _helper(<non-generic args>, <generic args>)

trim tangle
#

hmm I guess here's a related question ```py
class ClsDecorator(Protocol): # the protocol itself is not generic
def call[T: type](self, kls: T, /) -> T: ...

def do_something(deco: ClsDecorator) -> None:
# you're here
``` is it okay to do deco[int](bool)?

soft matrix
#

Seems kinda cursed but makes sense

trim tangle
#

Callable protocols like this are a thing. Something like ```py
@dataclass_transform(frozen_default=True)
def frozen(*, slots: bool = True) -> ClsDecorator:
...

#

There's probably a better example for the purposes of subscripting though

#

In TypeScript the equivalent would be ```typescript
type StrMapper = <T extends string>(arg: T) => T

function doSomething(arg: StrMapper) {
const s = arg<"a" | "b">("a")
// ok, s is "a"|"b"
}

soft matrix
#

I'll add it to the list

trim tangle
#

This seems oddly similar to the problem of whether a given callable is considered to be a method-like descriptor

#

Something like FunctionType & ClsDecorator could express this maybe

trim tangle
#

In general, it could be a functools.partial, a class, or some other custom callable

#

On one hand, it would be sound to prohibit indexing something you only know to be a Callable but not necessarily a function. On the other it means that pretty much every decorated function now doesn't support subscription

soft matrix
#

if its something that returns a Callable rather than a tv bound to one its not safe to support yeah

#

but I think if you're stipulating its a ClsDeco which has a tv in its type params it should technically be safe

#

not sure what the spec says though

oblique urchin
#

well, you're writing the spec here, I do think fix error's example is a problem for PEP 718

#

if PEP 718 is implemented type checkers will learn that functions are subscriptable, but that doesn't necessarily go for arbitrary callables

#

and the type system isn't always good at distinguishing between functions and arbitrary callables

trim tangle
#

typescript dodged so many bullets by erasing annotations from the runtime tbh

feral wharf
trim tangle
#

you can do hasattr(obj, "__iter__"), which tells you that the object has some attribute named __iter__ (though type checkers don't support these type guards)

#

you could do ```py
def is_iterable(obj: object) -> TypeIs[Iterable[object]]:
return hasattr(obj, "iter")

feral wharf
#

need to handle each item individually

trim tangle
#

can you provide more details? why do you want to check if something is iterable?

feral wharf
#

like I said, I need to check and handle each item

# users: MyClass(pages=[<item>, <item>, (<item>, <item>, ...), [(<item>, <item>, ...), ...], ...iterables], per_page=<num>)

# type Seq[T] = list[T] | tuple[T, ...]
# type Sequence[T] = Seq[T] | Seq[Seq[T]]

# actual handling
async def handle_pages(self, pages: Sequence[PageT], /, skip_formatting: bool = False) -> BaseKwargs:
    if not skip_formatting:
        self._reset_base_kwargs()
        formatted_pages = await discord.utils.maybe_coroutine(self.format_page, pages)
        return await self.handle_pages(formatted_pages, skip_formatting=True)

    for page in pages:
        if isinstance(page, (list, tuple)): # <---
            await self.handle_pages(page, skip_formatting=True)  # pyright: ignore[reportArgumentType]
            continue

        if isinstance(page, (int, str, discord.ui.TextDisplay)): ...

        elif isinstance(page, discord.Embed): ...

        elif isinstance(page, (discord.File, discord.Attachment)): ...

        # Handle dictionary updates
        elif isinstance(page, dict): ...

        elif isinstance(page, discord.ui.Item): ...

    return self.__base_kwargs
trim tangle
feral wharf
# 
class BaseClassPaginator[PageT: Any]:
  # not actually a classvar
  pages: Sequence[PageT]

or preferably the following with an overload on a kwarg that I sent in the other convo, but first need to figure out the type-safe loop:

_BoundPage = (
    str
    | discord.Embed
    | discord.File
    | discord.Attachment
    | discord.ui.Button[Any]
    | discord.ui.Select[Any]
    | dict[str, Any]
)
BoundPage = _BoundPage | Sequence[_BoundPage]
_BoundV2Page = (
    str
    | discord.ui.ActionRow[Any]
    | discord.ui.Container[Any]
    | discord.ui.File[Any]
    | discord.ui.MediaGallery[Any]
    | discord.ui.Section[Any]
    | discord.ui.Separator[Any]
    | discord.ui.TextDisplay[Any]
)
BoundV2Page = _BoundV2Page | Sequence[_BoundV2Page]

# PageT: BoundPage | PageV2Page
trim tangle
feral wharf
#

wait

trim tangle
#

also maybe change [PageT: Any] (which is the same as just [PageT]) to [PageT: (BoundV2Page, BoundPage)]

feral wharf
#

fml

trim tangle
#

if you only want to allow BasePaginator[BoundPage] and BasePaginator[BoundPageV2] to exist

feral wharf
#

yeah it can't really be anything else
unless the user wants to handle it in the format_page method

#

but then again, it should still return one of those

feral wharf
trim tangle
#

Typically this is not the right choice because it's a janky feature (or maybe just implemented poorly?), but if you want to support only BasePaginator[BoundPage] and BasePaginator[BoundPageV2], but not BasePaginator[BoundPage | BoundPageV2] or BasePaginator[str], this is what you should use

feral wharf
#

ahh

#

that's exactly what I want

#

is this correct?

type Seq[T] = list[T] | tuple[T, ...]
type Sequence[T] = Seq[T] | Seq[Seq[T]]

Or should I just use the builtin Sequence and raise if pages is str

#

because that still warns about list being invariant

#

I also can't figure out this one

def get_page(self, page_number: int) -> Sequence[PageT]:
    if not self.pages:
        raise ValueError(
            "No pages are available. Either provide a non-empty 'pages' sequence when creating the paginator, or assign to '.pages' before sending."
        )


    if self.per_page == 1:
        page = self.pages[page_number]
        if isinstance(page, (list, tuple)): # <--
            return page
        return [page]
    base = page_number * self.per_page
    return self.pages[base : base + self.per_page]

it should always return a sequence

#

imma just use list

#

even when using the builtin sequence, it complains

#

I don't get this one at all

    async def handle_pages(self, pages: Sequence[PageT | Sequence[PageT]], /, skip_formatting: bool = False) -> BaseKwargs:
        if not skip_formatting:
            self._reset_base_kwargs()
            formatted_pages = await discord.utils.maybe_coroutine(self.format_page, pages)
            return await self.handle_pages(formatted_pages, skip_formatting=True)

        for page in pages:
            if isinstance(page, (int, str, discord.ui.TextDisplay)): ...
            elif isinstance(page, discord.Embed): ...
            elif isinstance(page, (discord.File, discord.Attachment)): ...
            elif isinstance(page, dict):...
            elif isinstance(page, discord.ui.Item): ...
            else:
                await self.handle_pages(page, skip_formatting=True) # < ---
# Argument of type "Sequence[_BoundPage]* | Sequence[_BoundV2Page]* | Sequence[PageT@BaseClassPaginator]" cannot be assigned to parameter "pages" of type "Sequence[PageT@BaseClassPaginator | Sequence[PageT@BaseClassPaginator]]" in function "handle_pages"
#  Type "Sequence[_BoundPage]* | Sequence[_BoundV2Page]* | Sequence[PageT@BaseClassPaginator]" is not assignable to type "Sequence[PageT@BaseClassPaginator | Sequence[PageT@BaseClassPaginator]]"
#    "Sequence[_BoundPage]*" is not assignable to "Sequence[PageT@BaseClassPaginator | Sequence[PageT@BaseClassPaginator]]"
#      Type parameter "_T_co@Sequence" is covariant, but "_BoundPage" is not a subtype of "PageT@BaseClassPaginator | Sequence[PageT@BaseClassPaginator]"
#        Type "_BoundPage" is not assignable to type "PageT@BaseClassPaginator | Sequence[PageT@BaseClassPaginator]"
#          Type "Attachment" is not assignable to type "PageT@BaseClassPaginator | Sequence[PageT@BaseClassPaginator]"PylancereportArgumentType

        return self.__base_kwargs
trim tangle
feral wharf
# 
type File = discord.File | discord.Attachment

type _BoundPage = str | discord.Embed | discord.ui.Button[Any] | discord.ui.Select[Any] | dict[str, Any] | File
type BoundPage = _BoundPage | Sequence[_BoundPage]
type _BoundV2Page = (
    str
    | discord.ui.ActionRow[Any]
    | discord.ui.Container[Any]
    | discord.ui.File[Any]
    | discord.ui.MediaGallery[Any]
    | discord.ui.Section[Any]
    | discord.ui.Separator[Any]
    | discord.ui.TextDisplay[Any]
    | dict[str, Any]
    | File
)
type BoundV2Page = _BoundV2Page | Sequence[_BoundV2Page]

class BaseClassPaginator[PageT: (BoundPage, BoundV2Page)]:
trim tangle
# feral wharf I don't get this one at all ```py async def handle_pages(self, pages: Sequen...

What if you do ```py
async def handle_pages(self, page: PageT, /, skip_formatting: bool = False) -> BaseKwargs:
if not skip_formatting:
self._reset_base_kwargs()
formatted_pages = await discord.utils.maybe_coroutine(self.format_page, pages)
return await self.handle_pages(formatted_pages, skip_formatting=True)

    if isinstance(page, (int, str, discord.ui.TextDisplay)): ...
    elif isinstance(page, discord.Embed): ...
    elif isinstance(page, (discord.File, discord.Attachment)): ...
    elif isinstance(page, dict):...
    elif isinstance(page, discord.ui.Item): ...
    else:
        await self.handle_pages(page, skip_formatting=True)

    return self.__base_kwargs
#

(using Sequence = collections.abc.Sequence)

feral wharf
#

lots of

"X*" is not iterable
  "__iter__" method not defined PylancereportGeneralTypeIssues
trim tangle
#

Do you have the code somewhere in a runnable state?

#

Maybe I can load it up and see more context

feral wharf
#

oh it kinda works now with

type BoundPage = Sequence[_BoundPage]
type BoundV2Page = Sequence[_BoundV2Page]
soft matrix
soft matrix
feral wharf
trim tangle
# feral wharf oh it kinda works now with ```py type BoundPage = Sequence[_BoundPage] type Boun...

Actually what I meant was this ```py
async def handle_pages(self, page: Sequence[PageT], /, skip_formatting: bool = False) -> BaseKwargs:
self._reset_base_kwargs()
if skip_formatting:
self._handle_pages_unformatted(page)
else:
formatted_pages = await discord.utils.maybe_coroutine(self.format_page, pages)
await self._handle_pages_unformatted(formatted_pages)
return self.__base_kwargs

async def _handle_pages_unformatted(self, page: PageT) -> BaseKwargs:
    if isinstance(page, (int, str, discord.ui.TextDisplay)): ...
    elif isinstance(page, discord.Embed): ...
    elif isinstance(page, (discord.File, discord.Attachment)): ...
    elif isinstance(page, dict):...
    elif isinstance(page, discord.ui.Item): ...
    else:
        for p in page:
            await self._handle_pages_unformatted(p)  # or something
#

also, resetting base_kwargs and then returning them seems like you're going to have nasty concurrency issues

#

i.e.: two concurrent callers to handle_pages will share the same base_kwargs, erroneously

feral wharf
#

they should share the same kwargs, that's what is sent later

trim tangle
#

I'm assuming those ...s modify the base kwargs somehow?

feral wharf
#

yeah

oblique urchin
feral wharf
trim tangle
# feral wharf yeah

So you'll have this:

  1. caller A calls handle_pages and resets kwargs. It starts populating the new kwargs
  2. after some time, caller B calls handle_pages on the same object, also resetting kwargs, discarding all the work A's handle_pages did on the kwargs
  3. A's and B's handle_page both add stuff to the kwargs
  4. A's and B's handle_page both return the same combined kwargs that are probably wrong
#

a better solution would be creating a local object and mutating that, e.g: ```py
async def handle_pages(self, page: Sequence[PageT], /, skip_formatting: bool = False) -> BaseKwargs:
async def handle_pages_unformatted(page: PageT) -> BaseKwargs:
if isinstance(page, (int, str, discord.ui.TextDisplay)): ...
...
else:
for p in page:
await handle_pages_unformatted(p) # or something
self._reset_base_kwargs()
base_kwargs = BaseKwargs()
if skip_formatting:
self._handle_pages_unformatted(page)
else:
formatted_pages = await discord.utils.maybe_coroutine(self.format_page, pages)
await handle_pages_unformatted(formatted_pages)
return base_kwargs

#

(or as a separate method with a BaseKwargs parameter)

trim tangle
feral wharf
#

only reason why it's stored on the instance is because I needed to recursively loop the list and that was the best I could think of at the time

trim tangle
#

A common pattern is passing the state you're mutating as a parameter to the recursive function

feral wharf
#

oh yeah but that's ugly and confusing imo

trim tangle
#

why?

#

you like the closure approach better?

feral wharf
#

yeah

trim tangle
#

parameters intended for mutation do feel kinda bad

feral wharf
#

ye, parameters that may only be used in the function itself

feral wharf
#

can pyright not see the isinstance

trim tangle
#

page: Sequence[PageT]

feral wharf
#

should it be page: PageT?

trim tangle
#

probably

#

according to the method name at least

feral wharf
trim tangle
#

and this is what I meant by constrained type variables being janky joe_salute

#

at this point I'd recommend # pyright: ignore[reportArgumentType]

feral wharf
trim tangle
#

Here's more crazy stuff: if with exactly the same code in both branches as a workaround ```py
def concat[S: (str, bytes)](x: S, y: S) -> S:
return x + y

def concat_loose1(s: str | bytes) -> str | bytes:
return concat(s, s)
# ^ error
# Argument of type "str | bytes" cannot be assigned to parameter "x" of type "S@concat" in function "concat"
# Type "str | bytes" is not assignable to constrained type variable "S" (reportArgumentType)
# Argument of type "str | bytes" cannot be assigned to parameter "y" of type "S@concat" in function "concat"
# Type "str | bytes" is not assignable to constrained type variable "S" (reportArgumentType)

def concat_loose2(s: str | bytes) -> str | bytes:
# no error
if isinstance(s, str):
return concat(s, s)
else:
return concat(s, s)

feral wharf
#

is this a pyright only thing

trim tangle
#

no, mypy also complains

feral wharf
trim tangle
#

mypy has this though ```py
from typing import reveal_type, Literal

class TruthTable[
A: (Literal[True], Literal[False]),
B: (Literal[True], Literal[False]),
C: (Literal[True], Literal[False]),
]:
def init(self, a: A, b: B, c: C) -> None:
if a and not (b and c):
reveal_type(self)

main.py:10: note: Revealed type is "main.TruthTable[Literal[True], Literal[True], Literal[False]]"
main.py:10: note: Revealed type is "main.TruthTable[Literal[True], Literal[False], Literal[True]]"
main.py:10: note: Revealed type is "main.TruthTable[Literal[True], Literal[False], Literal[False]]"

#

apparently it just synthesizes a separate class body for each combination of constrained typevars and evaluates each of them

trim tangle