type-hinting

[trim tangle]

If pyright/pylance thinks a variable is unbound, it will use that word, it will never use None

Because x = None and x is-unbound are different

[iron vapor]

It was saying unbound before

I must have changed something and not noticed unbound went away

Sometimes an error gets frustrating, you start changing things, it fixes one thing, and you don't even notice because the overall error is still there

But yah, just adding a if not store: to the if block with get_by_id fixed it

I'm gonna call this rubber ducking 😛 Thank you though!

[grizzled ferry]

Hi everyone!

I'm currently working with a type-safe decorator that injects a lock into a function, as shown in the example below.

My question is: How do I replicate this typing approach inside a class? I’d like to create a similar type-safe decorator for class methods that automatically passes an instance attribute (e.g., a lock) to the method without having to explicitly include it in every call. Specifically, how do I handle the self attribute in the type annotations when using Concatenate and Callable?

The problem is that when using the method in a class, the linter doesn't detect anything.

Any insights or code examples would be appreciated!

from collections.abc import Callable
from threading import Lock
from typing import Concatenate

my_lock = Lock()


def with_lock[**P, R](f: Callable[Concatenate[Lock, P], R]) -> Callable[P, R]:
    """A type-safe decorator which provides a lock."""

    def inner(*args: P.args, **kwargs: P.kwargs) -> R:
        # Provide the lock as the first argument.
        return f(my_lock, *args, **kwargs)

    return inner


@with_lock
def sum_without_passing_lock(lock: Lock, numbers: list[float]) -> float:
    with lock:
        return sum(numbers)


def sum_passing_lock(lock: Lock, numbers: list[float]) -> float:
    with lock:
        return sum(numbers)


# We don't need to pass in the lock ourselves thanks to the decorator.
sum_without_passing_lock([1.1, 2.2, 3.3])
"""
When overing the function name

(function) def sum_without_passing_lock(
    numbers: list[float]
) -> float
"""

sum_passing_lock(my_lock, [1.1, 2.2, 3.3])
"""
When overing the function name

(function) def sum_passing_lock(
    lock: Lock,
    numbers: list[float]
) -> float
"""

[soft matrix]

yeah thats cause the function as its defined doesnt have the deco applied

[grizzled ferry]

what do you mean by 'deco'?

[frigid jolt]

decorator

[hollow gust]

is this a config issue

[restive rapids]

no, it is indeed not defined at the time of defining the method
either do "str | User" or from __future__ import annotations

[hollow gust]

thanks

[hollow gust]

Which one should i use?

[restive rapids]

they both do the same thing
if you often use types that are not yet defined, future annotations allows to have less of those noisy " in types, but if its a one off thing - i'd use quotes to explicitly forward ref it

[hollow gust]

thanks

❣️

[stiff acorn]

How do I disable type checking for a single function?

My use case is that i have a test function that has type error in only some versions I test for

@pytest.mark.skipif(some_cond)
def test_something() -> None:
    from lib import SomeThing  # <--- This import only exists when some_cond is False

If I add a type: ignore[import-not-found] comment, then mypy will only pass when some_cond is True (because the type ignore becomes useless when the import exists, and mypy correctly reports it as an unused type ignore)

[restive rapids]

!d typing.no_type_check, maybe?

[rough sluice]

typing.no\_type\_check

@typing.no_type_check```
Decorator to indicate that annotations are not type hints.

This works as a class or function [decorator](https://docs.python.org/3/glossary.html#term-decorator). With a class, it applies recursively to all methods and classes defined in that class (but not to methods defined in its superclasses or subclasses). Type checkers will ignore all annotations in a function or class with this decorator.

`@no_type_check` mutates the decorated object in place.

[stiff acorn]

Looked promising but it did nothing...

https://mypy-play.net/?mypy=1.15.0&python=3.12&gist=e735ee067a46dd916a326bc8c05a8e15&flags=strict

[[tool.mypy.overrides]]
module = ["lib"]
ignore_missing_imports = true

this worked

[rare scarab]

In mypy, it will only typecheck functions with annotations

[hallow flint]

you can also do type: ignore[import-not-found, unused-ignore]

[stiff acorn]

Oh wow didn't realise I can put unused ignore inside an ignore

Thanks

[grizzled ferry]

Hi there!

is there any way to refactor this types?

type Tuple[T] = tuple[T, ...]

type SelectRes1[T1] = tuple[Tuple[T1]]
type SelectRes2[T1, T2] = tuple[*SelectRes1[T1], Tuple[T2]]
type SelectRes3[T1, T2, T3] = tuple[*SelectRes2[T1, T2], Tuple[T3]]
type SelectRes4[T1, T2, T3, T4] = tuple[*SelectRes3[T1, T2, T3], Tuple[T4]]
type SelectRes5[T1, T2, T3, T4, T5] = tuple[*SelectRes4[T1, T2, T3, T4], Tuple[T5]]
type SelectRes6[T1, T2, T3, T4, T5, T6] = tuple[*SelectRes5[T1, T2, T3, T4, T5], Tuple[T6]]
type SelectRes7[T1, T2, T3, T4, T5, T6, T7] = tuple[*SelectRes6[T1, T2, T3, T4, T5, T6], Tuple[T7]]
type SelectRes8[T1, T2, T3, T4, T5, T6, T7, T8] = tuple[*SelectRes7[T1, T2, T3, T4, T5, T6, T7], Tuple[T8]]
type SelectRes9[T1, T2, T3, T4, T5, T6, T7, T8, T9] = tuple[*SelectRes8[T1, T2, T3, T4, T5, T6, T7, T8], Tuple[T9]]
type SelectRes10[T1, T2, T3, T4, T5, T6, T7, T8, T9, T10] = tuple[*SelectRes9[T1, T2, T3, T4, T5, T6, T7, T8, T9], Tuple[T10]]

The original idea was to create a type that accepts specific variables and wraps each one in a tuple.

type Tuples[*T] = ...

def some_func() -> Tuples[str, int, float]: ...  # tuple[tuple[str,...], tuple[int,...],tuple[float,...]]

thanks in advance

[trim tangle]

No, that's not possible

[grizzled ferry]

😦

[trim tangle]

It's also not possible to type some_func without overloads

if I understood what you want

no, I misunderstood what you want

[grizzled ferry]

My idea is to create a type that accepts specific variables and wraps each one in a tuple.

type Tuples[*T] = ...
def some_func() -> Tuples[str, int, float]: ...  # tuple[tuple[str,...], tuple[int,...],tuple[float,...]]

[restive rapids]

no definition for Tuples can exist that would achieve this
you cant do anything with each separate typevar in a typevartuple
you basically want type Tuples[*Ts] = tuple[tuple[T, ...] for T in Ts] but thats just not a construct that can be expressed currently

[grizzled ferry]

that's what i want yep.

Thank you though!

Then, i'll use the implementation before.

[iron vapor]

I have a number of repositories for a CRUD app I'm buiding, all of which so far have a create method, and it's common for them to throw an AlreadyExists error, defined as

class AlreadyExists(Exception):
    def __init__(self, cls) -> None:
        super().__init__("{type(cls).__name__} already exists")
        self.cls = cls

Where cls is just the main type of db model that repository handles. Obviously this means that setting a type cls isn't possible in a fixed manner, as it adjusts. Is there any way to set a definition? I know TypeScript has generics, which do this exactly, and while I see Python has TypeVar, I don't see a way to do this.

[stable fjord]

T = Typevar("T")
class AlreadyExists(Generic[T], Exception):
    def __init__(self, cls: T) -> None:
        ...

although I'm not sure if you're gaining anything by making it generic as opposed to just typing the signature as def __init__(self, cls: object) -> None

[brazen jolt]

or with a new syntax: ```python
class AlreadyExistsT:
def init(self, cls: T) -> None:
...

but yeah, this is only useful if you're using T somewhere else too

and if cls is always supposed to be a class, you can also restrict the typevar to type

class AlreadyExists[T: type](Exception):
    def __init__(self, cls: T) -> None:
        ...

or, with old syntax: python T = Typevar("T", bound=type) (though I'd recommend a name like T_INT here if you do that)

[stable fjord]

man, I'm always looking forward to old python versions dropping so I get to use new syntax in libraries 😶
I wanted to use switch/case the other day... nope

[brazen jolt]

yeah, same, maintaining libs can be pretty annoying if you want to use new things

at least most of the type stuff is backported through typing_extensions

[iron vapor]

In this case, it's even more specific as a pydantic BaseModel, which is what I have it defined as now

But this was as much a "how could i do it" question. Genetic types confuse me in any language...

You're definitely right that in this case, it adds little value

[brazen jolt]

not just little value, if I were reviewing that code, I'd ask for it to be removed, unless there's future plans to make use of it or something like that, if not, it's just pointless

at most, you could argue that it allows some more specific type hints if you're passing these exceptions around, but that's unlikely for exceptions

I don't think you'd ever write a function that only works with instances of AlreadyExists for a specific type

[iron vapor]

I'm not sure I'd agree that it's pointless, as it is strictly for type hinting, but yah, if ItemRepository.create raises an error, it likely has to do with item. But I'd say not 100% of the time. But it's definitely more useful that the error knows the type, as per the message.

[iron vapor]

Yah, I agree

[brazen jolt]

sure, but raised exceptions aren't type-hinted in a way return types would be, and when you catch AlredyExists, the type checker won't know what the generic type is, so you'd only get AlreadyExists[BaseModel] anyways, you'd need to cast it yourself and that's only useful if you have something that later expects a specific kind of AlreadyExists, which is very unlikely

if you just want to check what type the error was raised for, you'd just do issublcass(exc.cls, MyModel)

[primal lintel]

I'm in VSCode and trying to use dispatch from multipledispatch, I was trying out

@dispatch(int, int)
def add(a: int, b: int) -> int:
    return a + b

@dispatch(str, str)
def add(a: str, b: str) -> str:
    return a + b

@dispatch(list, list)
def add(a: list, b: list) -> list:
  return a + b

@dispatch(int, str)
def add(a: int, b: str) -> str:
  return str(a) + b

@dispatch(str, int)
def add(a:str, b:int) -> str:
  return a + str(b)

and I get Function declaration "add" is obscured by a declaration of the same name
from pylance for all but the last instance of add. Is there any way to fix this?

[iron vapor]

Oh, interesting, I did exactly what you folks showed me and it type hints in VSCode just fine?

[brazen jolt]

it's not that it wouldn't work, it's that it wouldn't give you any benefit

[iron vapor]

Yah, again, I do agree with that

[trim tangle]

type checkers do not understand how multipledispatch works

so you'll have to # type: ignore these definitions and probably every usage site

[glossy spear]

cough

[jovial lava]

Because of the invariance, is the only way to get a MutableSquence that will accept any combination of an union passed as the type parameter really to explode the entire union into every possible permutation? Meaning that instead of:

def spawn(self, cmd: MutableSequence[bytes | StrPath]) -> None: ...

cmd = ["some_exec", "with", "params"]
CCompiler().spawn(cmd)
# Argument of type "list[str]" cannot be assigned to parameter "cmd" of type "MutableSequence[StrPath | bytes]" in function "spawn"
#   "list[str]" is not assignable to "MutableSequence[StrPath | bytes]"
#     Type parameter "_T@MutableSequence" is invariant, but "str" is not the same as "StrPath | bytes"

I have to write:

    def spawn(
        self,
        cmd: (
            MutableSequence[bytes]
            | MutableSequence[str]
            | MutableSequence[os.PathLike[str]]
            | MutableSequence[bytes | str]
            | MutableSequence[bytes | os.PathLike[str]]
            | MutableSequence[str | os.PathLike[str]]
            | MutableSequence[bytes | str | os.PathLike[str]]
        ),
    ) -> None: ...

(the only reason it needs to be mutable is because the first item in the sequence is path-expanded and re-assigned, for all other purposes cmd could be a covariant Iterable)

[trim tangle]

So what will the first item in the sequence become?

[jovial lava]

It stays the same type, it just results in the path of the executable:

def spawn(cmd, ...):
    ...
    executable = shutil.which(cmd[0])
    if executable is not None:
        cmd[0] = executable

Source: https://github.com/pypa/distutils/blob/9fc465a79d59addde3a05711da9e7ae3fbfd87c1/distutils/spawn.py#L65-L67

bytes stays bytes, str stays str, StrPath stays StrPath. technically according to the type definition os.PathLike[str] because StrPath.
(StrPath is jsut an alias for str | os.PathLike[str])

[trim tangle]

Are you typing an existing library?

[jovial lava]

Yeah, setuptools (in this case this bit of code exists in pypa/distutils)

Tempted to just PR a change to spawn to work on iterables and just copy cmd as a list. But that could be a breaking change if anyone downstream relied on the mutation >.<

[trim tangle]

Yeah, that's why I asked

You could try something like ```py
def spawn[T: (bytes, str, bytes | str, bytes | str | os.PathLike[str]])](
self,
cmd: MutableSequence[T],
) -> None: ...

wait, it probably needs to work with <3.12. Then ```py
_P = TypeVar("_P", bytes, str, bytes | str, bytes | str | os.PathLike[str]])
...
def spawn(self, cmd: MutableSequence[_P], ) -> None: ...

!e
This is cursed btw

import shutil
from pathlib import Path

print(shutil.which(Path("cp")))

well... it doesn't work here for some reason

[rough sluice]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

None

[trim tangle]

on my computer it shows ```py

which(Path("bash"))
'/usr/bin/bash'

[jovial lava]

Well, that works. Same issue of having to list all permutations, but at least it's much shorter.

I've been told before that this may be out of spec.

Humm, actually when I was told that by @lunar dune it was specifically for restricting entirely whithin the return annotation: https://github.com/python/typeshed/pull/11137#discussion_r1421726578

In my current environment I also don't see pyright or mypy whining about it. And they both correctly error out if I add int to my list.

[jovial lava]

And on mine (3.10, Windows):

>>> import shutil
>>> from pathlib import Path
>>> print(shutil.which(Path("python"))) 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Avasam\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 1492, in which
    if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
  File "C:\Users\Avasam\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 1492, in <genexpr>
    if any(cmd.lower().endswith(ext.lower()) for ext in pathext):
AttributeError: 'WindowsPath' object has no attribute 'lower'. Did you mean: 'owner'?

On 3.13:

>>> import shutil
>>> from pathlib import Path
>>> print(shutil.which(Path("python")))
C:\Users\Avasam\AppData\Local\Programs\Python\Python310\python.EXE

Oh and in both cases folders return None

It could be that which is mistyped ! That may simplify my life here Actually not really, it's still valid to pass in a PathLike in most Python versions and environments. Even if your list becomes filled with only str it's still a valid list[str | os.PathLike[str] after the mutation...

[lunar dune]

I think using a TypeVar only once in a signature like this (to get around variance issues) is a known loophole in the type system that you can use to get away with some unsoundness if you're in need

But there's always the chance it might be banned in some future update to the spec 😛

I don't think this is very well specified right now

[trim tangle]

I don't know how it makes sense for shutil.which to accept paths... it accepts a command name, not a path, and Path("python") means "the file python in the current working directory"

[jovial lava]

>>> import shutil
>>> from pathlib import Path
>>> shutil.which(Path(".venv")/"scripts"/"python")
'.venv\\scripts\\python.EXE'

Anyway, I think this case is special enough, I'll consider the breaking change in distutils. Otherwise I'll probably just alias the union.

Still did just find a return type issue with which. So I'll go update that in typeshed.

[jovial lava]

It may not make sense if used directly. But it can come from somewhere else where you just forward the variable. Where the type can happen to be StrPath.

[trim tangle]

In that case which(Path("python")) would make sense to be (current working directory)/python

i.e. if I provide the path Path("./myprogram") for the executable, and I have a myprogram executable in my current directory, but there's also /bin/bash/myprogram, I would be extremely surprised if /bin/bash/myprogram is chosen instead

[jovial lava]

Oh I see what you mean, you wouldn't expect it so search for python on the PATH

[trim tangle]

Yeah, I'd expect Path("python") and Path("python").absolute() to have pretty much the same effect

[jovial lava]

Fair. Time to raise an issue at CPython? 😛

[trim tangle]

one day...

[rare vapor]

I can't think of a way, but is there a way to type an argument as accepting all objects except a specific type?

[trim tangle]

nope

[rare vapor]

thanks, that's what I figured

[trim tangle]

You could make something like this: ```py
@overload
def foo(x: str) -> NoReturn: ... # probably a programming error

@overload
def foo(x: Iterable[str]) -> ActualReturnType: ...

[rare vapor]

the best I can think is to do something like this:

@overload
def foo(bar: MyType) -> NoReturn: ...
@overload
def foo(bar: object) -> None: ...

hah

[trim tangle]

It doesn't totally prevent you from calling this function with a MyType either. For example:

def f(x: object) -> None:
    foo(x)

f(MyType())

[rare vapor]

yeah, but it's still not going do what I want, which is to raise a type checking error when MyType is passed in

ok, that's all I needed to know... I wanted to make sure I wasn't forgetting something

[rare scarab]

If only. It would solve the str vs Iterable[str] problem forever.

If only 😦 ```py
def not_foo(bar: object + ~MyType) -> None: ...

[rare vapor]

it would definitely be nice

[jovial lava]

To force it a bit further with more static hints:

@overload
@deprecated("MyType is not supported.")
def foo(bar: MyType) -> NoReturn: ...
@overload
def foo(bar: object) -> None: ...

[rare vapor]

I'd have to tell pyright to mark deprecated usages as errors

but that could work

[jovial lava]

Some editors (ie VSCode + Pylance) will also automatically strikethrough the symbol and display the message in a popup.
It helps catch some cases where calling a method that NoReturn is still valid (like at the end of a branch path, of if you don't check for unreacheable code)

[rare vapor]

I'm working on a project where not everyone is using an editor that does that

[rare scarab]

Does the category= kwarg have any effect in the type checker?

also @deprecated should be placed after @overload if you plan on using typing.get_overloads()

[jovial lava]

I don't think it does

[jovial lava]

Correct. Updated

[iron vapor]

I could be missing something, but I have this method:

    async def count(self) -> int:
        return await self.db_session.scalar(select(func.count(Store.id)))

But scalar's return type is int | None, so I get a pylance error. Except that unless the stores table is missing, I'll always get back an int. How can I tell Python that this is guarenteed to return an int? Or does someone see the scenario where it won't be an int?

[trim tangle]

What type is db_session?

[iron vapor]

a sqlalchemy async session

[trim tangle]

scalar() is typed as always returning an optional https://github.com/sqlalchemy/sqlalchemy/blob/dfc3bf988b27ab980d8da91fe38cbf152ae486a9/lib/sqlalchemy/ext/asyncio/session.py#L476-L496
so the best you can do is # type: ignore or an assert statement

[iron vapor]

Yah, that was my issue (scalar always having an optional)

How would I use assert here?

[trim tangle]

    async def count(self) -> int:
        count = await self.db_session.scalar(select(func.count(Store.id)))
        assert count is not None
        return count

[iron vapor]

I only know assertions from testing

Ah, ok

Never considered using assertions outside testing

I'm trying to think of any situation of when the query could fail other than the table not existing, and I'm not seeing it... and if the table doesn't exist, there are a LOT of problems.

I may just do the type ignore

[trim tangle]

If the table does not exist, you'll hopefully get an exception

[iron vapor]

If the table doesn't exist, I'd hit errors before this query ever comes up 😄

[trim tangle]

It could be deleted while your program is running

[iron vapor]

That's true, though then any query doing modifications to this table would also fail

And that's where I wonder about error catching... should catastrophic errors gracefully fail or be immediately visible?

[oblique urchin]

That's not really a typing question but I'd much prefer for a .count() method like the one you show to raise an exception if it's querying a table that doesn't exist

[iron vapor]

Yah, sorry, my brain kept going from the original question. 15 years as a developer, and I still feel like a mid. I've never found myself at a job where I've had the opportunity to learn something deeper, so I have a lot of shallow knowledge, but struggle with deeper stuff.

[jade viper]

When/why should I use -> Self instead of -> 'MyClass'? Is it an inheritance thing?

[trim tangle]

Self means that the concrete return type changes in a subclass, yes

class Foo:
    def copy(self) -> Self:
        ...

class Bar(Foo):
    pass

x = Foo().copy()  # x: Foo
y = Bar().copy()  # y: Bar

using Self as a parameter automatically breaks LSP

[jade viper]

using Self as a parameter automatically breaks LSP
You mean like for linters?

[trim tangle]

LSP = Liskov Substitution Principle

i.e.: "instance of a subclass must be usable whenever an instance of a superclass is usable"

one of the names of all time

[plain dock]

why does Self break LSP? shouldn't it align with LSP, while hardcoding the return value breaks it?

[jade viper]

LSP = Liskov Substitution Principle
Ah, I thought you meant Language Server Protocol lol

[brave magnet]

hi

[trim tangle]

Consider this: ```py
class Foo:
def copy_from(self, source: Self) -> None:
"Reset self and then copy stuff from source into self"
...

class Bar(Foo):
...

foo1 = Foo()
foo2 = Foo()
bar1 = Bar()
bar2 = Bar()
``` You're allowed to do foo1.copy_from(foo2), but not bar1.copy_from(foo2)

I'm talking about using Self in the parameter position specifically

[plain dock]

🤔 so what would be the compliant method of doing that, then?

[trim tangle]

Of doing what?

[plain dock]

this, but allowing bar1.copy_from(foo2)

[trim tangle]

class Foo:
    def copy_from(self, source: Foo) -> None:

[plain dock]

so in parameters, hardcode the class, but in returns, use -> Self, then, for maximum LSP?

[trim tangle]

-> Self is not necessarily right, it depends on the semantics of the operation

But if you want a rule of thumb, yes, something like that

[trim tangle]

For example, if you wanted to design a custom integer class: ```py
class Int:
def add(self, other: Int) -> Self: ...
def bits(self) -> Int: ...

add returns the same class as this integer. But the number of bits just happens to be an int, it shouldn't vary with subclasses

[jade viper]

That is almost exactly my use case lol, thanks!

In a completely different topic, is there any plans for implementing error raising type hints to Python? e.g. Java's throws SomeException signature

[trim tangle]

I don't think so

[jade viper]

Guido van Rossum has strongly opposed adding exceptions to the type hinting spec, as he doesn't want to end up in a situation where exceptions need to be checked (handled in calling code) or declared explicitly at each level.

Apparently definitely not lol

[trim tangle]

Polymorphism with exceptions gets messy very quickly ```py
class Iterable2[T, EIter: BaseException, ENext: BaseException]:
def iter(self, /) -> Iterator2[T, ENext] raise EIter: ...

class Iterator2[T, ENext: BaseException]:
def iter(self, /) -> Self raise Never: ...
def next(self) -> T raise ENext | StopIteration: ...

def map[A, B, EIter, ENext, EFn](
fn: Callable[[A], B raise EFn | StopIteration],
it: Iterable2[A, EIter, ENext],
/
) -> Iterator2[B, EIter | ENext | EFn]:

I guess you could simplify it to ```py
def map[A, B, E: BaseException](fn: Callable[[A], B raise E], it: Iterable2[A, E, E], /) -> Iterator2[B, E]:

[visual bone]

I would think that how copy is intended dictates that. If you say it returns your the parent class then that's saying you're only getting a copy of the parent. If you say it's copying Self then it's whatever one you're dealing with. But it's not like copy on a subclass wouldn't copy what's on the parent as well, in addition to anything else, it's still substitutable either way I imagine.

[trim tangle]

Yes, to be clear, returning Foo from a method of Foo does not violate substitutability

[tame flame]

Since pyright is complaining about a possibly unbound variable, how would you do this?

def check(val: int) -> bool | None:
  ...

def check_values(values: Iterator[int]) -> tuple[int, bool | None]:
  for val in values:
    res = check(val)
    if res is not None:
      return val, res
  return val, None  # <-- possiblyUnboundVariable

last_val, result = check_values(range(100))

Very simplified example but I have a function that accepts a iterator that I know will not be empty and I need to return a value that fulfils a certain condition or return the last value checked if no value fits.
Of course I could do next() and catch StopIteration and such but I'd like to keep the code simple since I can be certain that the Iterator won't be empty

[trim tangle]

# type: ignore[possiblyUnboundVariable]

[brazen jolt]

do you need to check that the iterator wasn't empty, or do you just want to get rid of the error? I'd just set val = None at the beginning and return tuple[int | None, bool | None]

[tame flame]

Fair 😅

[brazen jolt]

or if you want to check, check, just drop an if above that return ensuring it's not the initial None

if it is, raise ValueError

I'd do it this way instead of handling stopiteration

[tame flame]

Eh, that messes up the return type

[trim tangle]

That might be a better interface for the caller, yes

if an empty iterator is passed, the caller going to receive a sensible error

[tame flame]

Hm yeah, good idea

I can be certain it isn't empty right now but it sure as hell might one day be

[brazen jolt]

well, if it's not supposed to work with empty ones, raise an exc, if it is, you can adjust the return type, or still raise an exc if that'd be more meaningful to the caller

[tame flame]

It's not, I'll do the check for default none and raise

Thanks folks!

[jade viper]

Is it bad practice to use Union[SomeType, OtherType] instead of SomeType | OtherType in Python versions which support the pipe operator? I find the Union approach much cleaner

[oblique urchin]

Generally the | approach is preferred but the Union version isn't going anywhere. If you like it better, go for it

[jade viper]

Awesome, thanks! Just out of curiosity: if I write a Python package using the pipe operator, and someone in a Python version that doesn't support said operator tries to use my package, will that break anything? Or since I'm building (compiling?) the package it doesn't matter?

[oblique urchin]

Yes, it will break in many contexts. The pipe operator is only supported in 3.10 and newer

[jade viper]

Just to clarify, I meant the pipe operator exclusively in a type-hinting context and not in the merge dictionaries and etc. kind of way

[oblique urchin]

Right. Annotations are still executed (unless you use Python 3.14 or from __future__ import annotations), and some types appear outside of annotations

[jade viper]

Got it, thanks Jelle! 😄

[hollow copper]

When you write a package, you'll specify in pyproject.toml the minimum Python version needed to run your code. Then installers will be able to detect if your code is compatible with a user's environment. Then, as you develop your code, say you define the minimum version is 3.9. Then you must take care not to use features that were only introduced after 3.9 (like the pipe operator for type annotations, added in 3.10) in the package. You'll have to decide when you want to cut off older versions of Python. For instance, 3.8 is no longer a supported Python version, so you could reasonably decide to drop support for it. When you make 3.9 the minimum version, then you can start using 3.9 features like type annotations with "list", "dict", etc instead of "List" and "Dict" from the typing module.

You'll also want to see how compatible your package is with new versions (3.13 is the latest release, 3.14 is out in alpha for pre-release testing). But you can only use features that were available as of the oldest version that you support. And if a Python feature that you use is no longer available in a later version (3.12 and 3.13 have been dropping many old stdlib modules, for instance), then you'll need to work out how to handle that case in order to support the new version.

[uncut plover]

Hello. What’s the difference between Type[T] and type[T], if there is one? Is this like where originally we used List[T] but in more recent Python versions we now just use list[T] (upper vs lower case and Typing vs built-in)? Or do Type[T] and type[T] mean different things?

[oblique urchin]

it's like List and list

[uncut plover]

Cool. Thanks. ChatGPT doesn’t know what it’s talking about half the time lol. Glad that humans are still more helpful than AI. For now.

[warm topaz]

ok so i was working on the django-stubs and I ran into a little issue so for example there is this model here

class AbstractBaseUser(models.Model):
    REQUIRED_FIELDS: ClassVar[list[str]]

    password = models.CharField(max_length=128)
    last_login = models.DateTimeField(blank=True, null=True)  # This makes it nullable
    # if explicitly typehinted: models.DateTimeField[str | real_datetime | date | Combinable, datetime | None] 
    is_active: bool | BooleanField[bool | Combinable, bool]

But in the mypy plugin a field is defined as such

_ST_DateField = TypeVar("_ST_DateField", default=str | date | Combinable)
_GT_DateField = TypeVar("_GT_DateField", default=date)
_ST_DateTimeField = TypeVar("_ST_DateTimeField", default=str | real_datetime | date | Combinable)
_GT_DateTimeField = TypeVar("_GT_DateTimeField", default=real_datetime)

class Field(RegisterLookupMixin, Generic[_ST, _GT]): ...

class DateField(DateTimeCheckMixin, Field[_ST_DateField, _GT_DateField]):
    _pyi_private_set_type: str | date | Combinable
    _pyi_private_get_type: date
    _pyi_lookup_exact_type: str | date
    auto_now: bool
    auto_now_add: bool

class DateTimeField(DateField[_ST_DateTimeField, _GT_DateTimeField]):
    _pyi_private_set_type: str | real_datetime | date | Combinable
    _pyi_private_get_type: real_datetime
    _pyi_lookup_exact_type: str | real_datetime

Now the issue is if I don't include | None for the _GT_* fields I get this error from mypy when a field is declared nullable

django-stubs/contrib/auth/base_user.pyi:22: error: DateTimeField is nullable but its generic get type parameter is not optional  [misc]

but that forces an optional on all fields regardless of null being set, another option is the user manually types the fields as such,

last_login: models.DateTimeField[str | real_datetime | date | Combinable, datetime | None] = models.DateTimeField(blank=True, null=True)

which is a lot more tedious for users to do, is there a way i can infer the get type param somehow

[iron vapor]

I'm going through a tutorial on setting up fastapi with various other tools, and there's this code piece:

    lifespan = None

    if init_db:
        session_manager.init(*config.HOST)

        @asynccontextmanager
        async def lifespan(app: FastAPI):
            yield
            if session_manager._engine is not None:
                await session_manager.close()

Copied into VSCode, I get a notice on the first line that Type "None" is not assignable to declared type "(app: FastAPI) -> _AsyncGeneratorContextManager[None, None]". That makes sense, since lifespan is later defined as a function. But I can't figure out what type hint i could assign here to make it ok with the situation. If I don't define lifespan = None, when it's used it gets a possibly unbound notice, so I also tried just doing lifespan or None, which didn't remove the notice. Any thoughts on what I should do here? (in the guide, it says mypy is ok with this? I thought mypy also did static type checking?)

Oh, Callable | None doesn't work either

And if this is just a # type: ignore, I'll go with that

[frigid jolt]

Hi, I have this decorator

P = ParamSpec("P")
T = TypeVar("T")
Coro = Coroutine[Any, Any, T]


class KeyFunc(Protocol):
    def __call__(self, key_format: str, *args: Any, **kwargs: Any) -> str:
        ...


@overload
def redis_cache(
    prefix: str,
    /,
    key_func: KeyFunc = ...,
    key_func_format: str = ...,
    ...
) -> Callable[[Callable[P, Coro[T]]], Callable[P, Coro[T]]]:
    ...

@overload
def redis_cache(
    prefix: str,
    /,
    key_func: Literal[None] = ...,
    key_func_format: Literal[None] = ...,
    ...
) -> Callable[[Callable[P, Coro[T]]], Callable[P, Coro[T]]]:
    ...

def redis_cache(
    prefix: str,
    /,
    key_func: KeyFunc | None = None,
    key_func_format: str | None = None,
    ...
) -> Callable[[Callable[P, Coro[T]]], Callable[P, Coro[T]]]:
  ...

where I would like this to error because key_func is passed but key_func_format isn't (so it should error and require to pass it and vice versa)

@redis_cache("s", key_func=lambda key_format, *a, **k: "")
async def foo(s: int) -> int:
    ...

is there a way to do that

[stable fjord]

isn't the "problem" here that you're doing unpacking when calling and mypy doesn't know that those aren't gonna provide it?

[stable fjord]

this works:

from __future__ import annotations
from typing import Callable
init_db: int = 0
class MyClass:
    aoeu: Callable[[MyClass], None] | None
    if not init_db:
        def aoeu(self) -> None:
            pass
    else:
        aoeu = None
reveal_type(MyClass.aoeu)

[frigid jolt]

where am I doing that?

[stable fjord]

@redis_cache("s", key_func=lambda key_format, *a, **k: "")

[frigid jolt]

that are the lambda arguments

and it's recognized correctly

I'm also using pyright

[grave fjord]

That's weird code, just always define lifespan and move the call to session_manager.init into the lifespan function

you can have an if init_db: inside lifespan

Just remember the else: yield

[stable fjord]

oh sorry

ah yeah it's because you're giving default values to them

provide one overload that doesn't have the kwargs at all

and then one overload that has them, but no default

[iron vapor]

I have no idea if this code is any good, but finding code for SQLA 2 + fastapi has been hard. I figured once I got it working, I'd ask here about the parts I don't get.

[simple field]

what is the correct way to annotate a class decorator that should only be applied to subtypes of a particular class? this is as far as I've gotten but neither pyright or mypy accept it:

from typing import TypeVar, reveal_type

class Base:
    pass

class Child(Base):
    def child_method(self):
        pass

BaseSubtype = TypeVar('BaseSubtype', bound=type[Base])    

def decorator(cls: BaseSubtype) -> BaseSubtype:
    class DecoratedClass(cls):
        def decorated_method(self):
            pass
    return DecoratedClass

base_subtype: type[Base] = Child
    
decorated_child_class = decorator(base_subtype)
decorated_child_instance = decorated_child_class()

decorated_child_instance.child_method()
decorated_child_instance.decorated_method()

reveal_type(decorated_child_class)
reveal_type(decorated_child_instance)

pyright says

Type "type[DecoratedClass]" is not assignable to return type "BaseSubtype@decorator"
  Type "type[DecoratedClass]" is not assignable to type "BaseSubtype@decorator"  (reportReturnType)
Cannot access attribute "decorated_method" for class "Child"
  Attribute "decorated_method" is unknown  (reportAttributeAccessIssue)
Type of "decorated_child_class" is "type[Child]"
Type of "decorated_child_instance" is "Child"

mypy says

main.py:13: error: Variable "cls" is not valid as a type  [valid-type]
main.py:13: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
main.py:13: error: Invalid base class "cls"  [misc]
main.py:16: error: Incompatible return value type (got "type[DecoratedClass]", expected "BaseSubtype")  [return-value]
main.py:23: error: "Base" has no attribute "child_method"  [attr-defined]
main.py:24: error: "Base" has no attribute "decorated_method"  [attr-defined]
main.py:26: note: Revealed type is "type[__main__.Base]"
main.py:27: note: Revealed type is "__main__.Base"
Found 5 errors in 1 file (checked 1 source file)

[simple field]

also tried this with no success

BaseSubtype = TypeVar('BaseSubtype', bound=Base)    

def decorator(cls: type[BaseSubtype]) -> type[BaseSubtype]:
    class DecoratedClass(cls):
        def decorated_method(self):
            pass
    return DecoratedClass

[graceful nova]

can I type hint for a function that takes a class and return an initialize version? or force an attribute to be a specific type not matter the value because my linter is complaining about accessing an attribute

nvm the first statement, I realized the issue is not with the class itself but it is with the attributes

[trim tangle]

pylint is not a very good type checker, it often gets confused

[graceful nova]

yeah I can see that but as I said, can I force it to think otherwise?

[keen flicker]

to take a class and return an initialized version?

def weird_init(tp: type[T]) -> T: ...

but don't know how that'll help you

[graceful nova]

I thought that was the issue originally but no it was an ext issue. thank you tho

[graceful nova]

I am confused, did this fixed it?

tried the same typing in my example file and still

if anyone is interested

[rough sluice]

Click here to see this code in our pastebin.

[fallow vector]

is there a library that rigorously enforces monadic IO?

the sort you'd see in haskell

[grave fjord]

Use ParamSpec and Callable

def instantiate[**P, T](*args: P.args, **kwargs: P.kwargs) -> Callable[[Callable[P, T]], T]:
    def decorator(v: Callable[P, T]) -> T:
        return v(*args, **kwargs)
    return decorator

class _Foo:
    ...

Foo = instantiate()(_Foo)

Remember classes only support the identity decorator

So you need to make a private class and a separate public instance

[hoary pagoda]

no, enforcing isn't an option

[fallow vector]

hm, do we have anything that could do lsp-level annotating?

[rare scarab]

Maybe a flake8 plugin?

[trim tangle]

asyncio/trio/anyio 🙂

[fallow vector]

I'll try those out, thanks a lot!

[trim tangle]

Doesn't enforce it, but if you're doing synchronous I/O, you're going to get event loop blockages (which you can detect with https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.slow_callback_duration)

And there are existing lint rules for running blocking code in async: https://docs.astral.sh/ruff/rules/blocking-open-call-in-async-function/

[fallow vector]

woah thanks a lot for the references! will get reading

[trim tangle]

why do you want this though?

[fallow vector]

I've been programming in haskell for a while and got used to the type system, now have to do a little project for work in python

[trim tangle]

Well, just applying haskell idioms to Python will not work very well

[fallow vector]

I will in general strongly agree, with the sole exception of monadic IO hehe

[trim tangle]

depends on what you mean by monadic IO I guess

you won't get the >>= operator directly with async

well, I guess you can ```py
async def bind[A, B](aw: Coroutine[Any, Any, A], then: Callable[[A], Coroutine[Any, Any, A]]) -> B:
a = await aw
return await then(a)

[fallow vector]

hm... would there be a way to define such a function using a trivial output type instead of B?

[trim tangle]

what do you mean by trivial output type? None?

[fallow vector]

right yes

[trim tangle]

Well, if you have an py async def foo(x: SomeA) -> None: ... and an aw: Coroutine[Any, Any, SomeA], then bind(aw, foo) will produce a Coroutine[Any, Any, None]

In normal Python code you'd never need such a function, you'd just do py async def my_func(): foo = await get_foo() bar = await get_bar(foo) baz = await get_baz(foo, bar) return unbaz(baz) (for the same reason Haskell has do notation)

[fallow vector]

hm I see, and slightly tangentially is there an idiomatic way of doing something like

async def foo(x: SomeA) -> B:
  return (
    actual function body here
  )

[trim tangle]

what do you mean?

[fallow vector]

the actual content, body of the function being what follows return, sort of like:

async def foo ... :
  return (
    func1 >>=
    func2 >>=
    func3 >>=
    trivialvalue
  )

[trim tangle]

well, if you can structure the whole function as a single expression, then you can do just that

[fallow vector]

right I guess that's the painful part without do notation

[trim tangle]

async def foo():
    return compose(func1, compose(func2, compose(func3, lambda x: trivialexpr(x))))

[fallow vector]

hm right I see

tyvm for taking the time, very instructive

[rustic gull]

what does monadic mean here

[trim tangle]

presumably, using a monad

like IO in Haskell

[rustic gull]

reading all this to understand sum 🫡

[rustic gull]

still didn't get it also it all *looked* like a bad idea if not unpythonic

[trim tangle]

yeah, there's no reason to use haskell-style monads in Python

[tranquil turtle]

why does pyright think that enumflagmember.name is str | None?
it is clearly not None

[trim tangle]

!e

from enum import IntFlag

class Flag(IntFlag):
    foo = 1
    bar = 2
    baz = 4

print(Flag.foo, Flag.foo.name)
print(Flag.foo | Flag.bar, (Flag.foo | Flag.bar).name)
print(Flag(0), Flag(0).name)

[rough sluice]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

001 | 1 foo
002 | 3 foo|bar
003 | 0 None

[trim tangle]

IntFlag.name is marked as returning str | None in typeshed, and pyright doesn't do any special-casing to narrow Flag.non_zero_member.name to str

[tranquil turtle]

in my case I have a name for zero member as well

none = 0

:'(

I dont like enums >:(

[trim tangle]

class Thing(Enum):
    foo = 1
    bar = 2
    baz = 4

type ThingFlag = frozenset[Thing]

type Flag = Literal["foo", "bar", "baz"]
type ThingFlag = frozenset[Thing]

Apparently, you have an IntFlag with 30 distinct bits, and over the course of your program you create all possible combinations using |, your program will run out of memory

!e

from enum import IntFlag

class MyFlag(IntFlag):
    foo = 1
    bar = 2
    baz = 4
    spam = 8
    ham = 16

for i in range(32):
    MyFlag(i)

print(MyFlag._value2member_map_)

[rough sluice]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

{1: <MyFlag.foo: 1>, 2: <MyFlag.bar: 2>, 4: <MyFlag.baz: 4>, 8: <MyFlag.spam: 8>, 16: <MyFlag.ham: 16>, 0: <MyFlag: 0>, 3: <MyFlag.foo|bar: 3>, 5: <MyFlag.foo|baz: 5>, 6: <MyFlag.bar|baz: 6>, 7: <MyFlag.foo|bar|baz: 7>, 9: <MyFlag.foo|spam: 9>, 10: <MyFlag.bar|spam: 10>, 11: <MyFlag.foo|bar|spam: 11>, 12: <MyFlag.baz|spam: 12>, 13: <MyFlag.foo|baz|spam: 13>, 14: <MyFlag.bar|baz|spam: 14>, 15: <MyFlag.foo|bar|baz|spam: 15>, 17: <MyFlag.foo|ham: 17>, 18: <MyFlag.bar|ham: 18>, 19: <MyFlag.foo|bar|ham: 19>, 20: <MyFlag.baz|ham: 20>, 21: <MyFlag.foo|baz|ham: 21>, 22: <MyFlag.bar|baz|ham: 22>, 23: <MyFlag.foo|bar|baz|ham: 23>, 24: <MyFlag.spam|ham: 24>, 25: <MyFlag.foo|spam|ham: 25>, 26: <MyFlag.bar|spam|ham: 26>, 27: <MyFlag.foo|bar|spam|ham: 27>, 28: <MyFlag.baz|spam|ham: 28>, 29: <MyFlag.foo|baz|spam|ham: 29>, 30: <MyFlag.bar|baz|spam|ham: 30>, 31: <MyFlag.foo|bar|baz|spam|ham: 31>}

[trim tangle]

that's uhhh

I guess that's expected if you want (Foo.foo|Foo.bar) is (Foo.foo|Foo.bar) to hold

[oblique urchin]

You could probably achieve that with weakrefs too

[oblique urchin]

I feel enums are a great idea but the stdlib enum module tries to do too much stuff

[trim tangle]

that sounds smart actually

WeakValueDictionary maybe

[pale hill]

I have this function here

def process_fish_data(input_fish_list=None) -> tuple[list, list, pd.DataFrame,
                                                     pd.DataFrame]:
    """
    Processes fish data to determine caught and uncaught fish, and returns
    dataframes for uncaught fish in the Northern Hemisphere (NH) and Southern
    Hemisphere (SH).

    Args:
        input_fish_list (list, optional): A list of fish names that have been
                                          caught. Defaults to None.

    Returns:
        (tuple(list, list, pd.Dataframe, pd.Dataframe)):
            caught_fish (list): Caught fish names.
            uncaught_fish (list): Uncaught fish names.
            df_nh_uncaught (DataFrame): Uncaught fish in NH.
            df_sh_uncaught (DataFrame): Uncaught fish in SH.
    """

    ...

    return (caught_fish, uncaught_fish, df_nh_uncaught, df_sh_uncaught)

Is this the right way to type hint a tuple return? This is the only way I can make my mkdocs generation look somewhat decent

But it's a bit hard to read, I think... thoughts?

[oblique urchin]

Consider returning something like a dataclass or namedtuple instead of a complex tuple

[pale hill]

Sorry but I'm not sure what those are, where can I find info about them? I looked up dataclass and am unsure how to use that here in a meaningful way and namedtuple seems interesting...

Actually, is namedtuple just me storing the tuple in a variable and returning the variable lol? No 💀

[pale hill]

I did this, how would I type hint this?

def process_fish_data(input_fish_list=None) -> tuple[list, list, pd.DataFrame,
                                                     pd.DataFrame]:
    FishData = namedtuple(
        "FishData", ["caught_fish", "uncaught_fish", "df_nh_uncaught", "df_sh_uncaught"])

    return FishData(caught_fish, uncaught_fish, df_nh_uncaught, df_sh_uncaught)

[rare scarab]

Move the FishData type up to module level and use typing.NamedTuple

!d typing.NamedTuple

[rough sluice]

typing.NamedTuple

class typing.NamedTuple```
Typed version of [`collections.namedtuple()`](https://docs.python.org/3/library/collections.html#collections.namedtuple).

Usage:

```py
class Employee(NamedTuple):
    name: str
    id: int
```...

[pale hill]

thanks !

[wicked scarab]

a = True
while a:
    b = "1"
    a = False
print(b)
``` How to make this pass the type-checker (mypy and pyright both reports possibly unbound)?

[trim tangle]

You could refactor the while loop to be a while True: loop with a break

[wicked scarab]

I did that earlier but I just wanted to know if there's some shorter solutions like this

also idk why but I don't like seeing the while True statement..

[trim tangle]

you could do py b = b # type: ignore[reportPossiblyUnboundVariable] at the end if you really have while True loops, though it's kinda cursed

it is unfortunate that mypy/pyright don't understand that the body of the loop executes at least once

[wicked scarab]

alright thanks, I'll stick with the while True then

[trim tangle]

I opened an issue with basedpyright: https://github.com/DetachHead/basedpyright/issues/1144 (since pyright says they won't fix this)

[wicked scarab]

wow great! thanks

[prisma talon]

I have 2 classes Question and InputTextResponse. Another class TextQuestion uses multiple inheritance to inherit both of these. How can I typehint a list to tell it that the objects will be subclasses of both of the classes?

[trim tangle]

There's no way to express "a subclass of both Question and InputTextResponses" right now. Maybe just list[TextQuestion]?

[feral wharf]

Something for a protocol maybe?

[prisma talon]

I need it to also support DateTimeQuestion which also inherits from both of them

[trim tangle]

You could also express the requirements as a protocol, yes

[prisma talon]

What is a protocol?

[feral wharf]

!d typing.Protocol

[trim tangle]

https://typing.python.org/en/latest/reference/protocols.html
https://decorator-factory.github.io/typing-tips/tutorials/main/2-using-protocols/

[feral wharf]

Examples are PathLike etc

[prisma talon]

would this be perfered to having a dummy class
class Both(Question, InputTextResponse): pass?

[trim tangle]

depends on what you're going to do with that list later

Is it important that the objects in it are instances of Question and InputTextResponse? Or do they just need to have certain methods/attributes?

[prisma talon]

It is probably important that they are instances of

After a quick skim of protocols it seems like they are a way to define a set of methods and attributes that must exist on an object

[trim tangle]

yep

[trim tangle]

In that case, you might have to define a Both class

[prisma talon]

I guess I should mention that Question is partially an abc and InputTextResponse is fully an abc

[trim tangle]

Basedmypy supports intersection types: https://kotlinisland.github.io/basedmypy/based_features.html#intersection-types
But there hasn't been any recent progress in adding them to the main type system

[prisma talon]

^ this is what I was looking for, unfortunate that it is not in the main system

I will look over the choices after school, thanks!

[stray summit]

anyone aware of theres any new techniques to have multiple dependent types in a definition

correctly type-annotating a def get(name: str, default: D|None, convert: Callable[[str], T] = Identity) -> D|T|str|None

currently both of the "magic" parameters double the number of overloads as T Cannot be inferred by the convert parameter and the concrete choice of the type of default is not available for direct usage

[trim tangle]

Can you show the overloads?

Maybe you can use typevar defaults

[stray summit]

https://github.com/pytest-dev/iniconfig/blob/30a0fa8c8061a351c906e3e663e2988f4ca37dc0/src/iniconfig/__init__.py#L46-L81

last time i tried to make it nice i failed

[trim tangle]

from typing import TypeVar
from collections.abc import Callable

Def = TypeVar("Def", default=None)
Out = TypeVar("Out", default=str)

def _identity(x: str) -> str:
    return x

def get(
    name: str,
    default: Def = None,
    convert: Callable[[str], Out] = _identity,
) -> Out | Def:
    ...

def test() -> None:
    _test0 = get("foo")
    reveal_type(_test0) # str | None

    _test1 = get("foo", convert=int)
    reveal_type(_test1) # int | None

    _test2 = get("foo", convert=int, default=None)
    reveal_type(_test2) # int | None

    _test3 = get("foo", convert=int, default=69)
    reveal_type(_test3) # int

    _test4 = get("foo", default=69)
    reveal_type(_test4) # str | int

    _test5 = get("foo", default="bar")
    reveal_type(_test5) # str

typevar defaults are new, so you'll probably need to use TYPE_CHECKING and typing_extensions

Pyright accepts all of the above. Mypy doesn't like the definition of get, but all the usages in test() work

[stray summit]

hmm, so it looks like the techniques are there but until mypy gets fixed its gonna be tricky

i'll play around again with that when mypy is better

thansk for providing this example - i'll link it up for reference

[jade viper]

Assuming my HexCode class has .hex property, why isn't my linter recognizing that self.value[0] is a str and self.value[1] is a HexCode?

from typing import Tuple

from src.common.enums.better_enum import BetterEnum
from src.common.types.hex_code import HexCode

ShieldsIONamedColorsValue = Tuple[str, HexCode]

class ShieldsIONamedColors(BetterEnum):
    BRIGHTGREEN: ShieldsIONamedColorsValue = ("brightgreen", HexCode("#4c1"))
    GREEN: ShieldsIONamedColorsValue = ("green", HexCode("#97ca00"))
    YELLOW: ShieldsIONamedColorsValue = ("yellow", HexCode("#dfb317"))
    YELLOWGREEN: ShieldsIONamedColorsValue = ("yellowgreen", HexCode("#a4a61d"))
    ORANGE: ShieldsIONamedColorsValue = ("orange", HexCode("#fe7d37"))
    RED: ShieldsIONamedColorsValue = ("red", HexCode("#e05d44"))
    BLUE: ShieldsIONamedColorsValue = ("blue", HexCode("#007ec6"))
    GREY: ShieldsIONamedColorsValue = ("grey", HexCode("#555"))
    LIGHTGREY: ShieldsIONamedColorsValue = ("lightgrey", HexCode("#9f9f9f"))

    # aliases
    GRAY = GREY
    LIGHTGRAY = LIGHTGREY
    CRITICAL = RED
    IMPORTANT = ORANGE
    SUCCESS = BRIGHTGREEN
    INFORMATIONAL = BLUE
    INACTIVE = LIGHTGREY

    @property
    def slug(self) -> str:
        return self.value[0]

    @property
    def hex(self) -> str:
        return self.value[1].hex

[trim tangle]

What's BetterEnum?

And what do you see when you hover over self.value?

[jade viper]

It's just an Enum with an added metaclass to have the .names and .values properties:

class __MetaEnum(EnumMeta):
    @property
    def names(cls) -> List[str]:
        """
        Returns the names of the enum.

        Returns:
            List[str]: Names of the enum.
        """
        return cls._member_names_

    @property
    def values(cls) -> List[str]:
        """
        Returns the values of the enum.

        Returns:
            List[str]: Values of the enum.
        """
        return list(map(lambda x: x.value, cls._member_map_.values()))

[trim tangle]

you could just do [e.name for e in ShieldsIONamedColors] and [e.value for e in ShieldsIONamedColors]

[trim tangle]

Does it work here?

from enum import Enum

class HexCode:
    def __init__(self, s: str) -> None: ...
    @property
    def hex(self) -> str: ...

class NamedColors(Enum):
    GREY = ("grey", HexCode("#555"))
    GREEN = ("green", HexCode("#0f0"))
    RED  = ("RED", HexCode("#f00"))

    GRAY = GREY

    @property
    def slug(self) -> str:
        return self.value[0]

    @property
    def hex(self) -> str:
        return self.value[1].hex

[jade viper]

That's the code I have already :/

[trim tangle]

I mean, does the issue happen with the built-in enum?

[jade viper]

Ah, lemme check

It doesn't

What the hell lol

Wait maybe I know why

Nope

[trim tangle]

You're using Pylance/pyright, right? what version?

pyright playground

[jade viper]

I'm using Pylance but not sure how to check the version, I use it as a vscode extension

From the stdlib code:

class Enum(metaclass=EnumType)...

class EnumType(type)...

EnumType is an alias for EnumMeta

[trim tangle]

interesting ```py
from enum import EnumMeta

class __MetaEnum(EnumMeta):
pass

class BetterEnum(metaclass=__MetaEnum):
pass

class NamedColors(BetterEnum):
foo = 1
bar = 2

def test(self):
    reveal_type(self.value)  # Any

[trim tangle]

Are you sure you need this BetterEnum? you can accomplish the same thing with the above (also having the correct type for values)

[jade viper]

I mean it just makes some other code a bit cleaner

But if it's at the cost of linting it's not worth it

[trim tangle]

I don't know why it thinks the value is Any. You could raise an issue in the pyright repo

[jade viper]

Pylance thinks the same tho

[trim tangle]

yeah, pylance is just pyright with a couple closed source features

so if there's an issue with analysis, you'll be redirected to the pyright repo if you raise an issue on the pylance repo

[jade viper]

Got it

Give or take, starting with how many literals for a Literal[str] should I just type something as a str? lol

[trim tangle]

Apparently pyright has a limit of 63 ```py
def f(
v2: Literal["a", "b"],
v3: Literal["a", "b", "c"],
v5: Literal["a", "b", "c", "d", "e"],
v7: Literal["a", "b", "c", "d", "e", "f", "g"],
) -> None:
w27 = v3 + v3 + v3
w32 = v2 + v2 + v2 + v2 + v2

w36 = v2 + v2 + v3 + v3 
w48 = v2 + v2 + v2 + v2 + v3
w50 = v2 + v5 + v5
w63 = v3 + v3 + v7

# ^ Literal[...]
# ---
# V LiteralString

w64 = v2 + v2 + v2 + v2 + v2 + v2

[jade viper]

Thanks!

!e

from enum import StrEnum


class BadgeStyles(StrEnum):
    FLAT = "flat"
    FLAT_SQUARE = "flat-square"

    TRUE_FLAT = "flat"
    TRUE_FLAT_SQUARE = "flat-square"

print(BadgeStyles.TRUE_FLAT.name)

Can I bypass this?

[rough sluice]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

FLAT

[jade viper]

Without junk code that is

I found aenum but wasn't too keen on adding dependencies

[oblique urchin]

If anyone has suggestions for what to name a function for formatting types in a human-friendly way, I'd love to hear them. https://github.com/python/typing_extensions/issues/544

[wary ermine]

hi

I just want help for this code

sayi1=int(input("Sayınızı Giriniz :"))
if sayi1 % 2 == 0 :
q=0
sayi2=sayi1/2
else:
q=1
sayi2=(sayi1-1)/2
if sayi2 % 2 == 0 :
w=0
sayi3=sayi2/2
else:
w=1
sayi3=(sayi2-1)/2
if sayi3 % 2 == 0 :
e=0
sayi4=sayi3/2
else:
e=1
sayi4=(sayi3-1)/2
if sayi4 % 2 == 0 :
r=0
sayi5=sayi4/2
else:
r=1
sayi5=(sayi4-1)/2
if sayi5 % 2 == 0 :
t=0
sayi6=sayi5/2
else:
t=1
sayi6=(sayi5-1)/2
if sayi6 % 2 == 0 :
y=0
sayi7=sayi6/2
else:
y=1
sayi7=(sayi6-1)/2
if sayi7 % 2 == 0 :
u=0
sayi8=sayi7/2
else:
sayi8=(sayi7-1)/2
u=1
if sayi8 % 2 == 0 :
o=0
sayi9=sayi8/2
else:
o=1
sayi9=(sayi8-1)/2
if sayi9 % 2 == 0 :
p=0
sayi10=sayi9/2
else:
p=1
sayi10=(sayi9-1)/2
if sayi10 % 2 == 0 :
a=0
sayi11=sayi9/2
else:
a=1
sayi11=(sayi9-1)/2
if sayi11 % 2 == 0 :
s=0
else:
s=1
print(s,"(2^10)",a,"(2^9)",p,"(2^8)",o,"(2^7)",u,"(2^6)",y,"(2^5)",t,"(2^4)",r,"(2^3)",e,"(2^2)",w,"(2^1)",q,"(2^0)")

how am i gonna short this codes

[rare scarab]

!e Using a loop should work. ```py
sayi = 1000
items = []
for i in range(11):
items[:0] = [sayi % 2, f"(2^{i})"]
sayi /= 2
print(*items)

[rough sluice]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

0.9765625 (2^10) 1.953125 (2^9) 1.90625 (2^8) 1.8125 (2^7) 1.625 (2^6) 1.25 (2^5) 0.5 (2^4) 1.0 (2^3) 0.0 (2^2) 0.0 (2^1) 0 (2^0)

[rare scarab]

It's close enough

[rustic gull]

hey, for fast api and basemodel pydantic, does it matter if the variable is a float or int?

from fastapi import FastAPI
import joblib
from pydantic import BaseModel
import yaml
import uuid
from fastapi.responses import RedirectResponse
import numpy as np


with open("cof.yml", 'r') as ymlfile:
    config = yaml.load(ymlfile, Loader=yaml.SafeLoader)


MODEL_DIR = config['MODEL_DIR']
VERSION = config['VERSION']
HOST = config['HOST']
PORT = config['PORT']

app = FastAPI()
linreg_model = joblib.load("model/linreg.joblib")
lasso_model = joblib.load("model/lasso.joblib")
ridge_model = joblib.load("/model/ridge.joblib")
gbr_model = joblib.load("/model/gbr_model")

class input_data(BaseModel):
    year:int
    age:int
    beds:int
    baths:float
    home_size:float ```

[cinder bone]

why not just type_to_string instead of value_to_string?
Also I do like to_type_str

[oblique urchin]

Part of the thinking was that annotationlib is meant for annotations in general, which may not always be types (types are just the most common conventional use for annotations)

But given that what the function does is mostly useful for types, I think a name with type is fine

[warm topaz]

pretty_type sounds good
or maybe format_type, humanize_type

[obsidian oar]

Add a Intersection type
Intersection[A, B] == <subclass of A & B>

[restive rapids]

would be cool but its literally a 10+ year old problem at this point https://github.com/python/typing/issues/18

[obsidian oar]

I would like to add intersections in py3.14

assert A & B == Intersection[A, B]

type[A & B] == type[Intersection[A, B]]

[desert delta]

could anyone do one of two things:

• explain how to change this code to pass the type checker
• explain why this code will never pass the type checker

Basically I have to choose between using a covariant type parameter OR using that type parameter in a class method. But I would really like to do both.

from abc import ABC
from dataclasses import dataclass
from typing import ClassVar, Generic, Self, TypeVar


class Base:
    foo: int

# covariant=True is illegal because T is used as a parameter in the wrap method
# T = TypeVar('T', covariant=True, bound=Base)
T = TypeVar('T', covariant=False, bound=Base)

@dataclass
class Wraps(Generic[T], ABC):
    wrapped_cls: ClassVar[type[T]] # type: ignore[misc]
    foo: int

    def unwrap(self) -> T:
        return self.wrapped_cls(foo=self.foo)

    @classmethod
    def wrap(cls, wrapped: T) -> Self:
        return cls(foo=wrapped.foo)

class A(Base):
    bar: int

class WrapsA(Wraps[A]):
    wrapped_cls = A

x: Wraps[Base] = WrapsA(foo=1)
# error: Incompatible types in assignment (expression has type "WrapsA", variable has type "Wraps[Base]")  [assignment]
# this assignment is allowed if T is covariant, but then the wrap method loses its signature

[desert delta]

my current thinking is that I have to lose type safety in the wrap method, e.g. using wrap(cls, wrapped: Base)

[obsidian oar]

from dataclasses import dataclass
from typing import ClassVar, Generic, Self, TypeVar, cast

BaseT = TypeVar("BaseT", bound="Base", covariant=True)


@dataclass
class Base:
    foo: int


@dataclass
class Wraps(Generic[BaseT]):
    wrapped_cls: ClassVar[type[Base]]
    foo: int

    def unwrap(self) -> BaseT:
        wrapped_cls = cast(type[BaseT], self.wrapped_cls)
        return wrapped_cls(foo=self.foo)

    @classmethod
    def wrap(cls, wrapped: Base) -> Self:
        return cls(foo=wrapped.foo)


class A(Base):
    bar: int


class WrapsA(Wraps[A]):
    wrapped_cls = A


x: Wraps[Base] = WrapsA(foo=1)

All subclasses of Base must be callable with the Base signature (foo: int) -> Self

[desert delta]

So basically I have to lose type-safety in the wrap fn

[obsidian oar]

Yes, in the wrap function, wrapped is an instance of any subclass of Base. Since it only needs the foo attribute, there's no need to save it as BaseT, but the return type of Self would have to be determined.

Wraps[A].wrap(...)

@desert delta

If you have py>=3.13 this is possible

BaseT = TypeVar("BaseT", bound="Base", covariant=True, default=Base)

from typing import reveal_type

reveal_type(Wraps.wrap(A(5)))  # Wraps[A]

[desert delta]

thanks, that's interesting.

[obsidian oar]

from dataclasses import dataclass
from typing import ClassVar, Generic, Self, TypeVar, cast, reveal_type


@dataclass
class Base:
    foo: int


BaseT = TypeVar("BaseT", bound="Base", covariant=True, default=Base)


@dataclass
class Wraps(Generic[BaseT]):
    wrapped_cls: ClassVar[type[Base]]
    foo: int

    def unwrap(self) -> BaseT:
        wrapped_cls = cast(type[BaseT], self.wrapped_cls)
        return wrapped_cls(foo=self.foo)

    @classmethod
    def wrap(cls, wrapped: Base) -> Self:
        return cls(foo=wrapped.foo)


class A(Base):
    bar: int


class WrapsA(Wraps[A]):
    wrapped_cls = A


x: Wraps[Base] = WrapsA(foo=1)

reveal_type(Wraps.wrap(A(5)))  # Wraps[Base]
reveal_type(Wraps[A].wrap(A(5)))  # Wraps[A]

[desert delta]

one of the ideas behind the implementation of wrap(cls, wrapped: BaseT) is that it enforces at the type level that you should only call wrap on a specific type

[obsidian oar]

but Wraps must have a value for wrapped_cls for this case

[desert delta]

yes the idea is that every subclass of Wraps sets wrapped_cls

and it is used to check that only instances of wrapped_cls are wrapped

[obsidian oar]

If classproperty were still allowed, you could do this.

    @property
    @classmethod
    def wrapped(cls) -> type[BaseT]:
        return cast(type[BaseT], cls.wrapped_cls)

[grave kettle]

I'm running into something that's troubling me, and I can't figure it out (maybe because I've been coding all day).

Given this snippet:

from typing import TypeVar

TExc = TypeVar("TExc", bound=Exception)

def jawa(ewok: type[TExc] = Exception):
    raise ewok("Wooooo")

mypy spits this out:

dummy.py:5: error: Incompatible default for argument "ewok" (default has type "type[Exception]", argument has type "type[TExc]")  [assignmen
t]
Found 1 error in 1 file (checked 1 source file)

Shouldn't this be fine given that Exception is bound to the TExc TypeVar?

[grave fjord]

I think probably the error is making no sense, but you only use TExc once in that def

so type[TExc] is type[Never]

[obsidian oar]

There is no need to use TypeVar, since there is no need to retrieve the exception type elsewhere.

from typing import NoReturn

def jawa(ewok: type[Exception] = Exception) -> NoReturn:
    raise ewok("Wooooo")

[grave kettle]

This is just a toy example to show the mypy error, the actual function is:

def require_condition(
    expr: Any,
    message: str,
    raise_exc_class: type[TExc] = Exception,
    raise_args: Iterable[Any] | None = None,
    raise_kwargs: Mapping[str, Any] | None = None,
    exc_builder: Callable[[ExcBuilderParams[TExc]], TExc] = default_exc_builder,
):
    """
    Assert that an expression is truthy. If the assertion fails, raise an exception with the supplied message.

    Args:

        message:         The failure message to attach to the raised Exception
        expr:            The value that is checked for truthiness (usually an evaluated expression)
        raise_exc_class: The exception type to raise with the constructed message if the expression is falsey.
                         Defaults to Exception.
                         May not be None.
        raise_args:      Additional positional args (after the constructed message) that will passed when raising
                         an instance of the ``raise_exc_class``.
        raise_kwargs:    Keyword args that will be passed when raising an instance of the ``raise_exc_class``.
        exc_builder:     A function that should be called to construct the raised ``raise_exc_class``. Useful for
                         exception classes that do not take a message as the first positional argument.
    """

    if not expr:
        raise exc_builder(
            ExcBuilderParams[TExc](
                raise_exc_class=raise_exc_class,
                message=message,
                raise_args=raise_args or [],
                raise_kwargs=raise_kwargs or {},
            )
        )

[grave kettle]

A better toy example that shows the same behavior:
(yes, I know it's impractical and silly, but it shouldn't be a problem for mypy)

from typing import TypeVar, Any

TExc = TypeVar("TExc", bound=Exception)

def exc_builder(exc: type[TExc] = Exception, *args: Any, **kwargs: Any) -> TExc:
    return exc(*args, **kwargs)

[trim tangle]

I'm not sure there's a way to explain this to mypy without overloads 🤔

[bronze juniper]

Hello, I have defined a few global variables (that are supposed to be constant) on my code and I would like to use them to type hint one argument. Something like

class ConstantClass:
       ... # Whatever you put inside but it is constant.


CONSTANT1 = ConstantClass(1)
CONSTANT2 = ConstantClass(2)
CONSTANT3 = ConstantClass(3)


def my_function(arg: Literal[CONSTANT1, CONSTANT2, CONSTANT3] = CONSTANT1):
     ... # whatever

Of course this doesn't work as I can't say Literal[CONSTANT1], is there another way of doing this ?

[trim tangle]

I thought typevar defaults were supposed to solve this, but no, same problem ```py
from typing import TypeVar, Any

TExc = TypeVar("TExc", bound=Exception, default=Exception)

def exc_builder(
exc: type[TExc] = Exception,
*args: Any,
**kwargs: Any,
) -> TExc:
return exc(*args, **kwargs)

[trim tangle]

Literal only supports int, bool, str, bytes, and enum members

Maybe you want to look into the enum module?

[bronze juniper]

I know, that's why I am looking for an alternative

[trim tangle]

def my_function(arg: ConstantClass = CONSTANT1):
     ... # whatever

[bronze juniper]

Well, I really want to not make them enums bc they would not be unique (like I can't say CONSTANT1 = 1, CONSTANT2 = 2 etc.)

[bronze juniper]

issue with this is that it implies that all instances of ConstantClass is okay, which is not true

[trim tangle]

I'm not sure what you mean

[bronze juniper]

the ConstantClass is a little complex but I use them because they store a few values. But it is not unique. Like CONSTANT1 could have the exact same attributes as CONSTANT5 but I still want CONSTANT5 to not be accepted

it is more a logical point of view than a coding issue. Of course it would work if you use CONSTANT5, but you are not supposed to

(Maybe I shouldn't do that in the first time)

[trim tangle]

class MyFunctionOptions(Enum):
    OPT1 = ConstantClass(1)
    OPT2 = ConstantClass(2)
    OPT3 = ConstantClass(3)


def my_function(arg: MyFunctionOptions = MyFunctionOptions.OPT1):
    # use arg.value

[bronze juniper]

That would do, but I really wanted to use the exact same objects, to avoid having to many objects in the library

[trim tangle]

maybe you can provide more concrete details?

[bronze juniper]

Sure

I am working on defining anchors for an UI. Basically I use them so when I specify the position of something on the screen, I can give a x, a y and an anchor.
If the anchor used is CENTER, then (x,y) is the center of the object. If we use anchor.TOP_LEFT, then (x,y) is the top left of the object on the screen and so on.

In some cases, like when instancing a text entry, we can also use a 1D anchor (right, center, left) to specify how the text is justiftied. Now, I am making some progress bars. And I need to specify wether the fixed point is the left and the right part extends or retracts, or if it is the right that doesn't move and the left that moves. So for that I want to use something like Literal[RIGHT, LEFT]. So RIGHT and LEFT can be used to specified the right or left fix edge, and in other contexts, the same RIGHT, LEFT and CENTER objects can be used to specify the justification of text entries or labels.

It is very practical for the calculations behind to say that LEFT is equivalent to 0 and RIGHT is equivalent to 1. In the same way, TOP is 0 and BOTTOM is 1, CENTER is 0.5. So I cannot define the 5 and use an Enum, or TOP and BOTTOM would be aliases of LEFT and RIGHT, which I don't really want.

Maybe I should have an Enum with FIXED_RIGHT and FIXED_LEFT argument specifically for the progress bar but I really feel like the less constants the better

I can send the full code of the Anchor classes if you want to, I just don't know how.

[grave kettle]

yeah, pyright and basedpyright have no issue with this

[trim tangle]

class Anchors(Enum):
    left = Vec(-1, 0)  # value could be something else, maybe even a string
    right = Vec(1, 0)
    top_left = Vec(-1, -1)
    ...

LEFT: Literal[Anchors.left] = Anchors.left
RIGHT: Literal[Anchors.right] = Anchors.right
...

def float_ui_element(element: Element, to: Literal[Anchors.left, Anchors.right]) -> None:
    ...

float_ui_element(some_element, LEFT)  # ok
float_ui_element(some_element, Anchors.left)  # ok
float_ui_element(some_element, Anchors.top_left)  # error from a type checker
``` something like this?

[bronze juniper]

That's a good idea, I could just implement things such that the LEFT, RIGHT and all 1D anchors are just 2D Anchors without telling it. That way, no more duplicates (so no more aliases)

But apart from that, there is no way to type hint a parameter with the accepted values other that by using Literal ?

[trim tangle]

Yes, Literal is the only way to specify a range of fixed values

[bronze juniper]

okay thanks

[trim tangle]

I'm not sure if there's an existing type system where something like that exists

[obsidian oar]

ExcBuilderParams must defer the generic argument from the raise_exc_class argument, but since it always throws an exception, the type of the error is always lost and it makes no sense to give it type arguments
but all of the above is invalidated if you return an object that stores the type of the exception

[rare scarab]

I think None is also valid, but there's no reason to use it like that.

[trim tangle]

Actually, you're right https://typing.python.org/en/latest/spec/literal.html#legal-parameters-for-literal-at-type-check-time

I misread pyright's error message

[grave fjord]

I think you can do this with overloads

[jade viper]

How can I declare "private" dataclass attributes so that they don't appear in linters? e.g.:

from dataclasses import dataclass
from typing import Optional

@dataclass
class SomeDataClass:
    slug: str
    message: Optional[str] = None
    color: ShieldsIOColor = ShieldsIONamedColor.BLUE
    __color: Optional[str] = None

Not sure if this is the right channel for this tbh

[brazen jolt]

you can use dataclass.field and set init=False if you just want it to not be settable from constructor

[jade viper]

Ah perfect that's exactly what I want

[brazen jolt]

from dataclasses import dataclass, field

@dataclass
class Foo:
    public: int
    _private: int = field(init=False)

[jade viper]

And do I type-hint it as any type?

Yup

Ok, thanks so much! 😄

[jade viper]

_private

Is the correct prefix for private stuff on Python one underline or two? I never seem to grasp it

[brazen jolt]

one, two would result in something called name mangling

it's a very rarely used feature in python, that's mostly to do with conflicts in variable naming in subclasses

[jade viper]

Isn't name mangling what turns private things "unreachable" though, by adding the _Class__ prefix?

[brazen jolt]

it's not really unreachable

python doesn't actually have proper private variables

it's just that variables prefixed with _ should be considered as private by convention

[jade viper]

I know, I used quotes cause it just makes it harder to do

[brazen jolt]

but in practice, using two underscores like this for name mangling is very rare, I wouldn't use it just to make your variables private, a single _ is a much better option and fits with the standards that you see pretty much everywhere

even in stdlib, you don't really see a lot of name mangling, while you do see a bunch of single _ prefixes for private stuff, I'd recommend you stick to that

[jade viper]

Cool, thanks!

On another topic: how do I create docstrings with types for enum.Enum classes? e.g.:

from enum import Enum
from typing import TypedDict


class MyEnumValue(TypedDict):
    property_1: int
    property_2: int


class MyEnum(Enum):
    ONE: MyEnumValue = {'property_1': 1, 'property_1': 2}

    @property
    def property_1(self) -> int:
         return self.value['property_1']

    @property
    def property_2(self) -> int:
         return self.value['property_2']

How do I indicate that the values for this enum are of the typeddict class and that it's values can be accessed by properties without making someone open the class?

[brazen jolt]

This is kind of cursed, enum variants will always be of the MyEnum type and usually, the actual value is an int. Enums in python are kind of weird, as the variants are just "magically" populated class variables, where the original value of that classvar is replaced with an instance of that enum, holding that value.

Doing this likely won't do what you want, as ONE would be a class var, using MyEnum.ONE doesn't mean instantiation, the instantiation happens at class definition time, and having the value be mutable will only lead to issues.

[jade viper]

So what would be the correct way to do this?

What about for the enum values themselves? I found a post on StackOverflow, but not sure it is up to date. e.g.:

from enum import Enum
from typing import Annotated


class Color(Enum):
    RED: Annotated[int, "The color red"] = 1

[brazen jolt]

if this is just about docstrings for the enum variants, you can do it normally, same way you'd add them for any other class variables:

[jade viper]

That seems cluttered af though lol, having one block of comments for each variant

[brazen jolt]

well yeah, but that is the "proper" way to do this

in many cases, the name of the variant is perfectly sufficient though

I generally wouldn't be adding docstrings to the individual variants unless there's a need fo it

[tranquil turtle]

if not TYPE_CHECKING: ...

[brazen jolt]

I'd definitely not go with this route, their goal was just to not make that var a part of the constructor

[brazen jolt]

the Annotated approach wouldn't really work here, as documentation libraries like mkdocs or sphinx wouldn't pick that up. Also, specifying type annotations for enum variants is something you're not supposed to do at all

Annotated is really only meant to be used for embedding metadata into an annotation, that libraries can then extract on runtime, it's not meant for documentation

[jade viper]

Was just going off of the StackOverflow post I mentioned

[brazen jolt]

pyright doesn't even show the content of it for example

[jade viper]

Hmm

[jade viper]

It mentions this PEP which seems interesting, but it's still a draft

[brazen jolt]

Oh, I haven't seen that one before, that is interesting

still though, I'd just stick to the usual way to add docstrings for variables / class variables / constants

since like I said, enum variants shouldn't be annotated at all

and pyright still doesn't show the annotation with that, though that might just be because it doesn't yet support it, even though typing_extensions do seem to already contain Doc

[jade viper]

Well the PEP is still a draft anyway so it's likely not done

[trim tangle]

the PEP author certainly doesn't make it easy to like the PEP with this usage example
https://github.com/fastapi/fastapi/blob/8e76d4e5f4ee2e4b9f77378a39aa15940255e164/fastapi/applications.py#L64-L822
(that's the signature of FastAPI.__init__, 750 lines long)

[jade viper]

since like I said, enum variants shouldn't be annotated at all
That's fair, it's just that I was creating an enum for styles and wanted to give a short description on what each style is

[brazen jolt]

this is perfectly valid then

alternatively, you could describe the variants in the class docstring

[jade viper]

Hmmm I like that better as far as code cleanliness goes

Thanks! 😄

[trim tangle]

I don't understand why they surveyed 5 languages, included the results in the PEP and ignored them

😩

[brazen jolt]

lol

[jade viper]

We must hope

Jelle who is very active in this channel is actually the sponsor for the PEP

[trim tangle]

Yeah, I know

[brazen jolt]

yeah, jellie sponsors a lot of typing peps, even authors a whole bunch

[trim tangle]

Doc[int, "str"] would be better at least

[jade viper]

It's rejected

[trim tangle]

This idea was rejected as it would only support that use case and would make it more difficult to combine it with Annotated for other purposes ( e.g. with FastAPI metadata, Pydantic fields, etc.) or adding additional metadata apart from the documentation string (e.g. deprecation).
🤔 why would it be difficult

foo: Annotated[Doc[int, "The Foo value"], deprecated("use bar instead"), Field(gt=0)]

presumably Doc[T, S] would be an alias to something like Annotated[T, typing.DocInfo(S)]

Annotated[Annotated[T, A1], A2] collapses into Annotated[T, A1, A2], so it just works

[jade viper]

https://tenor.com/iT6gMPRoaCs.gif

Maybe contribute to the discussion: https://discuss.python.org/t/pep-727-documentation-metadata-in-typing/32566

Or tag Jelle

[trim tangle]

that's probably suggested somewhere in that 215-mesage-long discourse thread

[jade viper]

Yup lol, when I saw the size I decided not to look for it

[trim tangle]

https://discuss.python.org/t/pep-727-documentation-metadata-in-typing/32566/181
https://discuss.python.org/t/pep-727-documentation-metadata-in-typing/32566/183 🤔

we’ll probably keep it in typing_extensions indefinitely even if the PEP gets withdrawn or rejected, for backwards compatibility reasons.

[jade viper]

[trim tangle]

It's an interesting situation. If editors/documentation generators just keep supporting typing_extensions.Doc, it will be that the PEP got rejected but implemented

[oblique urchin]

hi

[jade viper]

Hi! We were discussing about PEP 727 and why it needed such a verbose implementation

[brazen jolt]

jellie!

[jade viper]

How can I type hint a enum class instantiated from the enum functional API? e.g.:

from enum import Enum


MyEnum = Enum('MyEnum', {'foo':42, 'bar':24})

Pyright recognizes MyEnum as a variable (Any)

[brazen jolt]

yeah, this is way too dynamic for type checkers, I mean, you could make a proper MyEnum enum class that matches this and type hint it as that 😛

[jade viper]

You mean like?

from enum import Enum


class _MyEnum(Enum):
    pass


MyEnum: _MyEnum = Enum('MyEnum', {'foo':42, 'bar':24})

Sad that type checkers don't work with some stdlib stuff :/

[brazen jolt]

you still won't get the variants here

[brazen jolt]

thing is, this is simply too dynamic for that

what if the 2nd arg was a dict passed as a variable?

[jade viper]

Oh, I just want it to get recognized as a type, nothing fancy

[brazen jolt]

hm

for me, pyright can actually dynamically pick it up as a type

what's kind of surprising is that it even knows about the x variant

maybe you're using an older version?

not with this one though

[jade viper]

I'm using the latest Pylance extension version in vscode, dunno how to check the underlying Pyright version

[brazen jolt]

that I don't know how to help you with, I'm using pure pyright as an LSP, with which it works properly, you can also see it working on pyright playground: https://pyright-play.net/?code=GYJw9gtgBApgdgV2gSwgBzCALlAooiAKFEiiwE81k4BzKVDbKEGANxgEMAbAfQrRiFCAWXL4kUALx4CACgBEo8RHkAaKAG95AD3kAuKACYAvgEpCLdtz6UYspQXOXOvfnYdIAdNqdsXNgXsxAk9yUyA

[icy obsidian]

class Signal:
    def __init__(self) -> None:
        self._callbacks: set[int] = set()
    
    def subscribe(self, f) -> None:
        self._callbacks.add(f)
        reveal_type(f)

Why in this case 'f' is of Unknown type? Or do I need to explicitly specify the type in function signature?

[icy obsidian]

Or why does it not produce a type error in add()

[grave fjord]

You didn't type annotate it

You need f: int

[grave fjord]

It doesn't produce a type error by default because mypy doesn't type error in partially annotated functions

[icy obsidian]

This somehow seems a bit counterintuitive for me...

    def subscribe(self, f) -> Self:
        self._callbacks.add(f)
        b = "asd"
        a: list[int] = []
        a.append(f)  # <<< No error
        a.append(b)  # <<< Gives an error

        return self

[spiral fjord]

It'll error if you use strict mode

[icy obsidian]

Yea, you are right.

[spiral fjord]

It's gradual typing and allows people to slowly upgrade their existing codebases to be typed without having the entire codebase that isn't typed yet error

[icy obsidian]

Is it possible to use TypeAlias of a ParamSpec callable in Generic class?

Something like this

SignalParameterTypes = ParamSpec('SignalParameterTypes')
SignalCallable: TypeAlias = Callable[SignalParameterTypes, None]

class Signal[**SignalParameterTypes]:
    def __init__(self) -> None:
        self._callbacks: set[SignalCallable] = set()
        reveal_type(self._callbacks)

It reveals to a "..." Callable

[restive rapids]

the [] introduce a new typevar

[icy obsidian]

And how does it interact in this case?

[restive rapids]

it doesnt, the SignalParameterTypes mentioned in SignalCallable and the one introduced by the new syntax have nothing to do with each other
i dont think you can type-alias like that

[icy obsidian]

So I have to use only one of those here?

[restive rapids]

no, you will have to provide the type parameter (the signature for SignalCallable, and probally SignalAction too but we cant see its definition)

[icy obsidian]

Oh, my mistake - forgot do remove it

Updated the previous code

[restive rapids]

^ here another issue is that there's nowhere to bind the parameter types on init from

[icy obsidian]

That is why I thought that specifying the class as Generic would work.

So I either use old style and explicitly specify type parameter, or the new style and... I have no idea how to alias some types then

[restive rapids]

it seems like pyright does allow using local typevars in scoped type aliasing, but mypy doesnt

from __future__ import annotations
from collections.abc import Callable

class Signal[**P]:
    type Handler = Callable[P, None]
    def __init__(self, handlers: set[Handler]) -> None:
        self.handlers = handlers
    def emit(self, /, *args: P.args, **kwargs: P.kwargs) -> None:
        for handler in self.handlers:
            handler(*args, **kwargs)

so have to do

from __future__ import annotations
from collections.abc import Callable

type Handler[**P] = Callable[P, None]

class Signal[**P]:
    def __init__(self, handlers: set[Handler[P]]) -> None:
        self.handlers = handlers
    def emit(self, /, *args: P.args, **kwargs: P.kwargs) -> None:
        for handler in self.handlers:
            handler(*args, **kwargs)

[icy obsidian]

future is not needed in the second case, right?

[restive rapids]

its not needed in the first case either, i just always use it to not have to worry about evaluation order and whats defined or not

[icy obsidian]

Oh

class Signal[**SignalParameterTypes]:
    type SignalCallable = Callable[SignalParameterTypes, None]

    def __init__(self, handlers: set[SignalCallable]) -> None:
        self._callbacks: set[SignalCallable] = handlers  # <<<< Error here (function) SignalCallable: Unknown

This is strange

[restive rapids]

remove the explicit annotation where you set the attribute, SignalCallable is not defined here

[icy obsidian]

Specifying it outside the class works though

[icy obsidian]

But what if I need to set it to an empty Set there (without explicitly passing Handlers in init)

[restive rapids]

how would typecheckers know what to bind SignalParameterTypes to?

[icy obsidian]

I'd assume to a generic type specified during instantiation?

[restive rapids]

the behaviour of pyright with the first one seems to be weird, it only works for attribute annotations in the class body & the signature of the init
so i guess just provide the type parameters explicitly when you use them

[restive rapids]

and yeah you can do that

from collections.abc import Callable
type Handler[**Signature] = Callable[Signature, None]

class Signal[**Signature]:
    def __init__(self):
        self.handlers: set[Handler[Signature]] = set()

signal = Signal[int]()

[icy obsidian]

type SignalCallable[**SignalParameterTypes] = Callable[SignalParameterTypes, None]

class Signal[**SignalParameterTypes]:
    def __init__(self, protect_callbacks: bool=False) -> None:
        self._callbacks: set[SignalCallable[SignalParameterTypes]] = set()
    def subscribe(self, f: SignalCallable):
        self._callbacks.add(f)


def f(i: int): pass
def f2(s: str): pass

a = Signal[int]()
a.subscribe(f)
a.subscribe(f2)

Does not produce any errors

And do I need to specify Signature in "self.handlers: set[Handler[Signature]] = set()" explicitly, as making an alias makes no sense in this case

[restive rapids]

you probally arent on a strict config, f: SignalCallable isnt valid - it doesnt provide the type parameters, so its practically just "any callable to None"

[icy obsidian]

def subscribe(self, f: SignalCallable[SignalParameterTypes]):

[restive rapids]

yeah

[icy obsidian]

This typo gives back an error

But it means ther are no aliases in use

[restive rapids]

yes

[icy obsidian]

So I guess I cannot use aliases in this case at the moment?

[restive rapids]

yeah it seems like you'll have to explicitly specify the typevars each time you need them
consider making shorter names for them 🍋 just P would be understandable, for parameters

[icy obsidian]

I wanted to shorten those as they will expand a lot there if specified explicitly :<

Well, yeah I guess. It's just my habit. Guess will have to shorten them in the first place

[icy obsidian]

Just to double check - the solution here would be to explicitly specify TypeVars and ParamSpecs where needed all the time?

[restive rapids]

seems like it. i had hopes for the local alias here but its weird, only works on attribute & init annotations. probally a bug that it works for them in the first place lol

[icy obsidian]

Yea. That's a bit unfortunate not to be able to use alias here.
Thanks for the help!

[cinder bone]

Just came across this https://discuss.python.org/t/pep-728-typeddict-with-typed-extra-items/45443/144, I'm hopeful 🤞

[jade viper]

What is the proper way to document exceptions raised by an exception group? e.g.

def some_method():
    """
    My method.

    Raises:
        ValueError: Some value error.
        TypeError: Some type error.
    """
    ex1 = ValueError("value_error")
    ex2 = TypeError("type_error")

    raise ExceptionGroup("exception_group", [ex1, ex2])

or

def some_method():
    """
    My method.

    Raises:
        ExceptionGroup: ???
    """
    ex1 = ValueError("value_error")
    ex2 = TypeError("type_error")

    raise ExceptionGroup("exception_group", [ex1, ex2])

[grave fjord]

What's your usecase?

You can't type hint it of course...

[jade viper]

I have a script that validates a json data file, and I want it to raise error for all invalid lines instead of just one, so I use a ExceptionGroup

[jade viper]

Not typehint it per-se, but document it in a docstring

[grave fjord]

So it's all ValidationError say?

[jade viper]

Yup

[grave fjord]

Like what cattrs does?

See how they document it?

Also look at Happy eye balls implementations

[jade viper]

Could you please link me those?

[grave fjord]

https://trio.readthedocs.io/en/stable/reference-io.html#trio.open_tcp_stream

This one raises OSError() from ExceptionGroup(...)

[thorn osprey]

Is there a way to simplefy
dict[str, dict[X, set[Callable[[X], None]]]]

[frigid nova]

I would use the first snippet. Telling your users that the method raises an exception group is IMO less informative than telling them what exact exceptions are raised. Happy to get feature requests in Griffe/mkdocstrings-python if you have ideas on how to better support exception groups ^^

[grave fjord]

You could use TypeAliases

[jade viper]

!e The issue is that error catching changes when you use ExceptionGroups:

def some_method():
    ex1 = ValueError("value_error")
    ex2 = TypeError("type_error")

    raise ExceptionGroup("exception_group", [ex1, ex2])

try:
    some_method()
except (ValueError, TypeError):
    print("Hi from normal except")
except Exception:
    print("Uncaught :(")

try:
    some_method()
except ExceptionGroup:
    print("Hi from normal except")
except Exception:
    print("Uncaught :(")

try:
    some_method()
except* (ValueError, TypeError):
    print("Hi from star except")

So I wanted to find a way to indicate that you should either catch ExceptionGroup or use except*

[rough sluice]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

001 | Uncaught :(
002 | Hi from normal except
003 | Hi from star except

[frigid nova]

Right, thanks 🤔 then yeah snippet 2 makes more sense. And you'd document ValueError and TypeError in the description of ExceptionGroup

[jade viper]

FYI I opened an issue in the Google styleguide repo since it's the one I use: https://github.com/google/styleguide/issues/907
Love your lib btw 🙂

[rare scarab]

Does sphinx or numpy have a convention for it?

[jade viper]

None that I could find

[rare scarab]

What if you specify *ValueError: Message?

[jade viper]

Hmmm that's a good one actually

Edited the issue to add that as a suggestion, thanks!

[uncut plover]

I just started using jsonpath-ng, and MyPy complains that the module is installed but missing library stubs or py-typed marker. I think because jsonpath-ng is old and stable it hasn’t been updated with the new type hinting syntax, and thus doesn’t export any types. I know I can silence the warnings with # type: ignore. But in general, how does one deal with this issue other than ignoring the warnings?

[grave fjord]

You can add your own stubs

You can event publish them on PyPI

It looks like they do have types but they forgot to export them

[grave fjord]

https://github.com/h2non/jsonpath-ng/issues/192

[icy obsidian]

What would be a better way of "forwarding" the signature of a method?

class A(ABC):
    def func(self, *args, **kwargs) -> int:
       return self.a_func(*args, **kwargs)
    
    @abstractmethod
    def a_func(self, *args, **kwargs) -> int:
        ...
    
class B(A):    
    def a_func(self, i: int) -> int:
        return i
    
b = B()
b.func()  # <<<<
b.a_func(5)

I would like to see b.func to have the same signature as the implemented a_func.
I can explicitly define func in B with the needed signature, or I can make A generic. But that would cause a lot of copying.
Are there any better ways?

[green bluff]

functools.wraps

[trim tangle]

that doesn't help with type information, it's only for copying runtime metadata

[trim tangle]

You can parametrize it with a typevartuple, but that can be verbose (and doesn't support kwargs of course)

class A[*Args](ABC):
    def derived_method(self, *args: *Args) -> int:
        rv = self.template_method(*args)
        if rv < 0 or rv >= 2**64:
            raise OverflowError("number too big", rv)
        return rv

    @abstractmethod
    def template_method(self, *args: *Args) -> int:
        raise NotImplementedError

[simple field]

are there any fleshed out resources/documentation for implementing mypy plugins? the official docs on plugins (https://mypy.readthedocs.io/en/stable/extending_mypy.html) dont really explain much

[trim tangle]

I made a simple mypy plugin out of curiosity a few years ago: https://github.com/decorator-factory/mypy-plugin-attempt
I had to read other mypy plugins and the mypy source code to understand how to do stuff

[green bluff]

TypedDict supports kwargs however

[trim tangle]

But you can't do ```py
class A[D: TypedDict]:
def init(self, **kwargs: Unpack[D]):

[green bluff]

but that doesn't sound like what they're after

if you have a known unpack, then you don't need the generic

[simple field]

cheers ill take a look

[trim tangle]

Can you provide a more concrete example? Do you have several classes deriving from A?

[green bluff]

(also, I kind of feel like *args and **kwds are often an anti-pattern and not what people want in their code)

[trim tangle]

That's probably not something you should use as a reference, as that's literally my first (and last) attempt at making a plugin
I would recommend looking into real mypy plugins, e.g. the ones in numpy or sqlalchemy

[icy obsidian]

Yea, this was one of my original soolutions - use ParamSpec and explicitly specify those either as a generic parameter of a class or pass a function into Init to infer the type.

But just like you said it causes a lot of copying...

[elder flume]

Dunno

[icy obsidian]

The thing that I needed was to have a base class responsible for some base logic and additional specialised logic in derived classes (there will be multiple of those).
Consider it some kind of a collection:

• base class has logging and notification logic that is both called before and after the derived logic
• derived class is responsible for the storage mainly (and they need the ability to specialise the signature that will both affect the func and the callbacks)

I can reverse the classes, but I will have to explicitly call the "base" logic then, which is not desirable.

[simple field]

all good, more examples helpful either way. and thanks ill take a look at those too.

[cosmic plinth]

when can I stop doing from __future__ import annotations? is that the default in 3.13?

[oblique urchin]

The behavior of from __future__ import annotations will not become the default, but 3.14 will make annotations lazy (via PEP 649)

So you won't need it in 3.14+

[rare scarab]

And python has a 5 year support period for each version, so you can expect 3.13 to be dropped in 2029

[cosmic plinth]

what's the difference between lazy annotations and the default? in my mind that's what it does

[oblique urchin]

the future import turns annotations into strings

[cosmic plinth]

ohhh okay thank you, I'll note this in our internal tracker to enforce from __future__ import annotations until 3.14

awesome to read about the performance, great job everybody! https://peps.python.org/pep-0649/#performance-comparison

[jade viper]

Have any of you guys used this project? Is it worth it? https://github.com/dry-python/classes

[icy obsidian]

I guess I managed to get a solution.

class Proto[**P](Protocol):
    def a_func(self, *args: P.args, **kwargs: P.kwargs) -> int: ...

class A[**P](ABC):
    def __new__[**V](cls: type[Proto[V]]) -> A[V]: ...

    def func(self, *args: P.args, **kwargs: P.kwargs) -> int:
       return self.a_func(*args, **kwargs)
    
    @abstractmethod
    def a_func(self, *args: P.args, **kwargs: P.kwargs) -> int: ...
    
class B(A):
    def __init__(self) -> None:
        super().__init__()
    
    def a_func(self, i: int, s: str) -> int:
        return i
    
b = B()
b.func(5, "asd")  # Good - resolved as [int, str] -> int
b.func(1)  # Bad

This lets me "forward" the signature to the base class

[icy obsidian]

However, how do I correctly type that new returns its own class?

    def __new__[**V](cls: type[Proto[V]]) -> A[V]: ...   # <<< A is not defined

Edit: Somehow I forgot that we can use quotations here 'A[V]'

[green bluff]

why do quotes matter here? I'd suggest using typing.Self in an inheritance situation like this

[icy obsidian]

That was the first thing Itried

Though ""Self" cannot be used in a function with a self or cls parameter that has a type annotation other than "Self""

[green bluff]

can't it?

I thought that was its whole point

oh that's confusing, why is your cls typed as something else? that's not how that works

a static method (special-cased so you need not declare it as such) that takes the class of which an instance was requested as its first argument.
https://docs.python.org/3/reference/datamodel.html#object.__new__

[icy obsidian]

I did that to get the Protocol generic out of there

[green bluff]

you shouldn't need to annotate it at all, it's implicit just like self is

[icy obsidian]

Yes, but A has no type information at that point without this

Or am I missing something?

[green bluff]

I don't think making __new__ generic makes any sense

[icy obsidian]

Removing the __new___ makes so that Type of "b.func" is "(...) -> int"

[green bluff]

well yeah

you forgot to pass any generics to your B declaration

[icy obsidian]

That was the idea - B is not generic

[green bluff]

correct, but A is

[icy obsidian]

A is and I wanted to get rid of a lot of explicit type specifications

[green bluff]

yeah, so just define what your generics are in the (A[...]):

[icy obsidian]

There will be like 5 lines of specifications that will effectively only copy what a_func says.

And this was one of my original ideas, which I didn't "like" just because it will take a lot of copying...

[green bluff]

I'm still not entirely sure what you're trying to accomplish here

[icy obsidian]

I tried to summarise the idea before ( #type-hinting message ), but not sure if it did the thing...

[green bluff]

yeah, I read that and I'm not getting it. can you give concrete examples, no generics, and how you imagine it will work?

it sounds like you're just trying to build overloads

varargs aren't really expected to work with different types, they should all be the same type

[icy obsidian]

Well... overloads with extra steps....

class StorageBase:
    def some_statistics(self, value: float): ...

    def some_callbacks(self, value: float, *args, **kwargs): ...

    def add(self, value: float, *args, **kwargs) -> None:
        self.some_statistics(value)

        self._storage_add(value, *args, **kwargs)

        self.some_callbacks(value, *args, **kwargs)

class ListStorage(StorageBase):
    def __init__(self) -> None:
        self._storage: list[float] = list()

    def _storage_add(self, value: float):
        self._storage.append(value)

class DictStorage(StorageBase):
    def __init__(self) -> None:
        self._storage: dict[str, float] = dict()

    def _storage_add(self, value: float, key: str):
        self._storage[key] = value

Most likely I am doing in a completely "wrong" way...

[green bluff]

is value expected to have a type?

[icy obsidian]

Yes

Value has some base type

But additional arguments are only defined in Children

So I wanted to propagate the types to the base class to be able to check the callbacks that will be registered

(and to type add in the correct way)

[green bluff]

I'm about to eat dinner, I'll look at your concrete example and tag ya later

[icy obsidian]

Sure, thanks for the help anyway!~

[jolly cipher]

not pep 749? :D

[obsidian oar]

use gettattr() and setattr() for access and modify the attribute

_PRIV = "_example"

class Example:
  def __init__(self):
    self.normal = 5
    setattr(self, _PRIV, 10)

  @property
  def example(self):
    return getattr(self, _PRIV)

[obsidian oar]

Will doing this be ok for type distribution of 3.5+ inline?

from __future__import annotations

def func(arg: object | None) -> str | None:
    return None if arg is None else str(arg)

[grave fjord]

This slows down the process though

[obsidian oar]

It does not produce errors at runtime, but may have problems with type analyzers at runtime.

[grave fjord]

What tool is this, I've never seen one show __ attributes

[grave fjord]

What's that?

[obsidian oar]

If you just want to hide the linter, it is not necessary to make it private, you just need to make it not readable with TYPE_CHECKING == True

[obsidian oar]

annotations for type checkers and linters

[grave fjord]

What's type distribution and what's 3.5+ inline?

[obsidian oar]

https://typing.python.org/en/latest/spec/distributing.html

[grave fjord]

And 3.5+ inline?

[grave fjord]

Stub files don't need future annotations

[obsidian oar]

3.5 online

union | syntax, with future annotations

[grave fjord]

You're not making any sense

[trim tangle]

inline means it's not in a stub, but in the code directly

[grave fjord]

• type distribution or stubs don't need from __future__ import annotations
• stubs are the opposite of inline
• Python 3.5 doesn't support from __future__ import annotations you need 3.7+

[obsidian oar]

oky

Therefore, when distributing a package with annotations starting with version 3.7 and with from __future__ import annotations, there will be no problems with the | union syntax in the annotations.

?

[grave fjord]

Unless you use pydantic

[obsidian oar]

only for type distribution of classes, functions & protocols

[grave fjord]

Or similar runtime annotations tools.
Runtime annotations won't work properly under future annotations

[obsidian oar]

Mechanisms like typing.get_type_hints don't work well if the version they're running doesn't support | syntax, but I don't know if linters can resolve them.

mypy, pyright, pylint, ...

[warm topaz]

https://github.com/typeddjango/django-stubs/pull/2590 is point no. 3 possible to do here like somehow add | None to the type based on the Field() null= param

I think it might not be possible out of the box without the plugin installed, and i would probably have to fiddle with the transformers https://github.com/typeddjango/django-stubs/blob/master/mypy_django_plugin/transformers/fields.py

[warm topaz]

yep had to mess with the set descriptor type method for it, but lmk if there is a method to do it without plugin stuff just with python typing

[warm topaz]

Hi!
I was planning to apply to gsoc this year under django organisation and have draft proposal ready - https://gist.github.com/FallenDeity/109bbe3eec1ab27bee665a4b922257c6
Would you folks mind giving it a look once, any suggestions or feedback is appreciated

Thank you

[polar aurora]

Hmm. I've got a function with this signature:
async def receive_packets(self) -> AsyncIterator[ServerPacket]:
..and I just can't get mypy to be happy with it.

I guess because async functions actually return coroutines that then get awaited and THAT is what returns the AsyncIterator? What's the right way to annotate this other than with an ignore comment?

    async def receive_packets(self) -> AsyncIterator[ServerPacket]:  # type: ignore[override, misc]
        """
        Receive packets from the server.

        Returns:
            An async iterator that yields received packets
        """
        while self._connected:
            try:
                packet = await self._packet_queue.get()
                yield packet
                self._packet_queue.task_done()
            except asyncio.CancelledError:
                break
            except Exception as e:
                logger.error(f"Error in packet iterator: {e}")
                break

[trim tangle]

What does mypy say?

This works fine for me ```py
from collections.abc import AsyncIterator

async def foo() -> AsyncIterator[int]:
yield 1
yield 2
yield 3

what version of mypy do you have?

[polar aurora]

Hmm. 1.15.0

[trim tangle]

Do you perhaps have a base class like this? py class Base(ABC): @abstractmethod async def receive_packets(self) -> AsyncIterator[ServerPacket]: raise NotImplementedError Because non-generator async function wrap the return type in Coroutine, this would be equivalent to: ```py
class Base(ABC):
@abstractmethod
def receive_packets(self) -> Coroutine[Any, Any, AsyncIterator[ServerPacket]]:
raise NotImplementedError

So you probably want this: ```py
class Base(ABC):
@abstractmethod
def receive_packets(self) -> AsyncIterator[ServerPacket]:
raise NotImplementedError

[polar aurora]

% mypy sqemu/infrastructure/protocol/slimproto_client.py 
sqemu/infrastructure/protocol/slimproto_client.py:138: error: Return type "AsyncIterator[ServerPacket]" of "receive_packets" incompatible with return type "Coroutine[Any, Any, AsyncIterator[ServerPacket]]" in supertype "ProtocolClientInterface"  [override]
sqemu/infrastructure/protocol/slimproto_client.py:138: note: Consider declaring "receive_packets" in supertype "ProtocolClientInterface" without "async"
sqemu/infrastructure/protocol/slimproto_client.py:138: note: See https://mypy.readthedocs.io/en/stable/more_types.html#asynchronous-iterators
Found 1 error in 1 file (checked 1 source file)

[trim tangle]

hey, my crystal ball is working 🙂

[polar aurora]

Yep class ProtocolClientInterface(abc.ABC):

with @abc.abstractmethod on the function def

[trim tangle]

Did you read the link that mypy suggested?

[polar aurora]

I thought I did, let me do it again

[trim tangle]

well, it explains this basically

[polar aurora]

Oh duh yeah, I didn't get the bottom-most portion I guess when I skimmed it last night

Thanks! haha

[trim tangle]

I hate this feature tbh

Generator doesn't work like this, AsyncGenerator doesn't work like this, only Coroutine works like this

And it makes the YieldType and SendType parameters of Coroutine absolutely useless (because in async functions, they're always Any)

[polar aurora]

Oh that's nasty

[trim tangle]

Well, it probably doesn't come up that often in applications

But you could use those parameters, for example:

• to mark a specific function as only working with trio/asyncio/anyio
• to safely use async/await syntax for something other than concurrency on I/O, like combinator parsers

I imagine it could also help developers of event loop libraries who use type annotations internally, though that's probably a sample size of 1

[sharp walrus]

Ignoring the fact that one should instead write this (as this is for education)

def foo(arg1: int, arg2: str, arg3: int, *args: int): ...

How can one hint that all remaining args should be int?

def foo(*args: *tuple[int, str, int, ...]): ...

As I understand it, ... above means that all remaining are Any, so a call like foo(1, "a", 2, "b") should be incorrect but isn't.

even though in pep0484 it says *args: int is deduced as tuple[int, ...]

edit:
like so it seems:

def foo(*args: *tuple[int, str, int, *tuple[int, ...]]):

[trim tangle]

yeah, the last codeblock

I don't think tuple[int, str, int, ...] is valid at all

[sharp walrus]

I think it is, as mypy seems to accept it?

[trim tangle]

def f(x: tuple[int, str, int, ...]) -> None:
    reveal_type(x)
``` ```
main.py:1: error: Unexpected "..."  [misc]
main.py:2: note: Revealed type is "tuple[builtins.int, builtins.str, builtins.int, Any]"

[sharp walrus]

ah thanks, I also just saw that error 😅

I can't seem to find any specific docs on how ... works with typing. It seems it works in different ways based on context

[trim tangle]

It doesn't have a unified meaning

It's acceptable in tuple[T, ...] and as an omitted default value in e.g. ```py
class Listener(Protocol):
def update(self, message: str = ...) -> None:
...

oh, also Callable[..., T] to mean "any arguments are acceptable"

[sharp walrus]

but how is that different from Callable[Any, T]

[trim tangle]

that's invalid

Callable[[Any], T] would mean a callable accepting exactly one positional argument, which is Any

Callable[..., T] means you can call it with any combination of positional and keyword arguments

[sharp walrus]

ah I see

thanks!

[polar aurora]

So, Matiss asked this question, and it's got me curious about the best approach. #python-discussion message

I can think of an easy 'Protocol' option that requires listing out the permitted attributes on the protocol

but now I'm trying to cook up a way that avoids that duplication of attribute names

How crazy is this? https://paste.pythondiscord.com/XRCQ

[hoary pagoda]

does it work?

[junior vessel]

Well, it doesn't for me (at least pylance does not care about any of that)

it seems a bit too runtimey to work with static type checkers

[polar aurora]

Aah I guess it doesn't work yet my bad, I get strict_attributes.py:22: error: Cannot assign to a method [method-assign]

Is there something like this that WOULD work? This is my first time trying to use __annotations__ directly

[restive rapids]

static typecheckers dont treat __annotations__ specially, for them its just an attribute, and it isnt used for typechecking
they just see [T](type[T]) -> type[T], so the decorator is basically a no-op, type-wise that is

[polar aurora]

Oh, whoops. I guess this whole approach is a bust then. Bummer.

I guess maybe generate a stub file automatically with all the known attributes?

I guess you could also codegen the Protocol definition so it always had the right attribute list

[junior vessel]

again, the Protocol is not suitable because it would require explicit type narrowing
and both of those approaches require an additional step to make use of them and this is not an installable package

I can live with the Any too, it's not thaaaat big of a deal, but it would be nice if I could do something about it

and yeah, if this were an installable package, I might just generate stubs for it, that'd probably work pretty well, but in here it's not the case

[junior vessel]

nvm, it do be kinda annoying when I'm trying to rename stuff and now I can't tell if I have renamed stuff properly or not 😬 (should've used the refactor tool for it ig, but still)

[jade viper]

Should I describe a class in the class docstring or the __init__ one?

Or should I describe it in the class and just have the init args for it's docstring?

[frigid jolt]

class docstring

and include there the arguments to build the object too

in a Parameters section

[jade viper]

Should the in-depth description of the oscillation rate ("If the oscillation rate is negative[...]") go in the method body or in the args section where I talk about the rate?

def smooth_value(x: float, starting_value: int = 100, oscillation_rate: float = -0.025) -> float:
    """
    A smoothing function that transforms the input value using an exponential function.

    The function is defined as:

    .. math::
        f(x) = a * e^{bx}

    If the oscillation rate is negative, the function will converge to 0 (the smaller the value, the faster the function will converge). If positive, it will diverge to ∞ (the larger the value, the faster the function will diverge).

    Where:
        - :math:`f(x)` is the transformed value.
        - :math:`a` is the starting value.
        - :math:`b` is the convergence rate.
        - :math:`x` is the input value.

    See:
        - https://www.geogebra.org/graphing/znwrsqgv
        - https://www.mathpapa.com/algebra-calculator.html?q=a%5Ccdot%20e%5E%7Bbx%7D

    Args:
        x: The input value.
        starting_value (optional): The function's starting value.
        oscillation_rate (optional): The rate of convergence.

    Returns:
        The transformed value.
    """

[feral wharf]

Does anyone do it on the init lol

[jade viper]

I used to until Ruff complained lol

[feral wharf]

Oh lol

[feral wharf]

Should be fine in the body

[jade viper]

I was just going to say I found this in the Google official styleguide: "If the description is too long to fit on a single 80-character line, use a hanging indent of 2 or 4 spaces more than the parameter name (be consistent with the rest of the docstrings in the file)."

[icy obsidian]

Is it possible to make a bound generic parameter?

class QueueBase[T, **P=[]](ABC):
    def wrong(self) -> NoReturn:
        raise OverflowError(self)  # <<<<<<<<<<< Argument of type cannot be assigned

class OverflowError[QueueType: QueueBase](Exception):
    def __init__(self, queue: QueueType):
        self.queue = queue

Providing a default empty ParamSpec breaks this.

[frigid jolt]

this is your choice, imo it's better to have it like you did

[green bluff]

no, I don't think it's possible to provide generic defaults today like that

[icy obsidian]

So for now the only option is to make the Error just the most generic type without bounds?

[iron vapor]

Is TypedDict the right way to type a dict with a certain list of possible keys? I don't mind doing it, but then it creates typing conflicts with inputs that actual dicts that have those keys. Is there a better way, or am I hitting an edge of Python typing?

[trim tangle]

Are all the values of the same type?

[iron vapor]

No

This is what I did right now:

PurchaseDict = TypedDict(
    "PurchaseDict",
    {
        "item_id": int,
        "receipt_id": int,
        "price": float,
        "notes": NotRequired[str],
    },
)

[trim tangle]

Maybe you want a dataclass?

Why do you need a dict specifically?

[iron vapor]

Honestly, this was just to keep typing consistent. This is a FastAPI app, so the data is coming in validated by pydantic, and I wasn't sure if swapping it to a dataclass made sense at that point.

Maybe just leaving it as a dict is best

[trim tangle]

Can you show more code maybe? Maybe you want a pydantic model?

[iron vapor]

All I'm getting out of the typed dict is auto complete

[trim tangle]

Is this for the input/output of a request handler?

[iron vapor]

No

Let me grab the endpoint code

@purchases.post(
    "/bulk",
    response_model=schemas.NewPurchaseBulkResponse,
)
async def add_purchase_bulk(
    purchase_input: schemas.NewPurchaseBulkInput, db_session: DBSessionDependency
):
    purchase_repository = PurchaseRepository(db_session)

    purchases_data: list[dict] = []
    for purchase in purchase_input.purchases:
        purchases_data.append(
            Purchase(
                **dict_from_schema(purchase, schemas.NewPurchaseBulkPurchaseInput),
                receipt_id=purchase_input.receipt_id
            ).__dict__
        )

    purchases = await purchase_repository.bulk_create(purchases_data)
    return {
        "data": {
            "purchases": [dict_from_schema(p, schemas.Purchase) for p in purchases]
        }
    }

in bulk_create

I have the parameter typed as PurchaseDict

That middle block I'm also not very happy about and have asked the associated question in databases

All it's really doing is creating a dict from a SQLA model (with hybrid properties)

[grave fjord]

Why not just a pydantic model?

What type is NewPurchaseBulkResponse? (And why not just a return type annotation)

[iron vapor]

Create a separate model representing said data, and pass it as a model, instead of a dict?

class NewPurchaseBulkPurchaseInput(BaseModel):
    item_id: int
    price: float
    notes: str | None = None


class NewPurchaseBulkInput(BaseModel):
    receipt_id: int
    purchases: list[NewPurchaseBulkPurchaseInput]

[trim tangle]

If you specify a return model, it will show up in the OpenAPI schema (and the Swagger UI)

[iron vapor]

And I'm fully open to other ways of doing it... this really is just me trying to figure stuff out as I go

I was reading up on return type vs response_model, and it felt like response_model was the way to go

[trim tangle]

oh, I missed that

[iron vapor]

They have a section in the fastapi docs on that very topic

[grave fjord]

response_model isn't needed

[iron vapor]

I know it's not needed if I put a return type

https://fastapi.tiangolo.com/tutorial/response-model/#response_model-or-return-type

I just add one to each route, after having read those docs

Again, I'm sure there are a lot of points I'm missing

And in fact, when the code is done, I plan on making it public and asking people for feedback and improvements so I can learn

So I should store the data into a new pydantic model and pass that along? Then parse the dict back out? Feels like an extra step, but it does maintain data integrity as well as perhaps solving the SQLA issue in #databases

[grave fjord]

You don't need to parse the dict back out

response model should be a pydantic model

Or your return type

So just make it BaseModel/Sequence all the way down

[iron vapor]

My input and output are pydantic models

But this is data passed to a function in my endpoint

[vale gull]

I have a decorator that accepts two arguments, functions for processing the inputs and outputs of the decorated function. The process_inputs function has the same input signature as the decorated function. The process_outputs function takes as input the output of the decorated function. With the snippet below, users of the decorator have the nice experience that mypy will complain if they define process_inputs or process_outputs in a way that violates these constraints.

The challenge is the decorator also works for async functions. I have tried various attempts at overloads and protocols to no avail. Any help appreciated!

from typing_extensions import ParamSpec

ParametersType = ParamSpec("ParametersType")
ReturnType = TypeVar("ReturnType")


def decorator(
    process_inputs: Callable[ParametersType, Dict[str, Any]],
    process_outputs: Callable[[ReturnType], Dict[str, Any]],
) -> Callable[[Callable[ParametersType, ReturnType]], Callable[ParametersType, ReturnType]]:
    """
    Imagine there is logic in this decorator to record inputs and outputs.
    """
    raise NotImplementedError


def process_inputs(a: int, b: int) -> Dict[str, Any]:
    raise NotImplementedError


def process_outputs(result: int) -> Dict[str, Any]:
    raise NotImplementedError


@decorator(process_inputs, process_outputs)
def decorated_function(a: int, b: int) -> int:
    raise NotImplementedError


decorated_function(2, 1)

[polar aurora]

Hmm. Are you open to having two different decorator functions, one for sync and one for async? That might make it a lot easier to conquer.

[vale gull]

Need it to be one.

[polar aurora]

I guess you could make a union type of the Callable signature you have, plus the one you'd get for an async version, and make that be the return type?

Oh I guess this is also what @overload can be used for?

make a version that returns Callable[[Callable[ParametersType, Awaitable[ReturnType]]], Callable[ParametersType, Awaitable[ReturnType]]] alongside the current one, and make them both @overload? Untested, just thinking aloud

You could use inspect.iscoroutinefunction() to decide which implementation to install?

[vale gull]

I had the same thought. mypy seems to struggle with the union return type. One sec, I'll show you.

[polar aurora]

I guess for sync functions you'd want something like result = cast(Callable[ParametersType, ReturnType], func)(*args, **kwargs)... but for async it would be uhhh

Like this I guess? result = await cast(Callable[ParametersType, Awaitable[ReturnType]], func)(*args, **kwargs), I guess that's all it needs

[vale gull]

Sounds like you have in mind using these casts inside the decorator itself?

[vale gull]

Example included. This results in the following error from mypy:

tmp.py:50: error: Argument 1 has incompatible type "Callable[[int, int], int]"; expected "Callable[[int, int], Coroutine[int, Any, Any]]"  [arg-type]
tmp.py:55: error: Value of type "Coroutine[int, Any, Any]" must be used  [unused-coroutine]
tmp.py:55: note: Are you missing an await?
Found 2 errors in 1 file (checked 1 source file)

It seems like mypy matches the type overload based on the input signature, not the return type, so it thinks every decorated function is a coroutine.

[rough sluice]

Click here to see this code in our pastebin.

[polar aurora]

Yeah I was picturing doing it in a 'wrapper' function that you have two definitions of, one for sync, one for async, and you use 'inspect' to decide at runtime which to wrap with

Oh that's an interesting mypy error too.. not the one I expected

[vale gull]

That how I have implemented inside the decorator. The problem still remains of the user-facing types on the boundary of the decorator.

Tricky

[polar aurora]

@vale gull OK this took longer than I expected, but what about this pattern? Seems to work locally https://paste.pythondiscord.com/QWCQ

Damn, no, mypy is still unhappy with this.. maybe I can fix it

Oh maybe it's just my lame lambdas as process_ functions

[vale gull]

I am seeing what I think is a similar issue on a slightly modified version of your code. Seems like mypy matches on the first overload, which in this case, causes it to think the coroutine function is a sync function.

tmp.py:41: error: Overloaded function implementation cannot produce return type of signature 1  [misc]
tmp.py:41: error: Overloaded function implementation cannot produce return type of signature 2  [misc]
tmp.py:119: error: Argument 1 has incompatible type "Callable[[str], Coroutine[Any, Any, str]]"; expected "Callable[[str], str]"  [arg-type]
tmp.py:126: error: Argument 1 to "run" has incompatible type "str"; expected "Coroutine[Any, Any, Never]"  [arg-type]
Found 4 errors in 1 file (checked 1 source file)

[rough sluice]

Click here to see this code in our pastebin.

[polar aurora]

Yeah this is really not what I expected mypy to do with this code. Hmm.

[obsidian oar]

class ProtocolClientInterface:
    async def receive_packets(self) -> AsyncIterator[ServerPacket]:
        raise NotImplementedError

[vale gull]

Thanks for giving it a shot! Appreciate.

[polar aurora]

I learned something at least.. thanks for the interesting problem

[polar aurora]

So, Dagger has this function he's been working with, that has the signature def split(a: Optional[Signal], b: Optional[Signal]) -> Tuple[Optional[Signal], Optional[Signal]]:

I just had a devil of a time getting mypy happy with it. His implementation was this:

class Splitter:
    @staticmethod
    def split(a: Optional[Signal], b: Optional[Signal]) -> Tuple[Optional[Signal], Optional[Signal]]:
        """
        a - Input signal A
        b - Input signal B
        """
        
        if a is None and b is None: 
            return None, None

        if b is None: 
            return a/2, a/2
            
        if a is None: 
            return b/2, b/2

        return (a + b)/2, (a + b)/2

The best thing I've got so far that passes mypy is this, which I don't exactly love:

def is_signal_not_none(s: Optional[Signal]) -> TypeGuard[Signal]:
    """Help mypy understand when a value is not None."""
    return s is not None

class Splitter:
    @staticmethod
    def split(a: Optional[Signal], b: Optional[Signal]) -> Tuple[Optional[Signal], Optional[Signal]]:
        """
        a - Input signal A
        b - Input signal B
        """
        
        if a is None and b is None:
            return None, None
            
        if b is None and is_signal_not_none(a):
            result = a / 2
            return result, result
            
        if a is None and is_signal_not_none(b):
            result = b / 2
            return result, result
            
        if is_signal_not_none(a) and is_signal_not_none(b):
            result = (a + b) / 2
            return result, result
            
        # This should never happen, but needed to satisfy mypy
        return None, None

I tried like 6 things and this is all I could figure out that actually worked

Is there a less-crazy way?

[pastel egret]

x is not None should work fine? For the last line I’d suggest raise AssertionError(…). Or change your ifs to be nested such that all branches are explicit:

if a is None:
    if b is None:
        …
    else:
        …
else:
    if b is None:
        …
    else:
        …

[obsidian oar]

Does anyone know why type variables are not allowed in ClassVar?

class A[T]:
    a: ClassVar[T]

[soft matrix]

theyre ambiguous because you dont know what they refer to by default

ie what is A.a?

how can A[str].a be different from A[int].a

[obsidian oar]

subclasses of int
example

from typing import ClassVar


class Example[T: int]:
    a: ClassVar[T]  # type: ignore

    def __init__(self) -> None:
        pass

    @classmethod
    def set(cls, a: T) -> None:
        cls.a = a


class MyInt(int):
    pass


e = Example[int]()
e.set(MyInt(5))
e.a

[polar aurora]

I feel like I tried this and mypy still got angry; I'll re-check, thanks for looking!

[oblique urchin]

That doesn't solve the problem. There's only one place to put Example.a, so the same value will be shared by all specializations of the class

[obsidian oar]

Yes, but the idea is that a is of the type specified to T, they are good for dynamic classes

[oblique urchin]

That's not how your code works, unless I'm missing something

[obsidian oar]

from typing import ClassVar, Protocol


class DType:
    pass


class DynamicClass[DTypeT: DType](Protocol):
    # config attr (final)
    dtype: ClassVar[DTypeT]  # type: ignore

    ...


def create_class[DTypeT: DType](dtype: DTypeT) -> type[DynamicClass[DTypeT]]:
    raise NotImplementedError

[oblique urchin]

I see, it does make more sense in a base class where you make specialized child classes

[obsidian oar]

Also, if they were valid, a kind of dataclass could be made but for metaclasses

[paper bloom]

Hi all. Is there any advice/consensus on copying the types on a method from superclasses when overriding? If the types are not copied over are type checkers aware that the overriding method has the same types as the parent?

[obsidian oar]

from collections.abc import Callable
from typing import ClassVar, Protocol
from pydantic import BaseModel


class Request:
    def __init__(self, data: dict[str, object]):
        self.data = data


class DynamicRoute[ModelT: BaseModel](Protocol):
    path: ClassVar[str]
    auth: ClassVar[bool]
    encrypt: ClassVar[bool]
    verify: ClassVar[bool]
    model: ClassVar[type[ModelT]]  # type: ignore
    request: Request

    @staticmethod
    def handler(model: ModelT, /) -> object:
        raise NotImplementedError

    def __init__(self, request: Request) -> None: ...

    def get_response(self) -> object:
        raise NotImplementedError


def route_init(self: DynamicRoute[BaseModel], request: Request) -> None:
    self.request = request


def get_response(self: DynamicRoute[BaseModel]) -> object:
    return self.handler(self.model(**self.request.data))


class RouteMeta(type):
    path: str
    auth: bool
    encrypt: bool
    verify: bool
    model: type[BaseModel]


def create_route[ModelT: BaseModel](
    name: str,
    path: str,
    auth: bool,
    encrypt: bool,
    verify: bool,
    model: type[ModelT],
) -> Callable[[Callable[[ModelT], object]], type[DynamicRoute[ModelT]]]:
    def inner(handler: Callable[[ModelT], object]) -> type[DynamicRoute[ModelT]]:
        return type.__new__(  # type: ignore
            RouteMeta,
            name,
            (),
            {
                "path": path,
                "auth": auth,
                "encrypt": encrypt,
                "verify": verify,
                "model": model,
                "handler": staticmethod(handler),
                "__init__": route_init,
                "get_response": get_response,
            },
        )

    return inner


class Home(BaseModel):
    param: str


@create_route("home", "/", False, False, False, Home)
def home(request: Home):
    return "<home-response>"


print(home.auth)
print(home.get_response)
print(home(Request({"param": "a"})).get_response())

[paper bloom]

I found some details here: https://microsoft.github.io/pyright/#/type-inference?id=parameter-type-inference

[obsidian oar]

As a standard, the SOLID principle is expected to be applied in OOP, Linters should warn you if your subclasses don't comply with this.
https://www.digitalocean.com/community/conceptual-articles/s-o-l-i-d-the-first-five-principles-of-object-oriented-design

[pastel egret]

That's not really the problem, all checkers verify Liskov Substitution. But mypy doesn't copy types across at all, you'll need to repeat itself. Definitely annoying, though one small advantage is that this ensures mypy will catch incompatibilities if you modify the superclass definition at some future time.

[slender basin]

Hi, new here. 👋🏻
I'm adding type hints for an existing large codebase, and am faced with a situation like:

from abc import ABC

class C: ...

class Base(ABC): ...

class Child1(Base):
    my_awesome_field: C

class Child2(Base):
    @property
    def my_awesome_field(self) -> C: ...    

i.e., for some classes in a hierarchy my_awesome_field is implemented as an attribute and for some as a property. I'm looking for some way to inform functions getting a Base they can safely use my_awesome_field:

def func(b: Base):
    maf = b.my_awesome_field
    ...

AND hopefully respect existing initializers in the code like c1 = Child1(my_awesome_field=<whatever>) .

If I specify a protocol like

class HasAwesomeField(Protocol):
    @property
    def my_awesome_field(self) -> C: ...

class Base(ABC, HasAwesomeField): ...

implementations of Base are restricted to properties. If I change Child1s implementation to a property:

class Child1(Base):
    _my_awesome_field: C

    @property
    def my_awesome_field(self) -> C:
        return _my_awesome_field

constructs like Child1(my_awesome_field=<whatever>) break.

Is what I'm looking for possible?

[polar aurora]

I think the answer is "No", because you'd need a Protocol where you could both directly assign to the thing in __init__ and via the 'descriptor' interface that "hello, I'm a property" indicates?
Maybe somebody has a better take.

I don't consider myself an expert, that's just my mental model right now.

[restive rapids]

is it bad that the answer is "use mypy because pyright maintainer decided that attributes dont fullfill a property protocol"
https://github.com/microsoft/pyright/issues/2072

[slender basin]

I really agree with mypy maintainers' comment here https://github.com/python/mypy/issues/10797#issuecomment-878094230:

Since properties are used to emulate attributes, it would certainly be very annoying if MyPy where to separate fields and properties as separate kinds of attributes.

It didn't occur to me this could be pyright-specific behaviour... Thanks @restive rapids !

[soft matrix]

I mean I completely agree with the take, someone needs to implement a read only special form

[polar aurora]

Is this different in basedpyright?

[obsidian oar]

You can create a protocol for your base class, but you must know the properties of its attributes (getter, setter). Properties are just getters, and those defined by annotations implicitly have getters, setters, and deleters, so your implementation depends on how you have your properties.

from typing import Protocol

# if your properies only implements getter
class Proto(Protocol):
    @property
    def a(self) -> int: ...


def check(cls: type[Proto]) -> None: ...


class A:
    a: int


class B:
    @property
    def a(self) -> int:
        raise NotImplementedError


check(A)
check(B)

A.a implements getter, setter and deleter
B.a implements only getter

[slender basin]

Not sure how to use that. Can I somehow use this Proto to mark arguments as 'having a readablea, property or other'?

[restive rapids]

https://github.com/microsoft/pyright/releases/tag/1.1.178

Behavior Change: Changed the interpretation of a property declared within a protocol class. It was previously interpreted only as a property (i.e. classes compatible with the protocol must implement a property of the same name). Compatible classes are now able to declare an attribute whose type is compatible with the property getter. Access to the property from the protocol class as a class variable is no longer allowed.
average eric traut
"by design"
changes the design in 3 months

sadly this seems to not like, transitively apply with an ABC inheriting from the protocol

from typing import Protocol
from abc import ABC

class Proto(Protocol):
  @property
  def field(self) -> int:
    ...

class Base(ABC, Proto):
  ...

class Child(Base):
  field: int

[obsidian oar]

Yes, but it must have the common attributes of your subclasses, including setter, deleter

[obsidian oar]

Protocols are not intended for inheritance of execution objects, but rather as a sort of interface to other languages.

[restive rapids]

can you try to remember the original issue we're solving and then also remember that we dont have intersection types
and understand that your proposed solution does not work

[obsidian oar]

from typing import Protocol
from abc import ABC


class BaseProto(Protocol):
    @property
    def field(self) -> int: ...


class Base(ABC):
    field: int


class MyClass(Base):
    pass


def check(cls: type[BaseProto]) -> None: ...


check(Base)
check(MyClass)

[restive rapids]

thats missing the "some classes implement it as a property" (you cant do class Child(Base): @property def field(self) -> int: return 42 with this, because the ABC defines it as just an attribute, so, needs to have a setter too), and you now made 2 distinct types: Base and BaseProto. not sure how would you even work with this in a real codebase.

[oblique urchin]

That's not true. Protocols can be used for inheriting executable (mixin) methods too, and they don't necessarily have anything to do with other languages.

[obsidian oar]

That is partially correct, the thing is that the subclasses you mention are no longer protocols.

from typing import Protocol, is_protocol


class Base[T](Protocol):
    value: T


class AImpl(Base[int]):
    value = 5


class BImpl[T: int](Base[T]):
    def get_value(self) -> T:
        return self.value


class CImpl[T](Base[T], Protocol):
    pass


def check_protocol(cls: type):
    caninst = True
    try:
        cls()
    except:
        caninst = False
    isproto = is_protocol(cls)
    print(cls.__name__, f"{isproto=}, {caninst=}")


check_protocol(AImpl)
check_protocol(BImpl)
check_protocol(CImpl)

Protocols should not be instantiated, and if possible, the abstract parts are expected to have already been implemented and cease to be a protocol.

[slender basin]

I'd say that qualifies for a new pyright bug, as the declared behavior change clearly doesn't apply to current pyright

[restive rapids]

same with ABC's, yet look into the example i shown

[restive rapids]

well, it does, with just a property protocol
but not what you have (ABC + protocol with a property)

[obsidian oar]

from typing import Protocol
from abc import ABCMeta


class SupportsFoo(Protocol):
    def foo(self) -> None: ...


class FooABC(metaclass=ABCMeta):
    def foo(self) -> None: ...


def check_proto(cls: type[SupportsFoo]) -> None: ...
def check_abc(cls: type[FooABC]) -> None: ...


class A:
    def foo(self) -> None: ...


class B(FooABC):
    def foo(self) -> None: ...


check_proto(A)
check_proto(B)
check_abc(A)  # fails
check_abc(B)

[slender basin]

Maybe I'm missing something?
https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAK4MYAxmADYBQVpFAhgM6NQASTAYkgKYUAmACiJgS5CgEoAXFSiyoAAQTgE3PHBly%2B3YFAhwA%2BsB78BjXsHFQAtAD5MKGJNkA6VzTpMWAYQHtGXXj4pDVk9Q2M%2BJ1QYKiA

[obsidian oar]

Here is a demonstration of why it works the same as interfaces in other languages.

[slender basin]

Doesn't look like ABC has something to do with it

[restive rapids]

the explicit subclassing of the protocol has to do with it, seems like
because

from typing import Protocol

class HasField(Protocol):
    @property
    def my_field(self) -> int:  ...

class C:
    my_field: int

def f(x: HasField):
    ...

f(C())

works

[obsidian oar]

When annotating with ABC, they are expected to be subclasses of this one, but when doing so with Protocol, their origins do not matter as long as they implement its interface.

Sorry, I assumed C inherited from some Protocol without reading it.

[restive rapids]

it does in the playground example, and thats what causes the issue. not sure why

[slender basin]

This reassures my perception of this behavior as a bug. If in f(x: HasField), x can satisfy HasField with just a my_field attribute - the same interpretation of "satisfy a protocol" should hold for class C(HasField)

[obsidian oar]

not inherit from a protocol

from typing import Protocol


class HasField(Protocol):
    @property
    def my_field(self) -> int: ...


class C:
    my_field: int


def check(cls: type[HasField]) -> None: ...


check(C)

[obsidian oar]

When inheriting, the subclass implementation is expected to maintain the type of the superclass, in this case the type is property and you overrode it with an instance variable

[slender basin]

This doesn't seem a technical necessity, just a design decision that could be taken either way. The same could be said about func(x: HasField) (Isn't "x is expected to maintain the type of HasField" here?). Hopefully pyright could at least be consistent in both cases (inheritance / direct decoration).

I gathered the courage to open a pyright issue with the wisdom collected here. Thank you! https://github.com/microsoft/pyright/issues/10257

[obsidian oar]

That's the magic of protocols, they are not tied to any type, but to the interface.

[slender basin]

As the mypy implementation proves, this is not a technical necessity.

Anyway, that went about as well as expected: https://github.com/microsoft/pyright/issues/10257#issuecomment-2781022666

[obsidian oar]

If it were the case that they are only linked to subclasses of this one, what would be the difference between the abstract classes? Exactly none.

When I say abstract classes I mean classes like ABC that rely on superclass-based validation.

[slender basin]

Which is exactly why I would expect the same rationale to apply when inheriting a protocol.

And obviously pyright doesn't respect this rationale when inheriting. What is the difference between inheriting a protocol and an ABC? seems none, sadly.

Anyway, I'm dropping this. Appreciate everyone's time and effort!

[obsidian oar]

I don't understand which part you are referring to, I consider the current implementation to be correct.

[slender basin]

I'm afraid not: https://basedpyright.com/?typeCheckingMode=all&code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAK4MYAxmADYCwAULaRQIYDOzUAEiwGJICmFAEwAURMCXIUAlAC5aUeVAACCcAl544chQN7AoEOAH1gfQUOb9gkqAFoAfJhQxp8gHTva9JqygBhIZzMPPwCMlryBsamAi6oMJ50NDp6wACuKKRCAB4ugcGCMlDurrRpGUL%2BkpJAA

[obsidian oar]

in your code

from typing import Protocol

class HasField(Protocol):
    @property
    def my_field(self) -> int: ...


class C(HasField):
    my_field: int

The override warning is fine, because it completely overrides the interface of its superclass.
property[int] != int

The correct way to subclass would be

class C(HasField, Protocol):
    @property
    def my_field(self) -> int: ...
    @my_field.setter
    def my_field(self, value: int) -> None: ...
    @my_field.deleter
    def my_field(self) -> None: ...

[pastel egret]

Possibly worth mentioning PEP 767 is a proposed change that solves this problem, by proposing the ReadOnly[] type qualifier.

[soft matrix]

oh i didnt realise that had finally progressed

thats wonderful

i cant wait for my cursed code to be fixed
https://github.com/Gobot1234/steam.py/blob/49be60ef7e7ad6c6ec7820f13a4c891dd7151dde/steam/_const.py#L214-L222

[rough sluice]

steam/_const.py lines 214 to 222

class _ReadOnlyProto(Protocol[T_co]):
    def __get__(self, __instance: Any, __owner: type) -> T_co: ...


ReadOnly: TypeAlias = T_co | _ReadOnlyProto[T_co]
"""PEP 705 style read only, should make things a easier transition when the feature gets added to nominal classes.

Currently does not mean anything to a type checker really.
"""```

[prisma talon]

Having some troubles with getting my IDE to recognize the correct type, I want to know where I am missing types. Note SlashCommand is a subclass of ApplicationCommand

@slash_command()
async def base_convert(ctx: ApplicationContext, value: str, to_base: int, from_base: int): ...

def setup(bot: Bot):
    print(type(base_convert)) # Returns discord.commands.core.SlashCommand
    bot.add_application_command(base_convert) # Type Checker Warning: Expected type 'ApplicationCommand', got '(ctx: ApplicationContext, value: str, to_base: int, from_base: int) -> Coroutine[Any, Any, None]' instead 

def slash_command(**kwargs):
    return application_command(cls=SlashCommand, **kwargs)

def application_command(cls=SlashCommand, **attrs):
    def decorator(func: Callable) -> cls:
        if isinstance(func, ApplicationCommand):
            func = func.callback
        elif not callable(func):
            raise TypeError("func needs to be a callable or a subclass of ApplicationCommand.")
        return cls(func, **attrs)
    return decorator

I tried typing slash_command as returning a SlashCommand but I think this is wrong and it changes the warning to
Expected type 'ApplicationCommand', got 'Coroutine' instead which does not make sense to me.

[polar aurora]

The type checker is confused about what base_convert is after it's decorated, basically

So that means the return type on the decorator is wrong probably

[prisma talon]

So why do i still get the warning show in the last 2 lines of my message?

[polar aurora]

The change that comes to mind for me looks like this..

def application_command(cls=SlashCommand, **attrs):
    def decorator(func: Callable) -> ApplicationCommand:  # Changed return type to base type
        if isinstance(func, ApplicationCommand):
            func = func.callback
        elif not callable(func):
            raise TypeError("func needs to be a callable or a subclass of ApplicationCommand.")
        return cls(func, **attrs)
    return decorator

and then slash_command's return type should probably be Callable[[T], SlashCommand] where T = TypeVar('T', bound=Callable)?

Maybe someone has a better approach

[prisma talon]

Have not really used type vars in python before. what does that do?

[polar aurora]

and application_command could be:

F = TypeVar('F', bound=Callable[..., Any])
def application_command(cls=SlashCommand, **attrs) -> Callable[[F], ApplicationCommand]:
  ...
``` ?

[prisma talon]

I am familiar with the concept of generics, but i dont understand bound, and why there are 2 arguments to callable

[polar aurora]

Basically they are placeholders for types that get determined later

bound is a 'limit' basically, and it's saying 'T has to be something that is Callable or a subtype.

Callable[] has two arguments.. the first is the type of the parameter it accepts, and the second is the return type

So ... is meaning 'any number of args of any type'

and Any is 'any single return value'

Callable[[str, int], bool] would be a thing that takes a string and an integer, and return a boolean for example

Here we need Callable[..., Any] because we want the decorator to work with any function

[prisma talon]

ok, thanks
how would callable work with kwargs?

[polar aurora]

I'm aware of two ways.. the first is to use ... which lets it handle any arguments whether they are positional or keyword

but typing also has a class in it called ParamSpec that is just for this

[rough sluice]

Not gonna happen.

No documentation found for the requested symbol.

[prisma talon]

Ah ok, it is more nested stuff

[polar aurora]

https://docs.python.org/3/library/typing.html#typing.ParamSpec

[prisma talon]

A Coroutine is a special type of callable correct?

[polar aurora]

So you can do like

P = ParamSpec('P')  # Captures all parameters (positional and keyword)
R = TypeVar('R')    # Return type

def decorator(func: Callable[P, R]) -> Callable[P, R]:
  ...

A Coroutine is not actually isinstance(obj, Callable), but they are super related

A regular function is Callable, as is an async function

but when you CALL an async function, it returns a Coroutine

[prisma talon]

ok

Your suggestions seemed to make the warning go away, thanks!

[polar aurora]

Where it gets tricky is using a single decorator to support both async and sync functions

THAT, I don't have a great solution for.. My best so far is just to have two different decorators, and that isn't always what you want

The problem is that your decorator wants to return probably THIS to be correct when decorating an async function:

def slash_command(**kwargs) -> Callable[[Callable[..., Coroutine[Any, Any, R]]], SlashCommand]:
  ...

To unpack that, we return a Callable[(stuff), SlashCommand] to indicate that we accept a thing I'll describe further, and return a SlashCommand.
The 'stuff' is Callable[..., Coroutine[Any, Any, R]], which is the signature of the async function we are trying to decorate.

That says 'takes any arguments, returns a Coroutine that yields Any, accepts Any via send(), and returns R, our type variable for return types.

I tried something like this last time and STILL got errors, so I'm not really sure what the most-elegant approach is:

from typing import Callable, Coroutine, TypeVar, Any, overload

R = TypeVar('R')

# sync function decorator
@overload
def slash_command(**kwargs) -> Callable[[Callable[..., R]], SlashCommand]: ...

# async function decorator
@overload
def slash_command(**kwargs) -> Callable[[Callable[..., Coroutine[Any, Any, R]]], SlashCommand]: ...

# The actual implementation
def slash_command(**kwargs):
    return application_command(cls=SlashCommand, **kwargs)

I would really like that last to work but mypy still hates it

[prisma talon]

The function will always be async in my case so I dont need to worry about that

[polar aurora]

Oh great. Then yeah the earlier stuff should work

[prisma talon]

thanks again!

[polar aurora]

No problem, I'm still learning all the tricks so explaining things helps me

[rugged anchor]

Does anyone here know what benefits there are when using the type soft keyword in Python 3.12+? It seems to be a less useful type alias at face value.

>>> A: typing.TypeAlias = str | bytes
>>> type B = str | bytes
>>> isinstance("hello", A)
True
>>> isinstance("hello", B)
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union

[trim tangle]

See https://docs.python.org/3/reference/simple_stmts.html#type:

• it's evaluated lazily
• it can specify a list of type arguments

Type aliases are supposed to be used as type annotations. But you could do something like isinstance("hello", B.__value__)

actually, pyright doesn't like isinstance(x, TypeAlias.__value__)

so I guess it's only usable as a type annotation (and inspectable at runtime)

[rugged anchor]

ah, thank you 🙂

[stiff acorn]

How do you order the decorators in an overload?

@overload
@shared_task
def func(foo: str, bar: str) -> str: ...
@overload
@shared_task
def func(foo: int, bar: int) -> int: ...

@shared_task
def func(foo: str | int, bar: str | int) -> str | int: ...

particularly, where what would be the correct order of decorators if I have another concrete decorator in the mix

[trim tangle]

What is the signature of the decorator?

[stiff acorn]

https://github.com/sbdchd/celery-types/blob/a526445789a38c6b4ce126349339bb29f7e29983/celery-stubs/app/__init__.pyi#L26

[rough sluice]

celery-stubs/app/__init__.pyi line 26

def shared_task(```

[trim tangle]

I don't think this is expressible

What type would func be?

[stiff acorn]

Task according to pylance?

My primary concern is can the order of the decorators affect runtime here?

[trim tangle]

Hm, it seems like this is allowed but it doesn't make sense

this is the same as just ```py
@shared_task
def func(foo: str | int, bar: str | int) -> str | int: ...

[stiff acorn]

yea

guess there's no point in overloading this?

[trim tangle]

yeah

[stiff acorn]

actually there might still be for places that call it func(foo, bar) instead of func.delay(foo, bar)

[trim tangle]

the inferred type of func will still be Task[[str | int, str | int], str | int]

so it won't remember the overload

[stiff acorn]

yep pylance does exactly this

but mypy manages it?

foobar/example.py:413: error: No overload variant of "__call__" of "Task" matches argument types "str", "int"  [call-overload]
    a = func("a", 12)
        ^~~~~~~~~~~~~
foobar/example.py:413: note: Possible overload variants:
foobar/example.py:413: note:     def __call__(self, foo: str, bar: str) -> None
foobar/example.py:413: note:     def __call__(self, foo: int, bar: int) -> None
foobar/example.py:414: error: overloaded function has no attribute "delay"  [attr-defined]
    b = func.delay("a", 12)
        ^~~~~~~~~~
Found 2 errors in 1 file (checked 59 source files)

[trim tangle]

What does it show for reveal_type(func)?

[stiff acorn]

note: Revealed type is "Overload(def (foo: builtins.str, bar: builtins.str), def (foo: builtins.int, bar: builtins.int))"