type-hinting

[wary pulsar]

Ok, but if I remove the defaults then a non-default arugment follows a default argument

Thank you, btw.

[oblique urchin]

Yes. To deal with this you need to split the second overload into two: one where other_arg doesn't take a default and copy: Literal[True] is positional

and another where it does, but copy: Literal[True] is keyword-only

If this is a function you're designing, it may also be worth making copy keyword-only anywhere, since it's probably clearer for users

[wary pulsar]

Oh, I see, because if copy is positional, then other_arg is also guaranteed to be positional

Yes, that is a good point.

The lights have finally turned on. Thank you.

[oblique urchin]

fyi we also recently added this to typing.readthedocs.io https://github.com/python/typing/pull/1894/files

[wary pulsar]

Great!

[iron vapor]

I think I asked this question before, but how can I get a staticmethod to indicate it's going to return an object of the type of class it's a part of?

Oh, nevermind, I should read pylance errors more carefully

If using double quotes on a union type, I have to double quote the whole thing

I personally find it ugly to look at, but I guess its the answer...

[trim tangle]

You can use from __future__ import annotations

(especially if you're not inspecting annotations at runtime)

[iron vapor]

Oh, then I don't need to double quote/

[trim tangle]

yes

[iron vapor]

I saw that that's expected in 3.14

I'll try that out

I don't mind double quoting on just the object (though I guess I'd prefer not to there too)

but double quoting unions is weird

[trim tangle]

It's more complicated. from __future__ import annotations implements behaviour specified in PEP 563. But that is going to be deprecated in the future, and PEP 649 will be implemented instead, as a default in 3.14

but you can still use the PEP-563 behaviour, you'll just need to remove the import in future versions

[iron vapor]

I guess I need to go read both peps to understand what's going on

[trim tangle]

It should only affect runtime behaviour, so if you're just using annotations for static analysis, nothing should change

[iron vapor]

I guess to keep going with type annoations... This method returns bool | Token. It returns bool if no flags are passed to the method, and a Token if the flag is passed. Can I make the interpreter aware of this somehow? Because it's leading to some type checking problems. if I have a subclass of token, how can I set up a type hint to say if the method is called with the param to return Token, that the subclass is valid? Right now, because its unaware, it's telling me bool cant be assigned to Token

[trim tangle]

Do you have the code right now?

You probably want typing.overload

[iron vapor]

    @staticmethod
    async def validate_token(
        token: str, email: Optional[str] = None, get_obj: bool = False
    ) -> "Optional[bool | Token]":
        async with session_manager.session() as db_session:
            get_token = select(Token).where(Token.token == token).limit(1)
            if email:
                get_token = get_token.join(Token.user).where(User.email == email)
            token_obj = await db_session.scalar(get_token)

        if get_obj:
            return token_obj if token_obj else None
        return True if token_obj else False

Mind you, I'm also open to being told a better way to set up the method

[trim tangle]

    @staticmethod
    async def validate_token(
        token: str, email: "str | None" = None
    ) -> "Token | None":
        async with session_manager.session() as db_session:
            get_token = select(Token).where(Token.token == token).limit(1)
            if email:
                get_token = get_token.join(Token.user).where(User.email == email)
            token_obj = await db_session.scalar(get_token)
        return token_obj

if you want a boolean, compare the result to None

(or maybe add another method)

[iron vapor]

Makes sense, drop the bool out of the method in general

Out of curiosity, do you prefer Token | None over Optional[Token]? Is there a difference I'm not aware of?

[trim tangle]

Yes, it's often a good idea to split a function like py def get_foo(also_bar=False): # foo stuff if also_bar: foo = barify(foo) return foo into ```py
def get_foo():
# foo stuff
return foo

[iron vapor]

Yah, that makes sense

[trim tangle]

Functionally it's the same, but Optional is just a confusing name

[iron vapor]

This is definitely one of those "I've been staring at my own code too long and missed it"

[iron vapor]

I kinda agree

[trim tangle]

because optional arguments are already a thing, and are not related to typing.Optional at all

[iron vapor]

Thank you for putting it like that. I think that was clicking around my brain, but I never got the phrasing I wanted for it.

Ok, typing is improved, for the moment

I may go through my code and remove all the Optional references

I do think | None is also verbose in a way I don't like, but verbosity is good sometimes I guess

[viscid spire]

is it verbose?

I mean it's less characters to use union operator with None

I like consistency with other unions

[young zealot]

Not sure if this counts for this channel, but I have been using mypy with vscode, and before when I autocompleted methods to override or protocol methods, it used to autocomplete with all the type information, but it just started autocompleting with no type information. Anyone know what is wrong and how to fix it?

[trim tangle]

mypy does not provide any autocompletion

can you provide more details perhaps? like screenshots

[young zealot]

An example is I have a protocol like:

class Test(Protocol):
    def test(self, param: int) -> int:
        ...```
I would create an implementation confirming to that protocol and get to this point where I'm defining the function:
```class Blah(Test):
    def te```
And as I am typing the function name it will show an autocomplete with a tooltip showing the implementation with all the info like:
```def test(
    self: Self@Test,
    param: int
) -> int```
And then when I autocomplete the function with tab I just get:
```    def test(self, param):
        return super().test(param)```
When I used to get:
```    def test(self, param: int) -> int:
        return super().test(param)```

[trim tangle]

That seems like a problem with pylance (assuming that's what you're using for autocompletion), I'd make an issue on their github repo https://github.com/microsoft/pylance-release

because I do get this ```py
class Foo(Test):
def test(self, param: int) -> int:
return super().test(param)

[young zealot]

ah now that I know its pylance I installed the last version and its working again, so might be an issue with the latest version

thanks

I'll see if its reported or not

[wraith linden]

Is there currently any way to type hint something like this function? ```py
import re

class A(int): pass
class B(int): pass
class C(int): pass

def parse_ints(s, *types):
return tuple(
t(num)
for t, num in zip(types, re.findall('\d+', s))
)

a: A
b: B
c: C

a, b, c = parse_ints('1 2 3', A, B, C)

[trim tangle]

the best you can do without significantly changing the code is

from collections.abc import Callable

def parse_ints[T](s: str, *types: Callable[[str], T]) -> tuple[T, ...]:

that would assign the type of A | B | C to each of the a, b and c variables

[wraith linden]

Oh right. Although I could do something with overloads if I limit the length of types right?

[trim tangle]

Yes, you can do an overload staircase. I probably should've mentioned that

from collections.abc import Callable

from typing import overload

type _FromStr[T] = Callable[[str], T]

@overload
def parse_ints[A](s: str, t0: _FromStr[A], /) -> tuple[A]:
@overload
def parse_ints[A, B](s: str, t0: _FromStr[A], t1: _FromStr[B], /) -> tuple[A, B]:
@overload
def parse_ints[A, B, C](s: str, t0: _FromStr[A], t1: _FromStr[B], t2: _FromStr[C], /) -> tuple[A, B, C]:
@overload
def parse_ints[A, B, C, D](s: str, t0: _FromStr[A], t1: _FromStr[B], t2: _FromStr[C], t3: _FromStr[D], /) -> tuple[A, B, C, D]:

[trim tangle]

There is a way to do it without this abomination, but it requires significantly warping the calling style in a very inconvenient way ```py
import re
from collections.abc import Callable

class ParseInts[*Ts = *tuple[()]]:
def init(self) -> None:
self._fns = []

def add[T](self, fn: Callable[[str], T]) -> "ParseInts[*Ts, T]":
    new = ParseInts()
    new._fns = self._fns + [fn]
    return new  # type: ignore

def run(self, s: str, /) -> tuple[*Ts]:
    return tuple(t(num) for t, num in zip(self._fns, re.findall(r"\d+", s)))

result = ParseInts().add(int).add(lambda s: float(s) + 0.5).add(bool).run("42 0.69 111")

inferred as: tuple[int, float, bool]

[wraith linden]

Oh right yeah that might be a good approach. Maybe with operator overloading it might not be too bad. Something like: ```py
ABCParser = Parse(A) + Parse(B) + Parse(C)
a, b, c = ABCParser(string)

[wraith linden]

Sorry, meant to reply.

Thanks for the ideas!

[grand furnace]

Hi,
When I try to use the strict mode like this mypy --strict my_file.py on this code:

def my_function(x: int) -> int:
    if x < 0:
        raise ValueError("x needs to be higher than 0.")
    return 10 ** x

value = my_function(20)

I get this error:

Untitled-1.py:4: error: Returning Any from function declared to return "int"  [no-any-return]
Found 1 error in 1 file (checked 1 source file)

I don't understand why it happen.
A positive integer exponent another positive integer will always return a integer, so why does the typing is Any?

[trim tangle]

power is notoriously difficult to type because the result type depends on the argument value. Python's type system can't track these complex value conditions (there's no way to write "this function returns a different type if the argument is positive"), so for now we have to live with Any

[grand furnace]

I couldn't find a issue on mypy repos, so should I open one?
So, for the moment, the only way to not get this error is to use # type: ignore?

[trim tangle]

Yes, for now you can use a # type: ignore[no-any-return]

There's this, I guess: https://github.com/python/mypy/issues/7765#issuecomment-544645704
You can open an issue on the python/typing repo about this

though I don't see any elegant solution to this that wouldn't add a lot of unnecessary complexity to the type system

[grand furnace]

Thanks, I didn't know about this

[grand furnace]

Ok, I opened a issue: https://github.com/python/typing/issues/1898

[trim tangle]

The reason this is the right repo is that it's not a problem specific to mypy, but a more general typing problem (it's going to be an issue with pyright, pyre, editor completions, autogenerated documentation etc.): what can be done about the design of the type system so that ** does not return Any? (or maybe it's fine as is?)

also the typing repo has fewer open issues 😛

[thick quiver]

Anyone know of a way to get the AST of a script with inferred types (via mypy, pyright, or otherwise)? I can't find any existing solution for this

[trim tangle]

You can check how this repo does it https://github.com/Zzzen/pyright-ast-viewer

though pyright is written in TypeScript, so it's going to be hard to transfer that information efficiently back to Python

[thick quiver]

Nice! But hmm yeah, seems like it's not easy to work with

I'm just surprised I haven't been able to find anything that already exists for stuff like that

[oblique urchin]

there's one in pyanalyze (which I wrote)

[thick quiver]

Ah cool! You mean using the ast_annotator functions? Or type_from_ast?

[oblique urchin]

yes, the ast_annotator module

[solid onyx]

hey all, qq about stubs/types. For context, I'm running ironpython scripts in a .net environment. Some things in my script are directly referenceable e.g. the literal "cat" which is a string variable. I don't have to import it, it's just available when my ironpython script is executed.

To my understanding, for local development, I'd have to import the stub that contains the type hints. Is there a way to make the stub always active that my editor has info on without having defined/imported it in the script?

Hopefully this makes sense, I'm terrible at explaining things. If it doesn't, here's the basic idea:

if my script is a single line:

print(cat)

this runs in my environment, but my editor has no idea what cat is. And if I create a stub the usual way, I'd have to add an import statement that breaks in runtime

maybe this is a question for the vscode dicord, but wanted to check here in case there's something in pep484 covers this and I'm missing it

[cinder bone]

you could prefix your script with

cat: str
# ...

[solid onyx]

I've been trying to find workarounds like that, but no dice. cat was just an example. In reality, I'll be referencing arbitrary classes that exist in loaded .net assemblies. I wrote a script that crawls these dlls and generates stubs from the signatures, but that only works for classes that are scoped within a namespace. Those that aren't don't fit into the python import model (to my understanding, would love to be proven wrong here)

to be more clear, I can create stubs for classes in namespaces because I can import them in ironpython like

from namespace import class

but classes not in a namespace are imported like

import class

but python interprets this as a module, not a class. It's almost like a need an __init__.pyi

[wary pulsar]

Does the difference between TypeIs and TypeGuard have to do with covariance/invariance? Like it says you can't use TypeIs to narrow list[object] to list[int], but you can use TypeGuard? In my case, I want to write type cooercion functions for checking the shape and type of np.ndarray.

[wary pulsar]

And, why would you write

def (lst: list[object]) -> TypeGuard[list[int]]: ...

When you could just write

def (obj: object) -> TypeIs[list[int]]: ...

ETA: ...which is to say, I'm trying to envision where you'd rather use the former...

[viscid spire]

I thought one raises and the other should return a bool

Maybe I got it mixed up tho

[wary pulsar]

No, they both return a bool.

[viscid spire]

Hmmmmm

Reading in the docs, they sound identical at face value

[wary pulsar]

This section of PEP-742 talks about the differences, I'm just struggling to grok it.

For example:

TypeGuard is the right tool to use if:

• You want to narrow to a type that is not compatible with the input type, for example from list[object] to list[int]. TypeIs only allows narrowing between compatible types.
• Your function does not return True for all input values that are compatible with the narrowed type. For example, you could have a TypeGuard[int] that returns True only for positive integers.

I get the second bullet point.

I'll keep reading and thinking about it.

[cinder bone]

the main thing is that you can do

def guard(x: list[object]) -> TypeGuard[list[int]]: ...

but not

def guard(x: list[object]) -> TypeIs[list[int]]: ...

because list[int] is not assignable to list[object].
Also TypeGuard only narrows in the positive case I believe, while TypeIs also includes the negative case

from typing import reveal_type, TypeGuard
from typing_extensions import TypeIs

def guard_str(x: object) -> TypeGuard[str]:
    return isinstance(x, str)
def is_str(x: object) -> TypeIs[str]:
    return isinstance(x, str)

def foo() -> None:
    x: str | int
    if guard_str(x):
        reveal_type(x)  # str
        return
    reveal_type(x)  # str | int
    
    y: str | int
    if is_str(y):
        reveal_type(y)  # str
        return
    reveal_type(y) # int

[young zealot]

Is there any reason TypeVar is always used with a single letter? Is there any issue with doing something like DescriptiveT = TypeVar('DescriptiveT') instead of just T = TypeVar('T')?

[restive rapids]

not "always", but usually its because well, its a generic type, you cant say much about it, and there are shorthands that people are used to that cover most of the usual usecases, like T for just "some type", K for a key type in some mapping (V for value then, respectively).
when defining something like fold[A, B]((A, B) -> A, A, Iterable[B]) -> A, what names would you even use? its not like it could become any more clear as they are just very generic things that describe a relationship (function should act on accumulator value and element from iterable, would you do fold[Accumulator, Element]((Accumulator, Element) -> Accumulator, Accumulator, Iterable[Element])? to me that is honestly just eye sore)
you can also commonly find variance being added as a prefix, e.g. T_co or T_contra

[young zealot]

ok thanks

[pliant vapor]

is there a proper way to type hint like a list of 2-tuples that have the same types in each tuple, but not across the list? like

[(1, 1), ('a', 'b'), ((1, 3), (2, 4))]

is fine but

[(1, 'a')]

wouldn't be
my actual use case is typehinting a list/dict of callables of the form Callable[[int | T, int | T], T], where each callable could be attached to a different type

[pliant vapor]

kinda something like

type _FuncType[T] = Callable[[int | T, int | T], T]
class Foo[*Ts]:
    def bar(self, *args: tuple[*map(_FuncType, Ts)]): ...

if that helps

[trim tangle]

How would you use it?

do you have an example maybe?

[pliant vapor]

i have a binary tree with ints as leaves, and the callables would be combining 2 values to calculate a value for parent nodes
after thinking it through, it probably makes more sense for it to be list[int] | T or something like that, but that doesn't matter for what i'm asking
so i could do like

tree = Tree(max=max, concat=list.__add__)

and then later have

isinstance(tree.some_attrib['max'], int)
isinstance(tree.some_attrib['concat'], list[int])

i could probably get a list of possible attributes and their types as well if that helps things

[trim tangle]

I don't think I fully understand

[pliant vapor]

segment tree is the proper term for what i'm doing

[trim tangle]

About your original question: every list of tuples fits this description. Every item in the (1, 'a') tuple has the type int | str for example

I don't know what a segment tree is

[pliant vapor]

ah that's a good point

[trim tangle]

Ah, so you want to have different "reductors" or "traversals" acting on the same tree in parallel or something like that?

[pliant vapor]

yes

[trim tangle]

If you want to do it the simple way, I'd just slap a bunch of Anys and just make a dict with the results or something like that

If you want to do it the hard way to preserve typing information, you can do something like this ```py
class Reductor[T]:
def init(self, name: str, fn: Callable[[T | int, T | int], T]) -> None:
self.name = name
self.fn = fn

def concat(left: list[int] | int, right: list[int] | int) -> list[int]:
if isinstance(left, int):
left = [left]
if isinstance(right, int):
right = [right]
return left + right

max_reductor: Reductor[int] = Reductor("max", max)
concat_reductor: Reductor[list[int]] = Reductor("concat", concat)
ducky_reductor: Reductor[Ducky] = Reductor("ducky", some_fn)

tree = Tree(reductors=[max_reductor, concat_reductor])
max_result = tree.reduction_result(max_reductor) # returns an int
concat_result = tree.reduction_result(concat_reductor) # returns a list[int]
tree.reduction_result(ducky_reductor) # error, reductor was not registered

this is achieved with ```py
class Tree:  # assuming it works with ints
    def __init__(self, reductors: Iterable[Reductor[Any]]) -> None:
        ...

    def reduction_result[T](self, reductor: Reductor[T]) -> T:
        ...

[pliant vapor]

hmm

this could work

thx

[harsh lantern]

i have a callback protocol```py
class SupportsWrappedByPythonFunction(Protocol):
"""Protocol for functions that support being wrapped by PythonFunction"""

#pylint: disable=too-few-public-methods

__name__: str

@abstractmethod
def __call__(self, meta: Optional[Meta], interpreter: 'ASTInterpreter', /, *args: Value
             ) -> ExpressionResult:
    ...

so if I have a function likepy

BLList and Int are subtypes of Value, and Null is a subtype of ExpressionResult

def list_insert(meta: Optional[Meta], interpreter: 'ASTInterpreter', /,
list_: BLList, index: Int, item: Value,
*: Value
) -> Nullit doesnt satisfy the protocol it said something likepy
Argument of type "(meta: Meta | None, interpreter: ASTInterpreter, /, list: BLList, index: Int, item: Value, *: Value) -> Null" cannot be assigned to parameter "function" of type "SupportsWrappedByPythonFunction" in function "init"
Type "(meta: Meta | None, interpreter: ASTInterpreter, /, list: BLList, index: Int, item: Value, *_: Value) -> Null" is not assignable to type "(meta: Meta | None, interpreter: ASTInterpreter, /, *args: Value) -> ExpressionResult"
Parameter 4: type "Value" is incompatible with type "Int"
"Value" is not assignable to "Int"

[trim tangle]

Suppose that you have this:

class Foo(Protocol):
    @abstractmethod
    def __call__(self, *args: int | str) -> None:
``` you should be able to do `foo()`, `foo("bar")`, `foo(42, "bar")` and so on. Therefore, a function that only supports an `int` as the second argument does not satisfy this protocol.

If you are doing some runtime inspection of annotations, you'll probably have to do *args: Any

This avoids many of the downsides of Any, because presumably you'll have an error at module import time if some of the parameters are wrong

[harsh lantern]

so i have to use Any?

[rustic gull]

Yo

[trim tangle]

yes

[eager tulip]

Hey

[eager tulip]

Yes you have to

[wicked scarab]

I'm working on a python class that mimics an iterable by implementing the __getitem__ method. However, when I try to pass an instance of this class to the max() function, pyright raises an error.

class Foo:
    def __init__(self) -> None:
        self.data = [1, 2, 3, 4, 5]

    def __getitem__(self, item: int):
        return self.data[item]

print(max(Foo()))

This is the pyright error I am getting:

Argument of type "Foo" cannot be assigned to parameter "iterable" of type "Iterable[SupportsRichComparisonT@max]" in function "max"
  "Foo" is incompatible with protocol "Iterable[SupportsRichComparisonT@max]"
    "__iter__" is not present Pyright (reportArgumentType)

From the python documentation, implementing __getitem__ allows an object to be iterable, even if __iter__ is not explicitly defined. This is why the code runs fine when executed. However, pyright seems to expect the class to have an __iter__ method to be recognized as an iterable. Am I missing something?

[trim tangle]

I think the __getitem__-based iterators are considered somewhat obsolete, perhaps Pyright just doesn't support them.

In the stubs for the standard library (typeshed), Iterable is just a protocol, not a union, which would be required to support the two flavours of iterables

If you want to make your class iterable, you will need to do ```py
from collections.abc import Iterator

...
def iter(self) -> Iterator:
return iter(self.data)

[wicked scarab]

why is it considered obsolete?

[trim tangle]

Wait, I think it might work if you add a __len__

[wicked scarab]

It doesnt

mypy also produces a similar error

[trim tangle]

Well, maybe not obsolete, but superseded
https://discuss.python.org/t/deprecate-old-style-iteration-protocol/17863/19
It was added to Python before 2001, and is kept for backwards compatibility reasons now that we have an actual iterator protocol with __iter__ and __next__

it's not deprecated, but tooling may decide to not support it

[wicked scarab]

I see, thanks

[tender steeple]

I tried to follow a YouTube tutorial for dice roll project

And I don't know what's wrong

[tender steeple]

That's the video one

[viscid spire]

what does this have to do with typehinting?

[tender steeple]

Idk I'm new here

[viscid spire]

if you have a question, post that question in text in the python help channel

#❓｜how-to-get-help

[tender steeple]

Okie ty

[wooden cipher]

hi
how do you type hints Iterables except dict?
meaning i can accept list, tuple, set, generator and the like, but not dict

[trim tangle]

Why though?

[wooden cipher]

well the code accepts those

[trim tangle]

why does the code not work with dict as an iterable?

[wooden cipher]

https://github.com/valkey-io/valkey-py/blob/main/valkey/commands/core.py#L3341

i'm working on this code
basically there is one name, it can have many values

[rough sluice]

valkey/commands/core.py line 3341

def sdiff(self, keys: List, *args: List) -> Union[Awaitable[list], list]:```

[wooden cipher]

or in this case, just many keys which will be compared

[trim tangle]

And you want to annotate the keys as an iterable?

[wooden cipher]

yes, it says list now, but it can accept other types

[trim tangle]

and why should dict not be allowed?

[wooden cipher]

well, it's not my code, i have no say in it
but i don't see how one would use a dict for this

[viscid spire]

combining List and list... nice

[trim tangle]

redis type annotations are pretty terrible, so I don't quite understand how I would call this method

Did they mean def sdiff(self, keys: KeyT, *args: KeyT)?

[wooden cipher]

it wil work either way:

v.sdiff("a", "b")
v.sdiff(["a", "b"])
v.sdiff(("a", "b"))
...

[trim tangle]

Well, suppose I have {"a": 1, "b": 2}. why can't I pass that to v.sdiff?

[wooden cipher]

what is 1 and 2?

but again, i don't control the code

it'll just error

[viscid spire]

the keys: List is confusing...

[viscid spire]

useless info in this case

[trim tangle]

Ah, you mean the dict won't work at runtime?

[viscid spire]

but perhaps you already have data in that format, and just want to use the keys for this method

[wooden cipher]

no, it'll error

[viscid spire]

huh

that's strange

[trim tangle]

do you have a traceback? I don't see from the code why it wouldn't work

[wooden cipher]

hmm
it seems like i was testing sadd, which errors

sdiff seems to work
i stand corrected, sorry for that 🫠

[wooden cipher]

https://github.com/valkey-io/valkey-py/commit/ff75e7eb47fb4040d154806a1b99914a2db69c65#diff-f9cbda6fb662c8cfd8129648136d4a8401b4f737eb1c22da4b4fe221fc8cce72L3351-R3352
this has been my solution so far

[trim tangle]

To answer your question, you can do this:

@overload
def my_function(x: dict[Any, Any], /) -> Never:
    """does not accept dicts!!!"""
@overload
def my_function(x: Iterable[str], /) -> ActualReturnType: ...
``` but that might just confuse the caller when they pass in a dict and see that half of the code is greyed out in the editor, and it doesn't cover all cases, for example:
```py
def f(things: Iterable[str]) -> None:
    my_function(things)
f({"a": 5})  # ok
``` this is the main problem with introducing some kind of "X but not Y": it can be hard to propagate this "but not Y" part without introducing false positives and a lot of complexity

One notorious example of this is the dict() class constructor. If given a dict, it copies the content. But if given any other iterable which is an iterable of pairs, it will treat them as key-value pairs. This leads to an inconsistency when you have a dict at runtime but only an Iterable at analysis time

from collections.abc import Iterable

things1 = {(0, 0): "a", (1, 1): "b"}
foo: list[Iterable[tuple[int, int]]] = [things1]
things2 = foo[0]  # should be the same...

bar = dict(things1)  # inferred as: dict[tuple[int, int], str]
baz = dict(things2)  # inferred as: dict[int, int] (incorrect)

[wooden cipher]

- def sdiff(self, keys: List, *args: List) -> Union[Awaitable[list], list]:
+ def sdiff(self, keys: Union[Iterable[str], str], *args: Iterable[str]) -> List:

does this makes sense here?

since it accepts

v.sdiff("a", "b", "c")
# and
v.sdiff(["a", "b", "c"])

i'm not sure if *args should be Iterable or Union[Iterable, str

[trim tangle]

str is an Iterable[str] already

Though it seems like the items can be valkey.typings.KeyT and not just str

[wooden cipher]

you're right, i should've tested that first

actually it seems like KeyT is also lacking
numbers are also supported

[trim tangle]

One thing that I don't know how to resolve well is the Awaitable bit. The same class being used for both sync and async operations messes everything up

Maybe it's better to make type stubs with two different classes

[wooden cipher]

we're removing Awaitable from valkey-py
that's plain wrong

[trim tangle]

ah, it's going to be sync only?

[wooden cipher]

as far as i can tell, Awaitable is for when you are returning an awaitable
valkey-py doesn't return any awaitable

async methods are awaitable themselves
the return value is not

https://docs.python.org/3/library/typing.html#annotating-callable-objects
this was my refrence

[trim tangle]

async def foo() -> int:
    pass
``` If you do `x = foo()`, then `x` is a `Coroutine[int, Any, Any]`, which is an `Awaitable[int]`. The `async def` part is important.

if you do py def foo() -> int: pass you can't do await foo(), as int is not awaitable

[wooden cipher]

async def on_update(value: str) -> None:

that's from the docs

[trim tangle]

Right. It means that you can do await on_update("foo") . It the async keyword is not there, you wouldn't be able to do await on_update("foo")

[wooden cipher]

well yes
the point is the retun value is not Awaitable

[trim tangle]

But the sdiff method in this example is not marked with the async keyword

If you just have def sdiff(self, ...) -> int:, you can't do await client.sdiff(...)

However, redis does support async mode: https://valkey-py.readthedocs.io/en/latest/examples/asyncio_examples.html#Sentinel-Client

ok = await r.set("key", "value")
assert ok
val = await r.get("key")
assert val == b"value"
``` but whether `r.set` returns something awaitable can only be known at runtime

[wooden cipher]

huh

well that'll need some reverting to fix

[trim tangle]

I think the async thing is the major problem with redis/valkey typings right now. It makes it very frustrating to use with a type checker or a type-aware editor

you're essentially forced to do a # type: ignore every time you run a redis command

[wooden cipher]

well i should go tell people my code was wrong
i'll throw in you're argument there, see how they feel about it

[trim tangle]

There must be some existing issues on this in the redis repo

[trim tangle]

this https://github.com/redis/redis-py/issues/2399

Apparently some stubs exist https://pypi.org/project/types-redis/

actually no, the stubs are not dead

[wooden cipher]

there are some efforts to bring the stubs to valkey-py
it'll takes some time
and the end goal is the have the type hints in the code base
so i'll make small commits here and there

[grave fjord]

You should probably use a Generic around an is_async flag, or two different classes

The coiled client is a good example

https://inspector.pypi.io/project/coiled/1.69.1.dev29/packages/04/17/74ef3c8799f6d3aa4a3d36e54d687b6b03cd98af7e60c54a21b04a764828/coiled-1.69.1.dev29-py3-none-any.whl/coiled/core.py#line.574 note the overload

It uses a self-type of self: Cloud[Sync] or self: Cloud[Async]

[trim tangle]

nice hacking attempt

[grave fjord]

Inspector is usually used to report malicious software to the PyPIpopo

[trim tangle]

Actually yeah, this is this pretty clever

[grave fjord]

And the init has a return type annotation

[wooden cipher]

https://github.com/valkey-io/valkey-py/issues/162

I've opened this issue
if you want to contribute

since I'm not the maintainer

[grave fjord]

Also it shouldn't be Awaitable[T] it should be Coroutine[Any, Any, T]

[wooden cipher]

do you have time to participate in the issue?
there is a counter argument that, whether valkey/redis use a sync method to return coroutine, or use an async method, they both return coroutines that are later awaited

[grave fjord]

What

[wooden cipher]

if we use async def, it returns a coroutine, right?

and if we use def, in this case, it still returns a coroutine

as per PEP 484
"""
Coroutines introduced in PEP 492 are annotated with the same syntax as ordinary functions. However, the return type annotation corresponds to the type of await expression, not to the coroutine type:
"""

[grave fjord]

No that's not correct

In an async def the type is automatically upgraded to Coroutine[Any, Any, T]

This means if you define a method to be overridden it needs to allow Coroutine to be returned

Or be async

It's like imagine you had an operator food like:

food gluten_free -> Ham

And a special operator sando which is like food but upgraded the type to Sandwich[T]

sando gluten_free -> Ham

People consuming the sando function would be expecting just Ham but get a Sandwich[Ham]

[wooden cipher]

hi
sorry if i'm taking to long to understand
so:

async def a() -> int:
    return 1

and

def b() -> Coroutine[Any, Any, int]:
    return a()

?

[trim tangle]

Coroutine[int, Any, Any]
But otherwise yes, those annotations would be correct

[wooden cipher]

ok thanks

on another note:

do you have any ideas on how to fix valkey-py/redis-py's issue?

the methods are defined in a class which is then inherited by both async and sync clients
they don't have the will not man power to maintain two separate classes for async and sync

so in both case, they have one method, which calls execute_command, execute_command is either async, if sing the async client, or sync, if using the sync client

[trim tangle]

graingert showed an example above #type-hinting message

[wooden cipher]

you mean the @overload part? cause the link is to a line with only ]

[trim tangle]

Yeah, the create_api_token method for example

[wooden cipher]

makes sense
i was thinking maybe a generic type would work
but haven't tested that, and not sure if it's supported on python 3.8

[trim tangle]

Yeah, this all should work on 3.8

Though 3.8 is EOL already

[grave fjord]

I'd probably use two types

With a generic and overloads you need 3x the code

With two classes it's only 2x

[grave fjord]

You could also use unasync

[wooden cipher]

yes but the code would need maintaining the logic, where as an overload is just types

i personally prefer your way, but I'm not sure if the valkey team has the man power to maintain that

I'll let them know tho

[wooden cipher]

I don't know what that is, I'll look around 👍

[grave fjord]

https://github.com/python-trio/unasync

You write the code once and generate your sync client from your async client

[wooden cipher]

thanks 🙏
I've suggested it in the issue https://github.com/valkey-io/valkey-py/issues/164

[foggy garnet]

can anyone recommend a small, readable example of pydantic-like functionality?

e.g., classes that, on __init__, execute type-annotation-resolved functions on each of their fields

[rare scarab]

via @dataclass_transformer?

[foggy garnet]

I was not aware of @dataclass_transformer, that's pretty interesting! I will see if this can work

[oblique urchin]

It's purely for static typing support, it doesn't make any runtime logic happen

[iron vapor]

Ok, so I'm not really sure if this can be done, but... I'm working on a FastAPI project. I have a globals.py file in which right now I just have

current_user: User | None = None

I have a middleware that checks if a JWT is passed, and if set/valid, sets globals.current_user to the appropriate user. But in an endpoint that's only accessible if you're logged in (I handle this via decorators), I'm trying to use globals.current_user, but since it has the | None, I get the pylance error that the property I'm calling is not set on None. The error makes sense, since the code doesn't know if it's a User or None at this point, but can I somehow tell it?

[brazen jolt]

you can use typing.cast

that said, this seems like a really weird use-case, why not use fastapi dependencies to inject the current user properly?

[brazen jolt]

from typing import cast, reveal_type

x: int | None = 5

reveal_type(x)  # int | None
reveal_type(cast(int, x)) # int

[iron vapor]

Eeer, mostly because I'm new to this and didn't know I should/how... can you point me to something? I know the general premise of FastAPI depdencies (I use it for the database connection), but I'm not sure how I'd use it for the current user without avoiding this problem there too?

[brazen jolt]

you can also just do a none check, which would be safer than assertions: ```python
x: int | None = 5

if x is None:
return

reveal_type(x) # int

[iron vapor]

Based on what you're saying, moving the code to depdencies wouldnt make sense

So I'm guessing I should be doing something different in the depdency?

Ok, both of those examples make sense

But it sounds like I'm generally doing something wrong?

[brazen jolt]

you can check out the docs for oauth2 based auth:
https://fastapi.tiangolo.com/tutorial/security/simple-oauth2/?h=auth#authenticate

[iron vapor]

Because it wouldnt make sense to test/cast on every endpoint that requires A LOGGED IN USER

Ok, thanks

[brazen jolt]

it's not exactly the same as usual token based auth, but you can adjust the code from this fairly easily-ish

[iron vapor]

Yah, anything to help point me is great, thank you

[brazen jolt]

for some reason, the fastapi docs only show the password-bearer oatuth2 flow, which is often not what you want if you have some other way of generating & distributing api keys, this stack overflow question provides a nice example with pure http-bearer security scheme:

https://stackoverflow.com/questions/62994795/how-to-secure-fastapi-api-endpoint-with-jwt-token-based-authorization

for the most part, the fastapi docs are outstanding, but for auth, they're actually quite lacking imo for certain things

[iron vapor]

Ok, I'll have to try that out. It's raising the unauthorized, which I wouldn't want to do in this case, I think? Or maybe I just need to rethink things.

[brazen jolt]

if someone without a token tries to access a protected endpoint, it does make sense to raise unauthorized

why don't you want to do that?

[trim tangle]

Having it as a global variable is definitely wrong, because multiple requests can be in progress at the same time

[brazen jolt]

just set the amount of workers to 1 and don't use any async, it's fiiine

[oblique urchin]

it would make your app much more interesting to have it as a global

[trim tangle]

I had a similar situation, an application was using thread-locals (not contextvars) for the "currently selected language", and if two users with different languages made requests at just the right (wrong?) time, one of them would have the page rendered in the wrong language.

(I didn't write that code!)

[iron vapor]

Interesting, I figured with multiple requests, it would have a different global var per request? I'm guessing I'm thinking about the flow wrong.

But I'll try out the dependancy. It was recommend I make all paths private, and have a decorator for the public ones,which may be why I'm having trouble thinking of how this should work.

And also why is a middleware right now.

But wouldn't the original type issue still arise in a situation where the user is optional? A public where a logged in user gets one set of data, non logged in another?

[brazen jolt]

it wouldn't and that's the problem, if an authorized request arrives right after an unauthorized request, it will override the variable, and as you then process the first unauthorized request, that global variable will hold the user from the previous request, allowing them to bypass auth entirely

[brazen jolt]

well, you enforce dependencies by requiring them from the specific router, you can make 2 routers, one with the dependency required (making it only allow authorized requests) and one without it specified, allowing all requests

[brazen jolt]

it won't be optional though, either the dependency returns a user, or it fails, raising an exception that leads to 401

[iron vapor]

Yah, that's the part I have to figure out...

[median gulch]

Is there a way to have the type checker infer the type of an Union created from a constant tuple[type, ...] ?
Example:

NUMERIC_TYPES = (int, float)
STRING_TYPES = (str, MyCustomStringType)
KNOWN_TYPES = NUMERIC_TYPES + STRING_TYPES # tuple[type, ...]
ValidType = Union[*KNOWN_TYPES] # basically a union-like alias
assert ValidType == Union[int, float, str, MyCustomStringType]

Perhaps there's a way of implementing this behavior with TypeGuard or TypeIs, but then again, you need to pass such union type to them like TypeIs[int, float, str, MyCustomStringType].

[median gulch]

Just Ctrl+C Ctrl+V 'd from the library code because the version is exact

[trim tangle]

No, Python doesn't have "type functions" like that

I'm assuming you want this from TypeScript ```ts
type NumericTypes = [bigint, number]
type StringTypes = [string, CustomString]
type KnownTypes = [...NumericTypes, ...StringTypes]
type ValidType = KnownTypes[number] // ValidType = bigint | number | string | CustomString

[kindred barn]

Is there a common way to use yapf and the new type syntax at the same time? I tried disabling yapf around my type definitions with #yapf: disable but it still errors out on parsing. Put my types in a separate dedicated file maybe? edit: yeah that works, but ive decided not to use yapf after all

[median gulch]

No. I want:

class CustomString extends String {}
const NumericTypes = [Boolean, Number] as const
const StringTypes = [String, CustomString] as const
const KnownTypes = [...NumericTypes, ...StringTypes] as const
type ValidType = typeof KnownTypes[number]

[trim tangle]

well, neither is possible 🙂

[harsh lantern]

if i override or implement a method is it good practice to repeat the type hints there as well

[brazen jolt]

yes, definitely

[wooden cipher]

hi
so in my package the set method's key param is type hinted as bytes | str | memoryview
https://github.com/amirreza8002/django-valkey/blob/main/django_valkey/base_client.py#L193

but, since key will go through make_key, you can pass in anything, even a class, or a generator
and it works fine, you can set and get values

but should the type hint be Any? i feel like this information is better untold

>>> class A:
...     pass
... 
>>> cache.set(A, 1)
True
>>> cache.get(A)
1

>>> v.scan()
(0, [b":1:<class 'A'>"])

[sudden kestrel]

anyone using typeguard and have thouts?

thoughts*? Or ideas?

[turbid nebula]

HI

[keen flicker]

I once made a generic for unlikely (or notrecomended) to type-hint this, and force them to either cast or type-ignore if the argument is not of a recomended type, but I haven't seen anything like that anywhere else, I'm also curious about wether if this is good practise or not

[wooden cipher]

since internally everything is turned into a string, it causes the unwanted side effect of allowing everything.

it's not a bug, per say, since you can retrieve the data, but it's weird

and I'm almost sure if i type hint it as Any people would think I'm just lazy and didn't type hint properly

but Any is the most accurate type hint here

but again, should people really know about this?

[trim tangle]

Typing aside, that doesn't seem like a good idea if what you're eventually storing is a string (or bytestring)

[wooden cipher]

I'm not sure what you mean by "that"

[trim tangle]

I'm assuming you're just calling str() on the key if it's not a string?

[wooden cipher]

return "%s:%s:%s" % (key_prefix, version, key)

this part is django's own functionality

[trim tangle]

So if I have ```py
class Foo:
pass

foo = Foo()
cache.set(foo, 100)
``` it's going to use ":1:<Foo at 0x7fa844efb200>" as the key?

[wooden cipher]

yes

[trim tangle]

that seems like a good reason to restrict the input to strings 🙂

[wooden cipher]

and cache.get(foo) will return 100

[trim tangle]

Accepting any object as a key may create false expectations with users. For example:

cache.set({"a": 1, "b": 2}, 100)
cache.get({"b": 2, "a": 1})  # not found...
``` if your storage system only supports strings as keys, expose that constraint

[wooden cipher]

valkey-py accepts int, float, str, bytes, memoryview
I'm not sure if valkey accepts the same, or they turn all keys into bytes

[trim tangle]

I'd restrict the key to str | bytes | memoryview

(assuming int and float don't have special support in redis/valkey)

[wooden cipher]

fair enough
i think i should open a ticket for django as well, since it's their functionality

I'm not sure if they know about this

[trim tangle]

about what?

[wooden cipher]

well the make_key functionality is in django itself
and their cache backend is doing the same as me

so technically you should be able to send anything to django's redis backend

tho i haven't tested that

[trim tangle]

At least here it says:

key should be a str

[wooden cipher]

    def set(self, key, value, timeout=DEFAULT_TIMEOUT, version=None):
        key = self.make_and_validate_key(key, version=version)
        self._cache.set(key, value, self.get_backend_timeout(timeout))

    def set(self, key, value, timeout):
        client = self.get_client(key, write=True)
        value = self._serializer.dumps(value)
        if timeout == 0:
            client.delete(key)
        else:
            client.set(key, value, ex=timeout)

[trim tangle]

it might not validate that it's a string at runtime, but that's pretty typical

[wooden cipher]

so you say i should only type hint it, and not validate?

[trim tangle]

I don't know the specifics of your project, but if it's eventually passed into "%s%s%s" , it makes sense for it to be a str

[wooden cipher]

fair
i ask too many questions because it's a package and has users
i take some caution

appreciate your input 🙏

[trim tangle]

If you do have some parameter that accepts literally any object, it's better to type it as object

Any usually means that the parameter is constrained in some way that's not possible to express in the type system

perhaps a better reason to use object is that it helps you in the implementation, preventing you from doing invalid operations on it

def shout(x: Any) -> None:
    print(x + "!")  # no error reported

def shout(x: object) -> None:
    print(x + "!")  # error: object doesn't support '+'

from the interface point of view accepting an Any argument and an object argument is the same

[molten crystal]

is anyone willing to do a bit of a cr/double check my logic with me, I'm trying to do a typed dataclass and not sure if im going about it the right way

#1320119636162904135 message
Here is a link to my full question explaining everything

[trim tangle]

Should I have the sidg typed as an optional
If all the fields are optional anyway, then it makes sense for the SIDG to always be present, it's just easier to work with (no need for extra checks).
(if sidg=SimpleInstrumentDescriptorGroup(None, None) and sidg=None don't currently have subtly different meanings)

And how can I then "lift" the values out of their option cages after doing some filtering
Depends on the situation. Sometimes you may need to insert an assert or # type: ignore question. Type checkers are not very smart

[molten crystal]

Yeah its that subtly different meaning part that is getting me a bit worked up

maybe im overcooking it

Essentially sometimes the messages have a sidg and sometimes they dont, and within that, when it does exist, some of the values are always there and some arent

[trim tangle]

If some fields of SIDG are always present when it is present, then it's better to make the SIDG value optional, and make those fields inside of it non-optional

[molten crystal]

is it "normal" when working with optional values to have asserts littered around the place? Im not really familiar with production level dynamic typing

[trim tangle]

well, if you have type annotations, it's not dynamic typing 🙂

Having a lot of nullable values sucks, specifically because sometimes you know (or you think you know) that a particular value is present at a particular point in time, but the compile/type checker doesn't know that.

[molten crystal]

true hehe I just meant in python

[trim tangle]

Can you show some code where the type checker complains?

[molten crystal]

well it goes away if I use assert, Im just worried that I have to use assert essentially every time I want to access/work with a value.

[molten crystal]

Maybe its easier to do away with all the float | None for the moment? I'm not actually making use of this functionality at this point in time, I just thought it was a natural thing to do

[trim tangle]

you mean remove the annotations?

[molten crystal]

No sorry I phrased that badly, just remove the optional part.

[trim tangle]

If that value can never be None, then sure

[molten crystal]

hmm I will have a rethink

thanks, its really useful to have someone to discuss it with

like sometimes im not even sure what the problem is until i try and explain it to someone else ygm

[trim tangle]

For example, if you have a set of fields that are always present together or missing together, it's better to group them. Something like this:

# before
@dataclass
class Message:
    id: MessageId
    video_duration: int | None
    video_file: str | None
    text: list[TextEntity] | None
    reply_to_id: MessageId | None
    reply_to_source: ChatId | None
     
# after
@dataclass
class Message:
    id: MessageId
    payload: TextMessage | VideoMessage
    reply_to: ReplyTo | None

@dataclass
class TextMessage:
    text: list[TextEntity]

@dataclass
class VideoMessage:
    duration: int
    file: str

@dataclass
class ReplyTo;
    id: MessageId
    sourec: ChatId | None
    

the first model is shorter, but it's missing a lot of information

If you stick to the first model, you eventually end up with https://discord.com/developers/docs/resources/message#message-object, which has 47 different combinations of which fields are present and which are missing. If you are ever bored, you can try to make a table of which fields are present for each message type

[molten crystal]

glad to see I'm not the first person to run into this haha

Essentially, your #after is what I'm really trying to express! There are multiple submessage types etc. But as I'm just starting I cheaped out a bit and ended up with something closer to the #before

I think I will do it again with placeholders for stuff that hasnt been implemented yet rather than throwing an optional on everything

thanks again!

So, if I had a list of video messages, and I wanted to check the duration of the first one,

l1: list[Message] = [...]
l1[0].payload.duration

I would probably get something like "Type 'TextMessage | VideoMessage' does not have member 'duration' " unless I do an assert

from dataclasses import dataclass

@dataclass
class MessageId:
    id: int = 0

@dataclass
class ChatId:
    id: int = 0

@dataclass
class TextEntity:
    text: str = ""

@dataclass
class TextMessage:
    text: list[TextEntity]

@dataclass
class VideoMessage:
    duration: int = 0
    file: str = ""

@dataclass
class ReplyTo:
    id: MessageId
    source: ChatId | None

@dataclass
class Message:
    id: MessageId = MessageId()
    payload: TextMessage | VideoMessage = VideoMessage()
    reply_to: ReplyTo | None = None

l1 = [Message(), Message(), Message()]

print(l1[0].payload.duration)

I threw in some default values.

The last line, I'm getting:

  Attribute "duration" is unknownPylancereportAttributeAccessIssue
(variable) duration: Unknown | int```

I guess what I really would like to know is, whats an elegant way to cast these structures if we know more about them through logic than the type checker does?

am I supposed to do

pl = l1[0].payload
assert pl is VideoMessage
print(pl.duration)

every time I want to access a nested structure?

[trim tangle]

Well, how do you know that l1[0] is a video message?

[molten crystal]

Hope this is making sense 😅

In my case, I know that all of my messages have a SIDG, because I specifically sent the API a request that can logically only return a list of messages with SIDGs. (They are options contracts). But in the future, if I wanted to look at futures contracts, I would tweak the api query, and get messages with CDGs instead

[trim tangle]

In that case you can encode that information in the type system in some way. For example, the function that makes a futures contract can return an instance of a different class

or you can use a generic:

@dataclass(frozen=True, slots=True)
class InstrumentSnapshot[T]:
    security_id: int
    payload: T
    maturity_date: int | None

def foo() -> InstrumentSnapshot[SIDG]: ...
def bar() -> InstrumentSnapshot[CDG]: ...

[molten crystal]

ooo

whats that?

[trim tangle]

are you familiar with any statically typed languages?

[molten crystal]

I know some cpp, is this kind of like templates?

[trim tangle]

Yep

It's a way to parameterize a class, just like you can give parameters to list or dict

[molten crystal]

looks awesome, I will check it out

[trim tangle]

You can read more here https://typing.readthedocs.io/en/latest/reference/generics.html or here https://decorator-factory.github.io/typing-tips/tutorials/generics/

[molten crystal]

Epic, thanks a bunch

[graceful nova]

how would I type hint a function that returns an object based on an attribute? that the attribute being a class
sample code:

class WrapperA:
  ...

class A:
    default_object = WrapperA
    def wrap(self, func: Callable) -> ???:
      return self.default_object(func)

class WrapperB:
  ...

class B(A):
    default_object = WrapperB

[trim tangle]

If WrapperB is not a subclass of WrapperA, this code should not be accepted by a typechecker, because it's overriding a method in an incompatible way

[graceful nova]

WrapperB is a subclass of WrapperA, I forgot to inculde that in the sample

so wrap(self, func: Callable) -> WrapperA is acceptable?

[trim tangle]

If you're okay with B.wrap being annotated as returning WrapperA

[graceful nova]

I'm fine with that. thanks

better than going into a crazy process

[trim tangle]

If you do want it to return WrapperB, you will need to do a lot of typing gymnastics (playground link)

[graceful nova]

scary link

[graceful nova]

!bookmark

mhm

[graceful nova]

.bm

[iron vapor]

So I know there's a term for this, but I can't remember, and I'd love any help. I'm trying to implement the starlette AuthenticationMiddleware, and it takes a backend of type AuthenticationBackend. My issue is, their Auth backend uses specific classes for user and credentials different than my own. The auth backend I created has the same general signature as theirs, only returning a different user and credential class. Of course, if I try to extend their class, I get an notice about the signatures not matching, and if I try to use my backend instead of extending theirs with the middleware, I get a notice that my class isn't accepted by the auth middleware.
Is there a way to tell the auth middleware to take my values? Or tell it a new signature to take? Or should I just basically copy the code and create my own signature? (that's easy enough in this case, but may not be in others?)

[trim tangle]

If you want to avoid runtime checks and silencing the type checker, you'll need to copy the code

Honestly this middleware isn't that useful, it's just glue code to avoid calling a function in a request handler

Actually wait, can you show the code? What signature in particular causes an issue?

[iron vapor]

The return of AuthenticationBackdend, which returns BasicUser

I have considered if I should just extend my user class from BasicUser.

And yah, in this case, a very basic function to use.

Hell, I can just write the request.user and what not myself.

[trim tangle]

Yeah, starlette has some batteries that might be easier to just write yourself

[iron vapor]

Thanks. The broader question was about overriding typings, using starlette as an example, but I got the answer I needed 🙂

[tame flame]

Hey folks, how do I typehint that a function returns a subclass?

class Base:
  @classmethod
  def parse(cls, s: str) -> ???:
    if s[0] == "A":
      return SubA.parse(s)
    if s[1] == "B":
      return SubB.parse(s)
    return SubAny.parse(s)

class SubA(Base): ...
class SubB(Base): ...
class SubAny(Base): ...

Assuming SubX.parse returns a SubX object

[trim tangle]

def parse(cls, s: str) -> "Base":
``` or if you have `from future import __annotations__`: ```py
def parse(cls, s: str) -> Base:
``` (I'm assuming you're returning an instance of a subclass and not the subclass itself)

[restive rapids]

typing.Self?

[tame flame]

It's not Self, i tried that too 😄

[tame flame]

Wait lemme try that 🤔

Oh huh, so simple

Thanks 😄

[restive rapids]

why is this even.. like that
this seems like a very weird method ngl

[trim tangle]

pathlib.Path.__new__ does exactly this

well, not exactly

[tame flame]

A different thing, mypy is complaining about -> Self when I return MyClass(...) from a function in MyClass. I get it, makes sense. Is it best practice to do self.__class__(...) instead? PyCharm starts crying about unexpected arguments when I do that because its a dataclass.

[trim tangle]

Maybe type(self)(...)?

[tame flame]

That does look better

[trim tangle]

From reading this channel, PyCharm's static analyser sometimes does tell you crazy stories

[tame flame]

Same pycharm warnings but eh 🤔

I'll ignore it then, thanks 😄

[tranquil wharf]

Hi, i justed started using a type checker and i have a question. I use a try except statement in every function so when the function catches an error it doesnt return anything, so the typechecker gives an error for the return type, whats the best practise to solve this? should i just add none to the possible return types or is there another solution?

[trim tangle]

Can you show the code?

Why do you have a try-except in every function?

But otherwise yes, if a function doesn't execute a return statement, it returns None. If a function can return None, you'll need to handle that case in the code.

def get_thing(thing_id: str) -> Thing | None:
    try:
        return get_thing_fallible(thing_id)
    except GettingThingFailed:
        print("oh no")

def print_thing(thing_id: str) -> None:
    thing = get_thing(thing_id)
    print(thing.name)  # bug: the case when `thing` is None is not handled.

[tame flame]

Hey folks, could you take a look at this?

from __future__ import annotations
from typing import TypeVar, Iterator, reveal_type
from typing import Iterable

class Base: ...
class SubA(Base): ...
class SubB(Base): ...

T = TypeVar("T", bound=Base)
X = TypeVar("X", bound=Base)
class Collection(Iterable[T]):
    def __init__(self, items: Iterable[T]) -> None:
        self.items: list[T] = list(items)

    def __iter__(self) -> Iterator[T]:
        return iter(self.items)

    def __add__(self, other: Collection[X]) -> Collection[T | X]:
        return Collection(self.items + other.items)  # mypy error: Unsupported operand types for + ("list[T]" and "list[X]")  [operator]

    def get_a_items(self) -> Collection[SubA]:
        return Collection(item for item in self.items if isinstance(item, SubA))

    def get_b_items(self) -> Collection[SubB]:
        return Collection(item for item in self.items if isinstance(item, SubB))

coll_a = Collection([SubA(), SubA()])
coll_b = Collection([SubB(), SubB()])

reveal_type(coll_a)  # Collection[SubA]
reveal_type(coll_b)  # Collection[SubB]

coll_mix = coll_a + coll_b

reveal_type(coll_mix)  # Collection[Union[SubA, SubB]]
reveal_type(coll_mix.get_a_items())  # Collection[SubA]

So far everything seems to work as intended except that mypy complains about the line where I add the lists together.
Please trust me that I have a reason to make a collection class like this.
Any ideas what the issue could be?

Oh, nevermind, I just replaced it with Collection([*self.items, *other.items]) 👍

[wooden cipher]

replace

from typing import Iterator, Iterable

with

from collection.abc import Iterator, Iterable

that is if you are in python 3.9+

[tame flame]

Ah right, i keep forgetting about that

[trim tangle]

You can use pyupgrade or this ruff rule or flake8-pep585

[tame flame]

I just added a couple hundred ruff rules an hour ago 😄

[tranquil wharf]

This is the code, i am building a chatbot, and to easily find bugs, i put a try statement in every function (most of them), is this bad?

[trim tangle]

Do you have an example of how you're using this method?

[tranquil wharf]

I have an entire hierarchy of classes that represent the dictionaries of a json structure that represents a message in the chat for example, this method i used to covert the dict in the json structure to create an instace of this class

[trim tangle]

Suppose that you call this method in some other function. It returns None. What can you do with that information?

[tranquil wharf]

nothing, it just means that the function ran in an error, otherwise it would return the action class object

[trim tangle]

But what should the caller do in that case?

[tranquil wharf]

just use an empty instance

its not that important because it wont happen normally

[trim tangle]

If an exception in those lines is not expected and indicates some problem, you should just let it propagate up the call stack

[north panther]

what does type hinting mean

[trim tangle]

"type hints" or "type annotations" are optional elements that help static analyzers and editors (and sometimes humans) make deductions about Python code (see https://typing.readthedocs.io/en/latest/ and https://peps.python.org/pep-0484/)

[tranquil wharf]

okay, but should i do that on al levels of the code then? because in some cases i dont want the bot to stop for an possible error

[north panther]

ok thank you but is there a channel that is for pepole to help others make not fix theyre code

[trim tangle]

if you have a general Python question, see #❓｜how-to-get-help and make a post in #1035199133436354600

[north panther]

that is for errors what i meant is help with making the code/writing it

[trim tangle]

No, #1035199133436354600 is for general help. If you don't know how to make something, you can ask there.

[north panther]

oh ok

btw you are a cool person

[trim tangle]

If you're not using some sort of framework and are making it from scratch, make a top-level exception handler. Something like this:

def main_loop():
    while True:
        event = bot.next_event()
        try:
            self.handle_event(event)
        except (SystemExit, KeyboardInterrupt):
            raise
        except BaseException as exc:
            logger.exception(f"Exception occurred while handling event #{event.id} ({event.type})", exc_info=exc)

[tranquil wharf]

Okay, and then i remove all other try statement in all of my code?

[trim tangle]

Generally, if you see a try-except that only logs the error (and ignores it or re-raises it), you can safely remove it.

[tranquil wharf]

Okay thank you for your help

[trim tangle]

Of course, there's some nuance to it. Suppose that you're making a book inventory system, and a warehouse service is sending you events for newly arrived books. At some point you're calling this function:

def parse_book(raw_data):
    author = parse_author(raw_data.get("author"))
    return Book(id=raw_data["id"], author=author, pages=raw_data["pages"], release_year=raw_data["year"])

parse_author() contains some pretty complex logic. Should this function raise an exception if parse_author fails? If so, this code is fine. Hopefully you have some sort of alert system that will send you an email if there's an unhandled exception in the program. Then you'll handle this book manually.

However, it might be important to add all the books on time -- maybe you arrive at work at 13:00 half asleep, but the inventory staff arrive at 6:00 and already need accurate information. You could have a bug in your program, or the warehouse service might have a bug in their program. Or you might have had some miscommunication about the details of the author field. It could be reasonable to do this then:

def parse_book(raw_data):
    try:
        author = parse_author(raw_data.get("author"))
    except ParseError as exc:
        logger.error(f"Failed to parse author for book #{raw_data['id']}", exc_info=exc)
        author = UNKNOWN_AUTHOR
    return Book(id=raw_data["id"], author=author, pages=raw_data["pages"], release_year=raw_data["year"])

It's common to support "partial failure" for more complex systems where there's a critical path and some less-critical branches.

Maybe this talk will be somewhat helpful https://www.youtube.com/watch?v=VvUdvte1V3s

[tranquil wharf]

Thank you very much for your response

[rustic gull]

hello everyone, how do I do this pre python 3.12?

def func[T: int | list[int]](a: T) -> T: ...

I know that I need to use TypeVar, but I am confused about how to make it int | list[int]

[restive rapids]

TypeVar("T", bound = int | list[int])
bounding to a union is pretty weird though. though i guess in the case of a single param -> output its fine, its just that a lot of the times people intend for it to be a constraint (only one of those types)

[pastel egret]

Going back even further,

from typing import Union, TypeVar, List
T = TypeVar("T", bound=Union[int, List[int]])

Constraints act rather weirdly in a lot of cases, it's probably better to avoid them.

[rustic gull]

yeah pylint now thinks T is "object", although return is correctly set to whatever is passed as an input

[rustic gull]

thanks!

if I do this in a class:

class MyClass[T]:
  ...

is that also supported on older python versions?

at the end I do this

def __init__(self, data: T): ...
def __iter__(self) -> Generator[T]: ...

I worry because in vscode my definition of T shows up as unused

[viscid spire]

3.12+

[viscid spire]

you didn't bind T to anything in the second overload

[viscid spire]

don't forget the overload decorator

[trim tangle]

That's a different method

[viscid spire]

oh yes, I was just uh... joking... /nj

totally read it wrong

[rustic gull]

I just found that you can't put class MyClass[T] in < 3.12

so then I guess there is no way to make it work

[trim tangle]

Before 3.12, it's spelled as class MyClass(typing.Generic[T]):

[rustic gull]

oh okay thanks

[viscid spire]

Bro, I said

[queen wraith]

Why when i defind a function with static parameters in python it work with different types too?

[viscid spire]

what do you mean by "static parameters"?

statically-typed args?

[queen wraith]

I mean parameter with static type

[viscid spire]

because python doesn't check them at runtime

we use external tools to check them

[queen wraith]

thanks

[wooden cipher]

hi
i have a base class that has methods like

def close(...):
    raise NotImplementedError

which works as a guide if a user wants to make his own version
but in my own code i inherit from this class for an async client and do async def close(...)
which makes mypy complain

how would i fix that?

note: the base class does not inherit from abc.ABC (at least not yet)

would ABC fix this issue?
or an overload?
or something else?

[brazen jolt]

async vs sync is always an annoying problem, the issue is that if your child class uses async but the parent class uses sync, it's incompatible, and for a good reason:

parent_list: list[Parent] = [Parent(), Child(), Child()] 

for el in parent_list:
    el.close()  # this should work, as Parent.close is sync, but doesn't for Child classes

the only real solution is to have 2 base classes, one being sync, another being async

or just drop the sync support entirely

one hacky way to solve it could be to make close return a union type of the return type T and Awaitable[T]:

from typing import override
from typing_extensions import Awaitable


class Base:
    def close(self) -> int | Awaitable[int]:
        raise NotImplementedError


class A(Base):
    @override
    def close(self) -> int:
        return 42


class B(Base):
    @override
    async def close(self) -> int:
        return 42

but know that if you expect people to work with the Base type at all, like having list[Base], it will be pretty annoying for the caller to deal with calling close

unless you really need it, I'd recommend just dropping support for synchronous alternatives

oh and btw, yes, the base class should inherit from ABC, as it has some methods that aren't callable and are expected to be overridden, these should be marked as abstract methods. The ABC class will also do some magic to prevent you from accidentally instantiating the base class itself, only allowing that for classes which override all of the abstract methods

[grave fjord]

It should be aclose for async

So it can be used with contextlib.aclosing and anyio.aclose_forcefully

[jolly cipher]

question regarding variance.

in both mypy and pyright, the assignment to a is intuitively caught as erroneous, but the second isn't:

a: list[float] = list[bool]()
b: list[float] = [True, False]

mypy:

file.py:1: error: Incompatible types in assignment (expression has type "list[bool]", variable has type "list[float]")  [assignment]
file.py:1: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
file.py:1: note: Consider using "Sequence" instead, which is covariant
Found 1 error in 1 file (checked 1 source file)

pyright:

/file.py
  /file.py:1:18 - error: Type "list[bool]" is not assignable to declared type "list[float]"
    "list[bool]" is not assignable to "list[float]"
      Type parameter "_T@list" is invariant, but "bool" is not the same as "float"
      Consider switching from "list" to "Sequence" which is covariant (reportAssignmentType)

why?
the list [True, False] contains only bools, and list[bool] is not a subtype of list[float] obviously (per the error for the first statement).

why does the first assignment take variance into account, while the other does not?

another example (again, happens for both type checkers):

def foo(spam: list[float]) -> None:
    pass


foo([True, False])  # no error

arg = [True, False]
foo(arg)  # does report an error about covariance

[jolly cipher]

i usually make _async-suffixed (you get the idea) coroutine functions
sync and async functions are hence always separate and more maintainable

unions with Awaitables are most often a bad idea

[jolly cipher]

i bet this has something to do with how inference works in different "stages" here, and i'm pretty sure that it's by design, i just don't know why.

OK, now at least I know how it's called. it's "bidirectional type inference".

i'm still pretty concerned though why bidirection inference ignores variance. how on earth is var4: list[float] = [4] correct?

hmmm, okay. i think i'm starting to get it. mypy also has a writeup on that. it seems like we're in the power to manipulate type checkers into seeing supertypes of things where they aren't present, but we want to pretend that they are.

and it's covariant in nature, hence supertypes. i concluded that from _: list[bool] = [1.0] not allowed.

i of course started to think about an inverted case. what if we had a structure that implied contravariance, and put that into the test of bidirectional inference.
if i understand correctly, bidirectional inference would let us silently ignore the unmet contravariance requirement.

my first thought was assigning a value of actual type Callable[[int, int], int] to Callable[[float, float], float].

Callable's paramspec arguments are contravariant, so an expression of type Callable[[float, float], float] cannot be assigned a Callable[[int, int], int]. it could be assigned a Callable[[object, object], bool] for instance, but not the other way around (the other way around would be for example Callable[[bool, bool], object]).
bidirectional inference should let us ignore it.

so for the following case, I'd expect no errors by type checkers.

we assign a value of actual type Callable[[int, int], int] to Callable[[float, float], float] "expected".
just as when assigning a value of actual type list[int] to list[float] "expected" worked.

from collections.abc import Callable


_: Callable[[float, float], float] = lambda a=1, b=2: a + b

mypy fails here with error: Cannot infer type of lambda [misc], and pyright reports no errors.

with no bidirectional inference, we expect an error.

from collections.abc import Callable


no_bidirectional = lambda a=1, b=2: a + b
_: Callable[[float, float], float] = no_bidirectional

mypy inferred def (a: Any =, b: Any =) -> Any so it's out of the game.
pyright:

  file.py:5:38 - error: Type "(a: int = 1, b: int = 2) -> int" is not assignable to declared type "(float, float) -> float"
    Type "(a: int = 1, b: int = 2) -> int" is not assignable to type "(float, float) -> float"
      Parameter 1: type "float" is incompatible with type "int"
        "float" is not assignable to "int"
      Parameter 2: type "float" is incompatible with type "int"
        "float" is not assignable to "int" (reportAssignmentType)

as expected. 🚀

and now again with bidirectional inference: since list[float] "expected" assigned a list[int] "actual" worked, but list[bool] "expected" assigned a list[int] "actual" not, I think I actually expect a lambda lambda a=object(), b=object(): 0.0...

assignable? not assignable?

to Callable[[float, float], float].
can't tell.

[jolly cipher]

intuitively, i'd let it be assignable, because it makes sense from the variance POV. but that means we can completely ignore variance of param spec when assigning callables with bidirectional inference!

yep. pyright allows that:

from collections.abc import Callable

_: Callable[[float, float], float] = lambda a=object(), b=object(): 0.0

from collections.abc import Callable

no_bidirectional_correct = lambda a=object(), b=object(): 0.0
_: Callable[[float, float], float] = no_bidirectional_correct

(both cases pass)

pretty interesting stuff. the conclusion is that with bidirectional inference, you are able to extend invariance into covariance and contravariance into contravariance+covariance (no variance).

lmk if you have any thoughts on this overly long, yet pretty short analysis.

[wooden cipher]

that was one example, there are other methods too

aclose is in there
i have close = aclose or vice versa

the problem is that there are a few related libraries that i have to stay consistent with/close to consistent, each with their own conventions

[trim tangle]

The reason the type checker is complaining is that it expects a subclass to have an interface that's compatible with the superclass. For example:

class SuperClass:
    def close(self):
        ...

class SubClass(SuperClass):
    async def close(self):
        ...

def do_something_and_close(sup: SuperClass):
    sup.close()
    assert sup.close() is None  # should always work

do_something_and_close(SubClass())

[grave fjord]

Your approach doesn't work with typing

You have to use a different design

Or have an is_async flag that's a generic parameter, and you can use overrides with self types

[pastel egret]

The way I think of this working is it's like mylist: list[float] = [(float)4, (float)true, (float)3.14] - we're not casting the list, we're casting every element individually.

[wooden cipher]

huh
there are hard choices to be made then
thanks for your time🙏

[pastel egret]

Having an async method in the subclass that overrides a sync seems dangerous anyway - you cannot ever use that in code expecting the superclass, since if it tries calling it it'll just never await.

[trim tangle]

Yeah, this is the same as having a list as an argument. For example, if you have a function with the signature py def add_colors(colors: list[Literal["red", "green", "blue", "white", "black"]]) -> None a type checker should correctly accept ```py
add_colors(["red", "blue", "blue", "white"])

There often isn't a single "correct" type to assign to an object, it depends on the context. For example, typescript has an as const operator that can influence the inference:

const colors = ["red", "blue", "blue", "white"];  // inferred as `string[]`
const colors = ["red", "blue", "blue", "white"] as const;  // inferred as `readonly ["red", "blue", "blue", "white"]`

[cinder bone]

Is PEP 728 going to be accepted, or is there still discussion going on?

it would be a nice to have thing for me currently lol

[jolly cipher]

please ping on reply, thx!

[vivid ore]

Is there a type I can use that’s like tuple in that it’s heterogenous and has a specific length but also like list in that it’s mutable?

[viscid spire]

perhaps array.array. wait no, that's homogenous

[vivid ore]

can I specify the length?

[trim tangle]

Right now the support for heterogeneous sequences is pretty bad

You can declare a generic type with several parameters using ParamSpec:

class Muple[*Ts]:
    def __init__(self, values: *Ts) -> None:
        self._base: tuple[*Ts] = values

    def starmap_me[R](self, fn: Callable[[*Ts], R]) -> R:
        return fn(*self._base)
``` but you won't be able to e.g. add a `__getitem__` method that says "given literal index `I`, return item of type `Ts[I]`"

[viscid spire]

heterogenous you mean?

[trim tangle]

yeah sorry

[vivid ore]

People are still confused about this :(

WHY THE FUCK IS slice NOT GENERIC

[soft matrix]

it is

🤔

[viscid spire]

!e slice[int]

[rough sluice]

:x: Your 3.12 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 1, in <module>
003 |     slice[int]
004 |     ~~~~~^^^^^
005 | TypeError: type 'slice' is not subscriptable

[viscid spire]

nope

[soft matrix]

oh did the runtime changes never happen

open a pr for it

it should get accepted cause it is at type time

!d slice

[rough sluice]

slice

class slice(stop)``````py

class slice(start, stop, step=None)```
Return a [slice](https://docs.python.org/3/glossary.html#term-slice) object representing the set of indices specified by `range(start, stop, step)`. The *start* and *step* arguments default to `None`.

Similar names: django.slice, term.slice

[tranquil ledge]

Ah. I’d asked in the typeshed PR for TypeVar defaults with generic slice if it needs a corresponding runtime change, then totally forgot about it when no one responded. Whoops.

[vivid ore]

Mypy says it’s not

Pyright says it is

[soft matrix]

you should maybe update your mypy version?

or it hasnt shipped in typeshed yet cause it breaks too much

[soft matrix]

i have pred

https://github.com/python/cpython/pull/128336

[tranquil ledge]

Thank you.

[oblique urchin]

I'd say we'll probably get some version of it but there's been disagreement over how it should be spelled. I need to spend some more time pushing it through.

[grave fjord]

What happened with types.CoroutineType and types.GeneratorType Generics?

[soft matrix]

they should be now

yeah merged

[grave fjord]

Nice, now just the mypy changes left

[hasty phoenix]

Is there a way to declare that the arguments of fn(cls=type[T], *args, **kwargs) -> T are in reality arguments for initializing T? I.e. fn calls cls(*args, **kwargs). It would be wonderful if it were possible to refer *args and **kwargs to match another function. I'm not sure the ParamSpec, P.args and P.kwargs is able to be used on an objects __init__() is it?

[soft matrix]

could copy the signature of the .__init__ using a deco

[hasty phoenix]

How can that be done?

[soft matrix]

https://github.com/python/typing/issues/769#issuecomment-903760354

youll need to slightly adapt it to remove self though

[hasty phoenix]

Type checkers complain if the signature of an inherited method has changed. Are there ways to specify in the base method that a specific methods signature is intended to be changed?

Type checkers use Liskov-principle, which is generally a good thing. But as with everything in Python wrt. typing, there are some cases where things are done otherwise.

[tacit sparrow]

Is there a way to accomplish this?

from typing import TypeVar

class A: ...
class B(A): ...
class C(A): ...

T = TypeVar("T")
# Cast that would work both statically and at runtime,
# just a shortcut for `assert isinstance(x, type)`
# that can be used inline.
def safe_cast(typ: type[T], value: T) -> T:
    assert isinstance(value, typ)
    return value

def get_element() -> A: ...

element = get_element()
# Should return B, currently returns A.
new = safe_cast(B, element)
# Should fail at static time as B is not related to value's type.
new = safe_cast(B, "5")

[jolly cipher]

i think i'd accept any type as the argument for typ (consider forward references and similar) and i as well would type-hint the value parameter as Any (or, actually, object).

then, the implementation of safe_cast would be

something similar to

def safe_cast[T](typ: Any, value: object) -> T:
    assert isinstance(value, evaluate_forwardref(typ) if isinstance(typ, ForwardRef | str) else typ)
    return cast(T, value)

just figure out a way to evaluate strings/forwardrefs if you want full cast() compatibility

also, remember that typing reality and runtime reality may differ, so the safe_cast function as a concept is flawed.

consider the following example: safe_cast(float, 1)

this will fail at runtime but succeed at typing time (the reason is that float is not a base of int at runtime, while by type checkers int is considered a subtype of float).

remember that "type" is a much more broader term than "class".

types are concepts, include various special forms and conventions, classes are real runtime objects who just so happen to be valid types if created conventionally (with the class statement).

you can learn more @ https://peps.python.org/pep-0483/#types-vs-classes

some technical texts tend to mix those up, but it's pretty useful to stick to the actual definitions.

cc @oblique urchin @lunar dune

[tacit sparrow]

but this signature would return Unknown type as T is not used

[jolly cipher]

can you provide a mypy/pyright report supporting that?

this function is generic.

you can still use T = TypeVar("T") on lower python versions, where you can't really use the def fun[T](...) syntax

but what i wrote can't really run into this trouble

[tacit sparrow]

https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoCCKcANFAIYDOlApngPrwI0CwAUOwCY3BSXnAa9AMZUYAbQAqAXQAUTAFyFiZAG7kANgFcaSsACMAVjWEwAlFAC0APiiSlAOift2VWgyY1Z-QSLGzgDTByGDIARgcABjMyQODzIA

[jolly cipher]

oh, OK. that must be my knowledge gap then

maybe try type[T] | Any as the hint of typ?

how would that work?

i'd check what is accepted by cast

i.e. the signature of cast

nooo!

i was correct

you are not using the function i gave you

T is used inside the function

[tacit sparrow]

I don't think it's supported

[jolly cipher]

ok, true, i wasn't correct

type[T] | Any may solve the problem

https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoCCKcANFAIYDOlApngPrwI1kDGVMAsAFA8AmNYFErlgNeu0owA2gBUAugAomALliIac%2BVAA%2BhYmQBu5ADYBXGmrAAjAFY1WMAJRQAtAD4oslTyh%2BK1HT4SJSoUuQorDSKxuYs6ghOvv4gNDBmIChQkjCKskamFkm83FS0DEzRImISHIrAJmDkMGQAjAB0AAxOZA1NzkA

[tacit sparrow]

class A: ...
class B(A): ...
class C(A): ...

reveal_type(safe_cast(float, 1.0)) # float, good.
reveal_type(safe_cast(B, A())) # B, good.
# Returns B without issues, need to detect it as an error.
reveal_type(safe_cast(B, dict))

basically it's the same as func(type: T, ...) -> T

[jolly cipher]

ok, you can achieve this, but it won't support as much cases as cast

ohh i see now

i think i got confused quickly, sorry

[jolly cipher]

what's wrong with my implementation's behavior exactly?

does safe_cast really cast anything?

or is it more of a TypeIs problem

i think i tried to give you a reimplementation of cast of some sorts, i misunderstood your goal

[jolly cipher]

i think i don't understand your use case. it may make sense to want to validate the cast type is the actual runtime type, and the present assertion does that. if you expect something else to happen, please give some more examples

[jolly cipher]

i don't see how safe_cast(B, dict) would return B here. it could return the dict class, but it can't due to assertion failure (unless __debug__ is false).

[jolly cipher]

honestly, it would be the best if generic functions accepted the type T via __getitem__ and you could make an optional runtime type parameter (validate against the type argument, or, if the typ parameter is present, validate against it). it would allow you to dodge typing vs runtime problem and support all cases.

but type parameters in generic functions can only be inferred, which i happen to forget about

i think https://github.com/bswck/runtime_generics could be used to make a class that intercepts the type argument to validate against.

although runtime generics is pretty far behind the latest typing intricate implementation details (pep 749)

i think i should limit the functionality to not support runtime type checking at all.

handling variance and MRO at runtime for some basic engineering tasks is really a waste of memory, CPU cycles, and my time.

[tacit sparrow]

Thank you for your patience 😅
I'll try to describe it more clearly. See example below.
Currently I meet the same case where I need to narrow down the type like in current_way - I have a base type A and I need to narrow it down to B.
Simple safe_cast implementation helps simplifying this to just 1 line - quicker_way.
What I'm looking for for safe_cast to also guard me from passing wrong value type by accident. E.g. in static_guard_from_casting_wrong_type get_element() type is B which is not superclass of float so this cast will always fail at runtime and would be nice if it can also fail at static time.

from typing import TypeVar, Any, cast

class A: ...
class B(A): ...
class C(A): ...

T = TypeVar("T")
def safe_cast(typ: type[T], value: Any) -> T: ...

def get_element() -> A: ...
def expecting_b(element: B) -> None: ...
def expecting_float(f: float) -> None: ...

def current_way():
    element = get_element()
    assert isinstance(element, B)
    expecting_b(element)

def quicker_way():
    expecting_b(safe_cast(B, get_element()))


def static_guard_from_casting_wrong_type():
    # Currently there's no errors but would be nice
    # if it can somehow guard from casting wrong types
    # and throw a typing error.
    expecting_float(safe_cast(float, get_element()))

[weak oriole]

Think I’m going crazy. I can’t convert an object colum to string and my pydantic validation is failing bc of it.

[harsh lantern]

is there a version of @Override in python

[restive rapids]

!d typing.override
?

[rough sluice]

typing.override

@typing.override```
Decorator to indicate that a method in a subclass is intended to override a method or attribute in a superclass.

Type checkers should emit an error if a method decorated with `@override` does not, in fact, override anything. This helps prevent bugs that may occur when a base class is changed without an equivalent change to a child class.

For example...

[harsh lantern]

👍

[harsh lantern]

this is not meant to be a constant
how do i prevent this without disabling the warning

[viscid spire]

That's what Final means...

[harsh lantern]

ok...

[lunar dune]

That appears to be a pylint complaint rather than a complaint from a type checker

[rare scarab]

What's the context of this? If you want the field to be readonly, wrap it in a @property.

  _this: "Instance | None"
  @property
  def this(self) -> "Instance | None":
    return self._this

[rain crow]

s

hello there, just want to ask how to solve my problem with pydantic i am using Generic TypeVar to make a Generic Data Fields response but it is not validating the json response from http requests ```py
class BaseRecord(BaseModel, Generic[T]):
record_id: str
fields: T

class ListRecordBodyResponse(BaseModel, Generic[T]):
class DataField(BaseModel, Generic[T]):
has_more: bool
page_token: Optional[str] = None
total: int
items: List[BaseRecord[T]]

code: int
msg: str
data: Optional[DataField] = None

class ExampleRecord(BaseModel):
record_id: list[dict]
field_agent: BaseFields.PersonField = Field(alias="Field Agent")
latest_location: BaseFields.LocationField = Field(alias="Latest Location")
last_updated_at: BaseFields.DateField = Field(alias="Last Updated At")

async def list_records(
self,
app_table: str,
record_model: Type[T],
page_token: str | None = None,
page_size: int = 500,
filter: str = None,
user_id_type: Literal["open_id", "union_id"] = "union_id",
) -> ListRecordBodyResponse[T]

[trim tangle]

Optional[DataField] implicitly means Optional[DataField[Any]]

In almost all cases you should avoid nesting classes, so instead do ```py
class DataField(BaseModel, Generic[T]):
has_more: bool
page_token: str | None = None
total: int
items: list[BaseRecord[T]]

class ListRecordBodyResponse(BaseModel, Generic[T]):
code: int
msg: str
data: DataField[T] | None = None

[rain crow]

still does not work it is still using the "Field Agent" when instantiation instead of field_agent

[trim tangle]

What do you mean by it not working?

[rain crow]

it still does not deserialize to the ExampleRecord with field alias even tho populate_by_name is True

[trim tangle]

Do you have some example data and the error that occurs?

[rain crow]

wait

ListRecordBodyResponse(code=0, msg='success', data=DataField(has_more=False, page_token=None, total=8, items=[BaseRecord(record_id='recuyM06qprFpo', fields={'Field Agent': [{'avatar_url': 'https://_________/static-resource/v1/&cut_type=default-face&quality=&format=jpeg&sticker_format=.webp', 'email': '[e-mail]', 'en_name': 'John Doe', 'id': 'on_0021290c15f126423b3e5c406e719b7b', 'name': 'John Doe'}], 'Last Updated At': 1735897603000, 'Latest Location': {'address': '', 'adname': 'Parañaque', 'cityname': 'Manila', 'full_address': 'Philippines', 'location': '121.98693397391317,15.522780856556325', 'name': '', 'pname': 'Metro Manila'}, 'record_id': [{'text': 'recuyM06qprFpo', 'type': 'text'}]})

fields={'Field Agent': [{'avatarurl': 'https://________//static-resource/v1/&cut_type=default-face&quality=&format=jpeg&sticker_format=.webp', 'email': '[e-mail]', 'en_name': 'John Doe', 'id': 'on_0021290c15f126423b3e5c406e719b7b', 'name': 'John Doe'}], 'Last Updated At': 1735897603000, 'Latest Location': {'address': '', 'adname': 'Parañaque', 'cityname': 'Manila', 'full_address': 'Philippines', 'location': '121.98693397391317,15.522780856556325', 'name': '', 'pname': 'Metro Manila'}, 'record_id': [{'text': 'recuyM06qprFpo', 'type': 'text'}

this part is the problem

it does not use the field name it still uses the field name from the http response

[trail kraken]

Is it unusual to have a parameter purely for type hinting?

def forward(send_type: type[_T]) -> Generator[None, _T, None]:
    while True:
        t = yield None
        # Do things with t.

[restive rapids]

it is "unusual" for most code, but phantom types are a thing. sadly functions cant be parametrized with [] like classes can, so passing a type argument is what you have to do
though, what is unusual is that you cant really do anything with the t in this case besides generic object operations, unless _T is a bound typevar, and in cases where you'd actually be able to do something to it - there would be other things that could bind the typevar, so the type argument could be omitted

[trail kraken]

It is a bounded type for my use, yea. I had wondered about alternatives. Thank you for essentially answering that too.

[crisp steppe]

Why do people keep using Generator[None, _T, None] instead of much nicer Iterator[_T]

I see it all over the place. 😮

[cinder bone]

In this case they aren't equivalent

but people sometimes use Generator[T, None, None] because they like to save the info that it's a Generator specifically

[rare scarab]

Because that is wrong. Generator[None, _T, None] is a generator that has a yield result

def foo() -> Generator[None, _T, None]:
  x = yield
  reveal_type(x) # type of x is _T

Generator[TYield, TSend, TReturn]

[rare scarab]

^

class Generator[TYield, TSend, TReturn](Protocol):
  def __iter__(self) -> Generator[TYield, TSend, TReturn]: ...
  def __next__(self) -> TYield: ...
  def send(self, value: TSend | None = None) -> TYield: ...
  def throw(self, exc: BaseException) -> None: ...

[crisp steppe]

IME they just do what the ide suggests.

[crisp steppe]

Ok! I got the type parameters incorrect

[rare scarab]

fyi TReturn is the result of yield from

[viscid spire]

the return type is the type of the arg for the StopIteration exception that is raised at the end of a generator (which can be done sooner/manually using return)

!e

from collections.abc import Generator
def g() -> Generator[None, None, int]:
    if False:
        yield
    return 123

try:
    next(g())
except StopIteration as e:
    print(e.value)

[rough sluice]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

123

[viscid spire]

it's literally the type for returning, which is why it's called the ReturnType in the docs

[rare scarab]

!e ```py
def foo():
return "Hello"
yield

def bar():
yield "world"
x = yield from foo()
print(x)

for y in bar():
print(y)

[rough sluice]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

001 | world
002 | Hello

[rare scarab]

yield from is what was used before await was added.

This is the only place the third generic in Generator matters, because StopIteration.value is Any

[green bluff]

hm, this is probably an appropriate place to ask. why does the type A = B form of alias break pydantic and tools like it when they work just fine with A: TypeAlias = B?

[brazen jolt]

A: TypeAlias = B makes A equal to B on runtime, while type A = B, doesn't, the A here becomes a TypeAliasType instance

you can access the underlying type from the type alias, but many libraries just don't support this yet

[green bluff]

hm, weird decision to have typing.TypeAlias right there but instead it converts to TypeAliasType, meanwhile it's all lies because it's actually of type B in the first place??

I was absent from python dev when typing.Annotated took over, and I'm still not quite sure why it was introduced

[brazen jolt]

sometimes, libraries use type-hints for stuff on rutime, like fastapi does for example, but type-checkers often diagree with those types, annotated allows you to specify both

[green bluff]

so it's a lie to the type system to make editor services work

I wish I could stop writing Python for my dayjob sometimes.

[brazen jolt]

no types are enforced in python on runtime, it's just type "hints", it's all lies, you don't need to follow them anywhere

here's an example I found in some random code that I wrote

this is a pydantic basemodel class, which uses StringConstrants as the object that you're supposed to pass in for the annotation, that pydantic then picks up on

but obviously, that makes no sense as an actual type for that attribute

so it's wrapped in annotated with the first attribute being the type for the type-checker, and the second one that libraries can get and work with dynamically

in many cases, it might be sufficient to handle this with default values, like with dataclasses.field for example

but sometimes, that's not possible due to some design choices

so libs that rely on annotations but aren't complatible with type-checkers often add support for Annotated, so that both can be happy

[trim tangle]

No, it's a tool to allow you to include metadata alongside a type annotation

See the Pydantic example above^

[trim tangle]

In this case it should be possible to do this with the pydantic.Field descriptor, no?

[brazen jolt]

yeah, that's why I mentioned it, but in some cases it's not, just didn't have a good example for that to quickly pull up

[trim tangle]

Pydantic actually has some cases where the annotated is required more or less https://docs.pydantic.dev/latest/concepts/fields/#the-annotated-pattern

like if you had list[Annotated[str, StringConstraints(...)]]

[green bluff]

nested generics, such a headache

[trim tangle]

I thought you liked Rust 🙂

[green bluff]

I do. It's not as nice in Python.

Particularly the second position being a function call

I really dislike dunders / magic methods and lack of traits (this applies to many more languages than Python, though)

constraining types is painful

my mental model is Annotated says "hey type system, we're actually a T, and here's a callable to pass incoming values through"

[trim tangle]

Annotated doesn't do anything on its own

[brazen jolt]

yeah, it's just for libraries that do type hint inspection

[green bluff]

so it's up to the framework to pull out __annotations__ and do stuff?

[brazen jolt]

yep

[green bluff]

feels like such a hack

[brazen jolt]

it is

it was a way to get both the type checking community and the rest of the people who didn't care as much and saw type-hints as a cool extra syntax for python that they could use to make their libraries fancier

[trim tangle]

I mean, Python is a dynamically typed language with some static analysis bolted on top. It is going to feel bolted on

[green bluff]

I wish __future__.annotations could've become default. I remember the fallout of them trying to move forward with that and suddenly breaking fastapi/pydantic for everyone

figuring out cyclic dependencies would also be nice...

[trim tangle]

PEP 649 is going to become the default soon, it's like __future__.annotations but better

(allegedly)

[green bluff]

I'll check it out later

[brazen jolt]

when is soon?

3.14?

[green bluff]

yeah, scheduled for 3.14 this year

[trim tangle]

Of course, if you don't like dynamic typing, you're probably not going to enjoy using Python. Not much you can do here

[green bluff]

it's been my dayjob for 8 years now, so I'm kind of stuck with it until Rust takes off more broadly commercially. although my workplace had full RTO leaked recently so that may be this year for me

the status quo has certainly improved over untyped flask code when I started to fastapi+pydantic of today, and sqlalchemy getting on board with types in 2.x 😅

[trim tangle]

My grievances with Python is less that it is not Rust, and more that it is not TypeScript, if you know what I mean

the type system is just not expressive enough

[brazen jolt]

(looking at you HKT)

[trim tangle]

I personally don't care about HKT much, but stuff like: TypeVarTuple being introduced and still being extremely niche, because you can't do transformations on TypeVarTuples

[brazen jolt]

yeah, that is true, the current state paramspec is pretty sad

[trim tangle]

I'm actually not sure what purpose ParamSpec serves

Wait no, I meant TypeVarTuple, I confused the two

ParamSpec is similar, but it does have some uses #1248451661349785610 message

[brazen jolt]

yeah, I mean paramspec is somewhat useful, it does allow better typed decorators, but it's far from perfect

typevartuple is something I don't think I ever actually used outside of experimenting with it for the sake of learning about it

[trim tangle]

TypeVarTuple does make some things possible to express, like this: #type-hinting message
But the resulting interface is really awkward, and not something you'd ever do if you had an untyped codebase

[brazen jolt]

that's an interesting use-case

but yeah, I don't think I ever saw any actual need for it in practice

[restive rapids]

with hkts something like

def generic_iterable_map[I: Iterable, A, B](xs: I[A], f: Callable[[A], B]) -> I[B]:
  return type(xs)(f(x) for x in xs)

could finally be a thing (though to make it safe I should be more constrained, it should have a constructor that takes an iterable)
and a generic functor/monad interface
though for most code i guess thats indeed not as useful as e.g. being able to map some type over a typevartuple, or, well.. record types

[trim tangle]

In TypeScript you'd have something like ```ts
type ComputeTuple<Fns extends unknown[]> =
Fns extends [(: string) => infer Item, ...infer Rest]
? [Item, ..._ComputeTuple<Rest>]
: []

export function parseThings<Fns extends ((_: string) => unknown)[]>(s: string, ...parsers: Fns): _ComputeTuple<Fns> {
// ...
}
``` then you do parseThings(myString, parsesFoo, parsesBar, parsesBaz) and you get back a [Foo, Bar, Baz] - the runtime interface is simple, and you wouldn't do it differently in JavaScript

(maybe it's not the best way to do it in TS, but it works as an example)

[restive rapids]

honestly i dislike this, thats basically a separate language for typing at this point
it would be nice to be able to operate on types as normal python values

[brazen jolt]

it's not something you'd use too often

but it's definitely useful and it's nice to be able to have that power

[trim tangle]

Yeah, if you're doing some advanced stuff in TypeScript, you're basically programming in Lisp (at the type level). But, well, it's deliberately simpler to statically analyze

pyanalyze has something like you're suggesting https://github.com/quora/pyanalyze?tab=readme-ov-file#extending-pyanalyze

[green bluff]

hkts in Python are at least able to be emulated, but falls into framework territory

too bad they didn't annotate this decorator correctly

[trim tangle]

Kwargs are named that way because that's the sound you make when you see **kwargs: Any in a function

Actually, you can now use typing.Unpack with typing.TypedDict, but perhaps it's not always possible

[heady dew]

Does anyone here like / use https://github.com/orsinium-labs/mypy-baseline ?

[jolly cipher]

would you recommend type annotating tests?

i personally think that it could help transitively find wrong type annotations.

[brazen jolt]

I don't enforce annotations on tests/ personally, like I do with the rest of my code, but I do leave type-checking enabled there, usually with a reduced rule-set though, so a bunch of things that would be enforced in my fairly strict default config aren't checked for tests/ dir. I do often add type-hints, where relevant.

[jolly cipher]

sounds like a reasonable tradeoff.

[cinder bone]

I usually don't, just because I use pytest with fixtures and it's irritating to type annotate those and doesn't have as much return on investment

[grave fjord]

For libraries I type hint everything strictly because in most cases the only place a type is used is in the tests and I want to see it work

[oblique urchin]

my experience is that type checking regularly flags issues in tests where the test isn't testing what it thinks it's testing

[stable fjord]

many a times I've found issues in typing from trying to type tests. If you're having a hard time getting your tests to pass type checking then users of your library/etc are probably having similar issues

[foggy garnet]

Any ideas for banishing the Any types here? I can't see how to bind to P.args or P.kwargs in the return types.

R = TypeVar('R')
P = ParamSpec('P')
def _returns_args(func: Callable[P, R]) -> Callable[P, tuple[R, Any, Any]]:
    """
    Returns a function that returns the result of calling ``func``, along with the args and kwargs.
    """
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> tuple[R, Any, Any]:
        return func(*args, **kwargs), args, kwargs
    return wrapper

[brazen jolt]

for pure args, you could do this: ```python
from typing import TypeVar, TypeVarTuple, reveal_type
from collections.abc import Callable

R = TypeVar("R")
Ts = TypeVarTuple("Ts")

def _returns_args(func: Callable[[*Ts], R]) -> Callable[[*Ts], tuple[R, tuple[*Ts]]]:
"""
Returns a function that returns the result of calling func, along with the args and kwargs.
"""

def wrapper(*args: *Ts) -> tuple[R, tuple[*Ts]]:
    return func(*args), args

return wrapper

def test(x: int, y: int, /):
return x + y

ret, args = _returns_args(test)(1, 6)
reveal_type(ret)
reveal_type(args)

but I don't think you can preserve the keyword args too

[foggy garnet]

thanks!

[crisp steppe]

I thought I would be clever and use KW = TypeVar('KW', bound=TypedDict, covariant=True) but mypy doesn't want a type to be bound over TypedDict as it's a type alias and not an actual type(?) So I tried bound=_TypedDictMeta but that also didn't work. 😮‍💨

[foggy garnet]

i'm not sure the function has a future anyway (haha) -- It doesn't do the right thing for async functions

the context is that I wanted a way to use asyncio.as_completed where the awaited values include their args, becase as_completed returns stuff out of order

since there's only 1 function I need this for, I'm wrapping it manually

[brazen jolt]

yeah, afaik there's no way to achive this with the current typing system's capabilities, kwargs were always a pain to work with in decorators, unless it's just something as simple as "this func has the same kwargs"

[young zealot]

I have a Generic class with the generic parameter P being a ParamSpec, and the init takes a Callable[P, None], is there a way to make it so it accepts a callable with no parameters?

[brazen jolt]

So, what you basically want is a way to write a type annotation for your generic type that would express an empty ParamSpec? e.g. something like MyGeneric[()]? AFAIK, this isn't possible.

[young zealot]

Yeah I tried MyGeneric[None] and MyGeneric[()] but no luck. too bad. My "workaround" is to use MyGeneric[None] and the callable I pass in is def my_callable(_: None = None) -> None: and it "works", just not the cleanest

[brazen jolt]

yeah, the current specification for ParamSpec is very limited and lacks a lot of things, sadly, this part of the typing system just isn't quite great yet

huh

this apparently works

in basedpyright at least

I don't think it's officially a part of the spec though

[brazen jolt]

yeah, passes pure pyright and mypy too

[brazen jolt]

it actually is, lol, I had no idea, just tried this randomly

@young zealot

[trim tangle]

That's a really weird feature... in part because you can't specify the kwargs. Kinda half-baked

but I guess it's better than nothing

[brazen jolt]

yeah, that's also why I thought it wasn't a thing at all, when I just randomly changed the () to [] and it worked, it kinda blew my mind lol

it's, definitely better than nothing, but I'd love to see some more support for keyword args as well, which, I've no idea how they would do syntax-wise now if this is a thing

[trim tangle]

Foo[Params(int, str, foo=str, bar=int)]
Foo[Params(int, *MyUnpackable, foo=str, **MyTypedDict)]

type shit

[brazen jolt]

yeah, but the list-like pos-only syntax will now likely forever remain a thing

[trim tangle]

||that's what happens when you add a feature first and design it later||

honestly that wouldn't be a big deal, the type system is evolving and leaves some old husks behind

like Union, TypeVar and all the PEP585 stuff

[young zealot]

interesting, not sure why I didn't think of that, but it seems to be working in my more complicated case too. perfect, thank you!

[ocean marten]

is there any way to define the following case, in the Child.__init__ i wan't describe that later this Child class is gonna be subclassed and have access to all attrs/instance methods of Family

class Child:
    def __init__(self) -> None:
        print(f"I am child of {self.father_name}")


class Family(Child):
    def __init__(self):
        self.father_name = "Alice"
        super().__init__()


if __name__ == "__main__":
    Family()

[grave geyser]

Is this really how you get access to the hashlib Hash type: ?

if TYPE_CHECKING:
    from hashlib import _Hash  # type: ignore

def use_hash(hash: _Hash):
    ...

(with type: ignore because _Hash is private)

Does not feel future proof

[pastel egret]

Define an @abc.abstractmethod and @property for father_name in Child. Type checkers will then require that to be overridden in children. (You might need to use __slots__ to override the property at runtime.)

[pastel egret]

It's probably better to define a protocol class yourself with the methods you want.

[jolly cipher]

just to make sure, is Any a materialization of Any?

from a practical perspective, I'd say yes.
the typing spec doesn't seem to negate that, either:

the glossary term doesn't apply to the (Any, Any) combination.

the relevant concept section refers to "replacing" Any with a type—a different type, according to the definition of replacing. but then it states that the materialization relation (defined on the set of gradual types, including Any) is reflexive.

right?

which implies that Any is more static than Any and more dynamic than Any at the same time, due to reflexivity (same for any other types, e.g. dict[str, str] is more static than dict[str, str] at the same time being more dynamic than dict[str, str]).

this reminds me of the fun truth about type being the instance and the class (so also the nominal type, and obviously the structural type) of itself.

[oblique urchin]

By the definitions in the spec, yes

[jolly cipher]

another question.

given

class Foo:
    __class__ = int

does nominal type int describe Foo?

[oblique urchin]

no

[jolly cipher]

i'm asking this as i've read

For a type such as str (or any other class), which describes the set of values whose __class__ is str or a direct or indirect subclass of it

[oblique urchin]

though I don't particularly care how type checkers interpret that code

[jolly cipher]

yeah it's more of a joke than serious case

[jolly cipher]

referring to type() seems more appropriate

maybe?

otoh changing __class__ dynamically does bear some semantics with it

i think it's not really about the runtime value of __class__ though. fully nominal types are always fully static. so from the static typing perspective, i think that assigning __class__ anything does not imply a thing (since it happens at runtime).

i could as well ask a question like, for the code ```py
import builtins

builtins.int = str

is `int` a nominal subtype of `str`

and the answer will still obviously be no.

even though at runtime it's "true"

hmmmm. from https://typing.readthedocs.io/en/latest/spec/glossary.html#term-gradual-type:

All types in the Python type system are “gradual”. (...) Gradual types do not participate in the subtype relation, but they do participate in consistency and assignability.

is that correct?

that would imply that all types in the python type system do not participate in the subtype relation (but fully static types do, and it is explicitly stated that fully static types are a sub-part of gradual types).
maybe we should have a separate term for a type that does contain some gradual form? or isn't it just "dynamic type"?

[oblique urchin]

This part of the terminology is somewhat confusing, yes

Does seem like you found a contradiction there

[jolly cipher]

i'd reword this to dynamic type (and additionally explain that dynamic types = gradual types - fully static types in the glossary + concepts maybe?)

ooor just simply describe that we mean types that do not contain gradual forms.

however, i think there may be other places with similar usages of "gradual types" instead of "types without a gradual form"

"dynamic type" should be fairly intuitive.

maybe even drop the "fully" from "fully static types" to further simplify that?

(that could be a breaking change!)

[jolly cipher]

i think it is very logical to assume that dict[str, Any] is not "static" (meaning: fully static) knowing that Any is a gradual form and knowing what gradual form means.

so instead of saying "fully static types" and "types with one or more gradual forms", say "static types" and "dynamic types"/"partially static types"

wdyt?

for backward compatibility (necessary to have because of e.g. references to the typing spec in other papers), maybe leave "fully static" be

and only add a new disambiguation term "dynamic type".

it is even indirectly used already!

A gradual type system is one in which a special “unknown” or “dynamic” type is used to describe names or expressions whose types are not known statically. In Python, this type is spelled Any.

otoh Any is defined as "an unknown static type"...

[jolly cipher]

or... "not fully static types". but that feels like they are at least partially static (even though it's not necessarily implied. "not full" does not mean "not empty").

(not pinging Jelle as you are active here) cc @forest vessel

maybe i'll just open a discussion without this wall of inconsistently formulated sentences (right after fixing the glossary term using the existing terminology).

[jolly cipher]

filed a PR https://github.com/python/typing/pull/1907

@oblique urchin could you please approve workflows to run there?

[stiff acorn]

What typing classifier should an extension module use?

Typing :: Typed or Typing :: Stubs Only

[tranquil turtle]

I would not use the second option

[trim tangle]

Stubs Only means that your package only provides stubs, like boto3-stubs https://github.com/pypa/trove-classifiers/issues/84
So if it's not that, use Typing :: Typed if you have types within your package (either inline or in pyi files)

it sucks that you have to hunt for the meaning of these classifiers, I don't know why the meaning of each classifier is not listed at https://pypi.org/classifiers/

[stiff acorn]

Thanks, that makes sense

[jolly cipher]

what are some of the typing bad and good practices you'd include in a talk about typing DOs and DON'Ts?

what i have so far:

opinionated suggestions are also welcome.

1. (opinionated) DON'T skip annotating return values.
2. DON'T use bare # type: ignore.
3. DO use overloads, especially when unions are involved.
4. DON'T interchange Any and object.
5. DO restrain from statically unknown types.
6. DON'T use dict[str, Any] for annotating fixed-structure data (used TypedDict, dataclasses, or other models instead).
7. DO type hint with generic built-in types (3.9+, PEP 585).
8. DO use X | Y instead of typing.Union[X, Y] in 3.10+ codebases (PEP 604).
9. DO use typing.LiteralString.
10. DO use typing.Literal.
11. DO use typing.Self.
12. (opinionated) DO prefer typing.Never to typing.NoReturn (no difference).

i'll also look for something more to include from @trim tangle's typing tips

[trim tangle]

I recently demolished some of that page because it was kinda terrible
But if you really want you can inspect the history 🙂

[jolly cipher]

i'll also add the parameter-widest and return-type-narrowest good practice, from the typing spec

[trim tangle]

• Read everything from https://typing.readthedocs.io/en/latest/

[jolly cipher]

well i did

and that resulted in my contribution

https://github.com/python/typing/pull/1907 :D

[trim tangle]

I meant as a point to your list

[jolly cipher]

aaah!

good!

thanks :)

[spiral fjord]

Just curious, why is Never preferred over NoReturN?

[trim tangle]

NoReturn has a confusing name

that's the only rationale for introducing Never

[fossil nest]

I have a very large question ready to paste. is this okay?

[trim tangle]

sure

[spiral fjord]

The bot will probably zap you

!paste

[rough sluice]

Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

[jolly cipher]

and it was a somewhat convention that NoReturn was reserved only for function return types, but Never has a broader use case

[fossil nest]

I have a problem with pydantic 😅
context:

class ServerEventType(enum.StrEnum):
    CREATE = enum.auto()

class ServerEventData(pydantic.BaseModel):
    pass

class ServerCreateEventData(ServerEventData):
    username: str

class ServerEvent(pydantic.BaseModel):
    type: ServerEventType
    data: ServerEventData

note: other ServerEventType and subclasses of ServerEventData have been omitted for brevity

I can initialise a ServerEvent like so:

ServerEvent(
    type=ServerEventType.CREATE,
    data=ServerCreateEventData(
        username="Evorage",
    ),
)

this is fine however I would like to use the following because it reduces duplicated code and sometimes I mess up matching the type and data by accident:

ServerCreateEvent(
    username="Evorage",
)

these two expressions should evaluate to the exact same object being produced

here lies the problem, I can't find a way to achieve this without either losing vscode type analysis or without generating a warning by pydantic.

my two approaches:

def ServerCreateEvent(*, username: str) -> ServerEvent:
    return ServerEvent(
        type=ServerEventType.CREATE,
        data=ServerCreateEventData(
            username=username
        ),
    )

not ideal because I have to duplicate the parameters for each ServerEventData in the functions signature

and

class ServerCreateEvent(ServerCreateEventData):
    @pydantic.model_validator(mode="after")
    def model_validator(self):
        return ServerEvent(
            type=ServerEventType.CREATE,
            data=ServerCreateEventData(
                username=self.username,
            ),
        )

which keeps type information but generates a warning and doesn't return an instance of ServerEvent
any help would be appreciated

[trim tangle]

I have a very large answer ready so be prepared

wait I don't

[fossil nest]

there is a third approach similar to the second:

class ServerCreateEvent(ServerCreateEventData):
    @pydantic.model_validator(mode="wrap")
    @classmethod
    def model_validator(cls, data: typing.Any, handler: pydantic.ModelWrapValidatorHandler[typing.Self]):
        return ServerEvent(
            type=ServerEventType.CREATE,
            data=ServerCreateEventData.model_validate(data),
        )

which also generates a warning :/

[trim tangle]

You probably want discriminated unions

[trim tangle]

Well, type annotations are used by different people in a wide variety of contexts. So it's hard to give context-free tips that aren't just "use new feature instead of old feature"

Not everyone has full type coverage as a goal

Maybe I just want some autocompletion for my code, as long as it doesn't make me refactor it

[fossil nest]

I don't think this solves the problem

not to keep vscode's type checker happy at least

[trim tangle]

the pet example seems like exactly what you want, no?

[trim tangle]

class ServerCreateEventData(BaseModel):
    username: str

class ServerCreateEvent(BaseModel):
    type: Literal["create"] = "create"
    data: ServerCreateEventData

class ServerShootEventData(BaseModel):
    x: float
    y: float
    vx: float
    vy: float
    projectile_id: str

class ServerShootEvent(BaseModel):
    type: Literal["shoot"] = "shoot"
    data: ServerShootEventData

AnyEvent = ServerCreateEvent | ServerShootEvent | ...

Event = Annotated[AnyEvent, pydantic.Discriminator("type")]

[fossil nest]

I don't see how I could customise the type of data with those unions apart from that it's 'one of the following possibilities'
Ideally, it's pretty strict and only always the CreateEventData for a CreateEvent

[lavish owl]

!e

[trim tangle]

You'll have a class per event type, like in the snippet above

having a "tag" and data varying on that tag is called a "tagged union" or a "discriminated union" (or an enum in Rust for some reason)

[fossil nest]

could you provide a really short code sample demonstrating this sorry, I'm having a little trouble understanding

lets say I want to end with a subclass of a ServerEvent

the subclasses being ServerCreateEvent and ServerShootEvent

[trim tangle]

Are you only using pydantic to serialize events? (i.e. only go from BaseModel to a dict)

[fossil nest]

yeah with fastapi, i'm attempting to create a strict schema for websocket messages

It works fine currently except I wanted to simply the event creation process slightly

with a dedicated object rather then specifying type and data each time

[trim tangle]

If you want to keep the current scheme with subclassing, you could add a class variable to each ServerEventData that indicates its event type. Then you can have something like ```py
class ServerCreateEventData(ServerEventData):
_event_type: ClassVar = ServerEventType.CREATE
username: str

def make_event(data: ServerEventData) -> ServerEvent:
event_type: ServerEventType = data._event_type # type: ignore[attr-defined] # or whatever the error code in your type checker
return ServerEvent(event_type=event_type, event_data=data)

event = make_event(ServerCreateEventData(username="alice"))

[trim tangle]

If you only want dumping, then your life is significantly easier. Here's a runnable example https://paste.pythondiscord.com/5NSWAXE6PUTGLQH234RBYJCP54

server_event = make_event(CreateEvent(username="alice"))
print(server_event.model_dump_json())
# {"type":"create","data":{"username":"alice"}}

Note that I'm using data: object, because apparently, if you specify data: BaseEvent, it outputs an empty object (because BaseEvent doesn't have any attributes) even if you provide a subclass of BaseEvent to data

[oblique urchin]

not sure I agree with "do use overloads"

overloads are a powerful tool and sometimes the best way to describe an API

but when writing new code prefer to write it in a way that doesn't need overloads

[jolly cipher]

yes, exactly :)

[fossil nest]

thank you for your help @trim tangle in the end I reworked my code to be more simple (so simple it didn't need any of the above classes xD) and it's fully pydantic typed. Here's a snippet in action:

    async for json in websocket.iter_json():
        client_event = ClientEvent.validate_json(json)

        match client_event:
            case ClientChatEvent():
                ...

            case ClientCreateEvent():
                server_event = ServerCreateEvent(username="Evorage")
                ...

@ClientEvent.on("create")
class ClientCreateEvent(pydantic.BaseModel):
    ...

@ServerEvent.emit("create")
class ServerCreateEvent(ServerEvent):
    username: str

socket io style messaging but pydantic™️

[trim tangle]

sadly some things require overloads, e.g. when you have variadics and you need to do ```py
def zip[T0, T1](self, t0: Iterable[T0], t1: Iterable[T1], /) -> Iterator[tuple[T0, T1]]
def zip[T0, T1, T2](self, t0: Iterable[T0], t1: Iterable[T1], t2: Iterator[T2], /) -> Iterator[tuple[T0, T1, T2]]

...

Some people will be okay with this^, some people will make a crazy contraption using TypeVarTuple to avoid this, and some people will be happy with zip(self, *its: Iterable) -> Iterator[tuple]

[trim tangle]

Are you actually making a talk?

[jolly cipher]

yeah, pywaw #117

[trim tangle]

What audience do you have in mind?

[jolly cipher]

regular python users who want to type their code better

(the points i sent initially changed btw)

i can send in DM

[trim tangle]

can you post here?

[jolly cipher]

key points (slides in progress):

General Tips

1. DO remember about context (how your interfaces will be used).
2. DO restrain from statically unknown types.
3. DO read the typing spec.
4. DO treat typing and runtime as separate worlds coexisting with each other.

New Typing Features

1. DO adopt typing.Self (PEP 673)
2. DO adopt TypeVar defaults (PEP 696)
3. DO adopt built-in generics (PEP 585)
4. DO adopt | union syntax (PEP 604)
5. DON'T use dict[str, Any] blindly

Typing Tips

1. DON'T confuse Any with object
2. DON'T abuse Any
3. DON'T use deprecated aliases from typing
4. DO adopt stub files for annotating extension modules
5. DO use overloads especially when unions are involved

Opinionated:

1. DON'T use bare # type: ignore
2. DON'T skip annotating return values
3. DO use PEP 563 if it helps, despite future deprecation
4. DO prefer typing.Never to typing.NoReturn
5. DO avoid T | Awaitable[T] union return types
6. DON'T use TYPE_CHECKING if whatever evaluates your type hints at runtime
7. DO be pragmatic about TYPE_CHECKING

Opinionated, for libraries:

1. DO type-check at tail (minimum supported version)
2. DO adopt __all__ to control re-exports
3. DO minimize runtime overhead if using inlined types

ask me if you'd like me to cover what message i'm planning to convey specifically for a given key point

they may sound vague

some of those are also pretty specific -- they come mainly from my experience

funnily, chatgpt also has some cool points to add i had thought about (typing.LiteralString, Final)

• typing-extensions may be worth mentioning

aside from that, the topic of the talk is very open-ended. i think i'll pick something more specific next time (e.g. detailed talk about some new feature(s) from changelog/etc.)

i'd say that this talk should be compiled from the most pressing issues currently common among the users of typing

[trim tangle]

If I had to cover the most important stuff (from my point of view):

1. The type system is essentially a second language added to Python. There isn't a good tutorial or comprehensive book for it, but there's a specification: https://typing.readthedocs.io/en/latest/, and mypy has its own docs https://mypy.readthedocs.io/en/stable/. I would advise reading the typing site thoroughly. Understanding how the type system works is key to using it effectively.
   I know it sounds like RTFM, but it's really true. It's going to introduce you to all the things that are in typing (like generics, overloads, protocols) and will explain concepts like variance

2. If you want to make the most of your type checker, there are a few things you can do (this is the opinionated part for people who care about type coverage)
   2.1. avoid Any at all costs (which includes unparameterized types like list instead of list[Foo])
   2.2 you can check how much of your library is typed, see testing and ensuring annotation quality -- it's like test coverage, but for types
   2.3. Sometimes you will need to refactor your code to make types work. Be aware of this tradeoff. Sometimes it's worth changing the interface of (non-published) things if it provides better developer experience (autocompletion, jump to definition etc.) or helps catch more errors through static analysis.

3. Think about the design of your types. When you're making a function, a module, a class, or an entire library, consider whether using it is nice with a type checker (the redis library would be a good study example with its T | Awaitable[T] nightmare).

4. Don't lie to the type checker. For example, if x is sometimes an int and sometimes None, do x: int | None, not x: int (even if it's inconvenient)

And maybe
5. The typing ecosystem is evolving over time, so check what's new from time to time. Maybe that annoying problem you had a month ago is now solved.

IMO, new syntax doesn't matter in the grand scheme of things. Your types will work the same if you're using typing.List and typing.Union. It's the more strategic things that are important

However, if you only have 45 minutes, maybe it's also a good idea to just rapid-fire a few commonly occuring snippets and discuss the larger lesson behind them. For example, you could distill the redis situation into a slide-sized snippet:

class DictionaryClient:
    def __init__(self, is_async: bool): ...
    def get_definition(self, word: str) -> str | Awaitable[str]:
``` on the next snippet, show how one would (try to) use this with a type checker and fail. Then you could show alternative solutions: separating a `SyncDictionaryClient` and `AsyncDictionaryClient`; and a more advanced solution using generics

(I've never given a live technical talk in my life, so take all I've said with an industrial shipment of salt)

[tacit sparrow]

Is it expected?

import types

def test(x: types.FunctionType): pass

print(type(test) == types.FunctionType)
# Argument of type "(x: FunctionType) -> None"
# cannot be assigned to parameter "x" of type "FunctionType" in function "test"
#   "function" is not assignable to "FunctionType"
test(test)

[wooden cipher]

i have a base class with generic types
class BaseConnectionFactory(Generic[Base, Pool]):

then i inherit from this
class ConnectionFactory(BaseConnectionFactory[Valkey, ConnectionPool]):

now i want to inherit from the second class
class SentinelConnectionFactory(ConnectionFactory):

but i don't want ConnectionPool as Pool in this case
i want to intrduce a new type

how would i do that?

[jolly cipher]

if you can change ConnectionFactory and you have to inherit from ConnectionFactory, i'd do

class BaseConnectionFactory(Generic[Base, Pool]):
    ...

ConnectionFactoryPool = TypeVar("ConnectionFactoryPool", default=ConnectionPool)

class ConnectionFactory(
    BaseConnectionFactory[Valkey, ConnectionFactoryPool],
    Generic[ConnectionFactoryPool],
):
    ...

class SentinelConnectionFactory(ConnectionFactory[NewPool]):
    ...

this will preserve the behavior of type ConnectionFactory being assignable to BaseConnectionFactory[Valkey, ConnectionPool].

at the same time allowing you to inherit from ConnectionFactory with BaseConnectionFactory's Pool as NewPool instead of ConnectionPool.

[wooden cipher]

is that much code really necessary?
i feel like there should be a way with less code

[trim tangle]

In 3.13, that can be spelled as ```py
class ConnectionFactory[P: ConnectionPool = ConnectionPool](BaseConnectionFactory[Valkey, P]):

[wooden cipher]

no i should support 3.10+

tho that's an interesting syntax i should learn

[trim tangle]

Of course, I was just saying that it's going to get more compact in the future

[trim tangle]

Can you show the code? Why do you have these type variables in the first place?

[wooden cipher]

https://github.com/amirreza8002/django-valkey/blob/api/django_valkey/base_pool.py

this is the base class
which is being inherited from by two classes

one of them is
https://github.com/amirreza8002/django-valkey/blob/api/django_valkey/pool.py#L20

and
https://github.com/amirreza8002/django-valkey/blob/api/django_valkey/pool.py#L47
inherits from the above class

the base class is also used in the async part of the library

if the codebase is to messy pls do mention 🫠
i'm doing a lot of refactoring for a major release

[trim tangle]

Do you have an example of how one would use the ConnectionFactory or the SentinelConnectionFactory classes?

[wooden cipher]

it is used internally
https://github.com/amirreza8002/django-valkey/blob/api/django_valkey/base_client.py#L86-L91
this gets the class

https://github.com/amirreza8002/django-valkey/blob/api/django_valkey/client/default.py#L1213-L1224
these use it

https://github.com/amirreza8002/django-valkey/blob/api/django_valkey/pool.py#L93
this function is used in the first link to get the class

[trim tangle]

Maybe you can simplify the connection factory to this? I don't see why you need those type variables ```py
from valkey.connection import ConnectionPool

class BaseConnectionFactory:
_pools: dict[str, ConnectionPool] = {}

def __init__(self, options: dict) -> None: ...
def make_connection_params(self, url: str | None) -> dict: ...
def get_connection_pool(self, params: dict) -> ConnectionPool: ...
def get_or_create_connection_pool(self, params: dict) -> ConnectionPool: ...

def get_connection(self, params: dict) -> Valkey:
    raise NotImplementedError
def connect(self, url: str) -> Valkey:
    raise NotImplementedError
def disconnect(self, connection: Valkey) -> None:
    raise NotImplementedError
def get_parser_cls(self) -> type[Any]:
    raise NotImplementedError

[wooden cipher]

but the async part uses the same base class

https://github.com/amirreza8002/django-valkey/blob/main/django_valkey/async_cache/pool.py

[trim tangle]

That's an LSP violation, disconnect in the base class returns None, not Coroutine[None, Any, Any]

so you can't use AsyncConnectionFactory in place of BaseConnectionFactory

[wooden cipher]

emm
well i can delete disconnect from the base class

[trim tangle]

Same with connect

[wooden cipher]

yes i think i have to delete four methods from the base class

but the rest are shared

i also have a pool for the cluster client

[trim tangle]

I don't really know how django works. Do you have some example django application that uses this library?

or a small demo at least

[wooden cipher]

well you won't see much of these in a project
most of it is hidden behined django's cache object

you only give the client name in a CACHES setting

[trim tangle]

so who's the user of a ConnectionFactory?

[wooden cipher]

the second link
it's used internally

the user sees it's effect tho
if sentinel pool is used, they can configure the client easier

[trim tangle]

So from perspective of the user of a connection factory: you have a connect method that returns a Valkey, and a disconnect method that takes a Valkey and returns None. Right?

everything else is internal machinery

[wooden cipher]

the user will never see Connection Factory

they can configure which one is used
but never interact with it themself

[trim tangle]

by the user I mean the user of ConnectionFactory, this class

class DefaultClient(SyncClientMethod):

[wooden cipher]

yes, only connect and disconnect

[trim tangle]

I'm assuming you want to parameterize Base to allow for either Valkey or AsyncValkey. But the only user of the bareConnectionFactory can't use the async functionality because that would require then to call await factory.connect()

[wooden cipher]

a DefaultClient is not the only client tho

AsyncDefaultClient exists
along with a number of otheres

sorry i know too much about this package that i forget people don't 😅

[jolly cipher]

it's just one line of additional code

because you need to have the typevar

you already have typevars for other type arguments

and yes, you can do the same with PEP 695

in 2 years

[jolly cipher]

well, maybe 2-3

in case you need to import TypeVar from typing_extensions

to be able to use default=...

+mypy doesnt support PEP 696 defaults with PEP 695 syntax yet

afaik

[jolly cipher]

also, i don't like using P for that because P is often associated with ParamSpecs

[trim tangle]

yeah, Pool or PoolT would be better

[trim tangle]

This is what I understand about the library so far.
You have these client classes:

• DefaultClient uses ConnectionFactory
• HerdClient uses ConnectionFactory
• SentinelClient uses SentinelConnectionFactory
• ShardClient uses ConnectionFactory
• AsyncDefaultClient uses AsyncConnectionFactory
• AsyncHerdClient uses AsyncConnectionFactory
• AsyncSentinelClient uses AsyncSentinelConnectionFactory
  Each of them knows what BaseConnectionFactory subclass it uses.
  Subclassing serves two unrelated purposes: code reuse and polymorphism. You're not using it for polymorphism here, as it wouldn't make sense to use SentinelConnectionFactory inside of a ShardClient (if I understand correctly). And async factories have an incompatible interface anyway.

So you're really only using the base class for code reuse. Let's see what methods you are able to reuse.

• All the factories can reuse __init__ and make_connection_params because they don't do any I/O
• get_or_create_connection_pool and get_connection_pool can only be reused by sync classes because they do I/O
• all the other methods are abstract

So I would refactor all this to:

• A base class that only has __init__ and make_connection_params for code reuse, it doesn't do I/O and doesn't use any type variables
• A SyncConnectionFactory class with no type variables (from which you derive SentinelConnectionFactory), operating on Valkey instances
• An AsyncConnectionFactory class with no type variables (from which you derive AsyncSentinelConnectionFactory), operating on AValkey instances

[wooden cipher]

i should note that get_connection_pool and get_or_create_connection_pool don't do i/o

they get the pool class

[trim tangle]

In that case they can be in the base class as well

[wooden cipher]

i think i should change their type hints to type[Pool]

[trim tangle]

where?

[wooden cipher]

nvm that
i think i'm losing battery

[wooden cipher]

but for the types
you suggest the same thing bswck suggested?

[trim tangle]

I suggest removing the type variables entirely to simplify the code 🙂

or at least removing the Base type variable. Then you can have ```py
PoolT = Generic("PoolT")

class BaseConnectionFactory(Generic[PoolT]):
_pools: dict[str, Any] = {}

def __init__(self, options: dict) -> None: ...
def make_connection_params(self, url: str | None) -> dict: ...
def get_or_create_connection_pool(self, params: dict) -> PoolT: ...
def get_parser_cls(self) -> type[Any]:
    raise NotImplementedError

[wooden cipher]

tho the problem is in Pool

[trim tangle]

well, the problem is, you don't know what Pool is - it's specified with import_string

[wooden cipher]

but since it's internally used
perhaps removing the type hints is a viable option

[wooden cipher]

the codebase has a lot of import_string s :p
so i decided to only type hint as what the library provides
not what the user can self define

[trim tangle]

class SyncConnectionFactory(
    BaseConnectionFactory[valkey.Pool]
):

class SyncSentinelConnectionFactory(
    SyncConnectionFactory,
    BaseConnectionFactory[valkey.sentinel.Pool],
):

I just realized that you can do this

[wooden cipher]

would you, yourself, do this or just delete the type hint?

[trim tangle]

I'm not sure, depends on whether the code is going to get more complex and will need to use the pool in more places

I think having the pool as a typevar is not a bad idea, because SentinelConnectionFactory and the async factories do work with different, unrelated pools

I would flip a coin

!e

import random
print(random.choice(["keep PoolT", "remove PoolT"]))

[rough sluice]

:white_check_mark: Your 3.12 eval job has completed with return code 0.

remove PoolT

[wooden cipher]

hmm
python has spoken

[trim tangle]

seriously though, I think either would be fine

[wooden cipher]

but for my info
how does the mro not prevent this?
the methods are all defined in SyncConnectionPool
so base class shouldn't be read

[trim tangle]

You mean at runtime?

[wooden cipher]

no for type checking

like

class Base(Generic[T]):
    def a(self) -> T: 


class A(Base[int]):
    pass


class B(A, Base[float]):
    pass

doesn't A being first make the type int without the base being used for B?

[trim tangle]

This will be wrong because float is not a subtype of int

However, if you have e.g. A(Base[int | str]) and B(A, Base[str]), that's totally fine. Because Base is covariant, Base[str] is a subtype of Base[int | str] (because str is a subtype of str | int)

[wooden cipher]

valkey.sentinel.ConnectionPool
is not a subclass of valkey.connection.ConnectionPool

[trim tangle]

Then SentinelConnectionFactory cannot be subclass of ConnectionFactory

Because the contract of a cf: ConnectionFactory is that cf.get_connection_pool() returns a valkey.connection.ConnectionPool

[wooden cipher]

droping the Pool type hint then?

[trim tangle]

That is possible

Btw @wooden cipher this is probably a bug 🙂 https://github.com/amirreza8002/django-valkey/blob/779dff7feb21df6fa388e6465758100967e87443/django_valkey/base_pool.py#L70-L71

[rough sluice]

django_valkey/base_pool.py lines 70 to 71

cp_params = params
cp_params.update(self.pool_cls_kwargs)```

[trim tangle]

(this is the same as params.update(self.pool_cls_kwargs), the first line doesn't create a copy)

[wooden cipher]

ah
thanks!
it was

        cp_params = dict(params)
        cp_params.update(self.pool_cls_kwargs)

before
i deleted the dict() but forgot to merge it into one line 🫠

thank you for the help🙌
sorry i take a while to understand things

[trim tangle]

As for what I would do, I might just delete the whole ConnectionFactory hierarchy. A pool is already a connection factory

If you need to parse some settings that go through Django, I'd frame it as that -- a function/method to parse valkey settings, producing a ValkeyOptions object or just a dictionary

[harsh lantern]

will python has a void type
for functions like print
that returns None but you don't want to use its result as a value
so using it as a value (assigning it to a variable) would result in a lint error