#type-hinting

1 messages · Page 27 of 1

cunning plover
# 
job = Job(payload['source'], payload['target'])
await job()

it is called like this

grave fjord
#

-> None:

olive parrot
#

Hello, flask expose a config property on the Flask object, it is a dict with additional methods, the issue is, when i try to app.config["VALUE"] i get an Unknown value type, anyone know how i can specify the type of the returned value ?

rare scarab
#

x: Something = app.config[key]

rare scarab
olive parrot
rare scarab
#

Maybe typing.cast

# 
import typing as t

x = t.cast(Something, app.config[key])
olive parrot
#

from what i've seen i should be able to subclass flask and config class but even when i do that pylance is not picking up the new keys i did a ticket for and i'll see what they say

trim tangle
olive parrot
#

i can't get a value from the config object without Pylance complaining

#

config: Mainconfig will not work

trim tangle
#

you mean, you have Pylance on strict and it's complaining about a partially unknown type?

olive parrot
#

yep

#

and i really don't wish to make ignore comments

trim tangle
#

do you really want the strict setting? it's a bit of a pain
there's a standard setting nowadays which has most of the options you probably need

#

aiohttp also has a willy nilly config like this, and it made a pretty clean solution for this. Instead of strings, you can use a typed key: py class AppKey(Generic[T]): def __init__(self, name: str, tp: type[T]) -> None: ... and you use it like this: ```py

config.py

DB_SETTINGS_KEY = AppKey("db_settings", DbSettings) # inferred as AppKey[DbSettings]

main.py

from config import DbSettings, DB_SETTINGS_KEY

app[DB_SETTINGS_KEY] = DbSettings(...) # setitem is type-checked
db_settings = app[DB_SETTINGS_KEY] # inferred as DbSettings

olive parrot
#

getting the same error on basic level

trim tangle
#

can you show the full snippet and the error you get?

olive parrot
trim tangle
olive parrot
#

why ? it was able to do it once, and i never was able to reproduce it

trim tangle
#

the value of config_class is not automatically "linked" in any way to Flask.config. At least from the typing standpoint

olive parrot
#

Is it possible to do it by refactoring flask code ? or it's not possible with current type features

trim tangle
#

It might be possible with some significant changes

#

but I don't think flask is going to accept that

olive parrot
#

i'm more asking about how could you implement it

trim tangle
#

You would have to make Flask generic, like class Flask(App, Generic[ConfigT]): and then accept a ConfigT value or a Callable[[], ConfigT] factory in the __init__

olive parrot
#

ok, so that how i though it would be xD

#

and yes, tons of refactoring then

trim tangle
#

sometimes you just gotta have a # type: ignore

#

or maybe typing.cast

#

This is probably going to be a core function in your code, and will be exercised thousands of times in tests. So you don't really gain anything by being "type safe"

olive parrot
#

it's more about auto-completion than type safe

trim tangle
#

this will give you auto-completion if you do main_config(app).some_field_of_MainConfig

olive parrot
#

i went with this then


def get_config(app: Flask) -> DefaultConfig:
    return cast(DefaultConfig, app.config)
eternal creek
#

Should I annotate types for my code as said in this video?
https://www.youtube.com/watch?v=wlbkON0h8Y0

YouTube
Numeryst
Basic Types - Python Type Hints for Beginners #2

In this tutorial, I explain how to use type hints for basic types like str, int, float, and Bool. In addition, I explain the relation between these types. Annotating types for basic data types in Python pays off in the long run and in large Python projects.

Check out our Python Type Hints for Beginners playlist:
https://www.youtube.com/play...

▶ Play video
rare scarab
olive parrot
#

ah ye, forgot about that xD

eternal surge
#

I have a function that takes in an instance of a subclass of a class Message and returns an instance of a different subclass of class Message
Like so:

class Message:
  pass

class SimpleMessage(Message):
  pass

class ErrorMessage(Message):
  pass

def f(message: SimpleMessage | ErrorMessage) -> SimpleMessage | ErrorMessage:
  ...
  return different_message

How can I annotate f to refer to instances of all subclasses of the Message?

oblique urchin
#

Or is there some reason I'm missing why that's not appropriate?

eternal surge
oblique urchin
eternal surge
#

Say I have a function like this:

def f(x: str | int):
    if isinstance(x, str):
        return int(x)
    elif isinstance(x, int):
        return str(x)
y = f(100)```
Shouldn't a type-checker be able to infer that the type of `y` is `str` in this case? It shows me `int | str`, but I know for a fact that for all inputs of type `int` I should be getting back an output of type `str`
oblique urchin
#

!d typing.overload

rough sluiceBOT
#
typing.overload 

@typing.overload```
Decorator for creating overloaded functions and methods.

The `@overload` decorator allows describing functions and methods that support multiple different combinations of argument types. A series of `@overload`-decorated definitions must be followed by exactly one non-`@overload`-decorated definition (for the same function/method).

`@overload`-decorated definitions are for the benefit of the type checker only, since they will be overwritten by the non-`@overload`-decorated definition. The non-`@overload`-decorated definition, meanwhile, will be used at runtime but should be ignored by a type checker. At runtime, calling an `@overload`-decorated function directly will raise [`NotImplementedError`](https://docs.python.org/3/library/exceptions.html#NotImplementedError).
eternal surge
noble token
#

hi

rose mist
#

Greetings guys,

Hope all are well. When should I use .pyi files? I am making a library, and want to see what's the best practice, and how to use and store these type of files.

trim tangle
#

otherwise just use normal type annotations in your library and place a py.typed file in your source

rose mist
#

I usually annotate my .py classes, methods, and functions thoroughly.

#

May I ask what py.typed is?

trim tangle
rough sluiceBOT
trim tangle
cunning plover
# rose mist Greetings guys, Hope all are well. When should I use `.pyi` files? I am making ...

i would recommend by default adding py.typed and declaring all yout code as type good (just don't run any actions on import)
And for stuff that is exclusively type incompatible, make file copy with .pyi extension and cover it with those stubs.
TLDR: .pyi for exceptional cases when it is impossible to type some file (usually almost nothing is impossible to type)
.pyi will overshadow regular file for its typing

rose mist
#

I see. I mean, I always annotate, so I shouldn't add the .pyi files?

#

So it's not a good practice?

viscid spire
#

it's not good or bad

#

it's just "what is needed when you wanna do certain things"

cunning plover
rose mist
#

I see!

viscid spire
#

any C extension code would need .pyi

#

because obviously you can't write a python typehint in C

rose mist
#

Ok, so just to clarify, all of my library code is in .py.

#

So I shouldn't use them (pyi I mean)?

cunning plover
# rose mist So I shouldn't use them (pyi I mean)?

in this case u need .pyi only if u have files with bad code that runs on file import (defend such stuff with if __name__=="__main__")
.pyi can overshadow such files and offer their types-stubs to remove such file executions

#

as long as u design code for .py only files well, u don't need .pyi

rose mist
#

Can you kindly classify between need and should?

#

I would like to know what's good practice.

cunning plover
#

but some situation can force your hand to do that

rose mist
#

I see!

cunning plover
#

for example if your code has import ddtrace datadog importing library.
it is nasty one, as it hacks all code, and dinamicaly patches all code on runtime during its library import

#

it can be nice to protect typing code executions from it with .pyi file overshadowing where u import such nasty lib

#

tldr: we need .pyi against bad code only that we can't control directly (in case of having .py only files)

viscid spire
#

you might also want to create your own .pyi file for someone else's library

#

if they have not typehinted it

cunning plover
#

oh yeah. that's a choice too.
OR... u could just inject py.typed file to their library after install 😅
as a less effort solution to get all library types working for you (as long as they don't have bad code running on import, it should be fine)

otherwise .pyi for third party libs can serve as more esoteric defined interfaces/protocols

rose mist
#

Ok, that makes better sense for me now.

#

Thank you so much! I'm gonna remove the pyi files then.

viscid spire
#

👍

rustic gull
#

how do I type hint this

# 
def flatten(x):
    if isinstance(x, (list, tuple, dict)):
        return [a for i in x for a in flatten(i)]
    else:
        return [x]
#

the issue is that when I run it on a list[bool], it (pyright) doesn't like it and doesn't know the function returns list[bool]

#

ok I fixed it I said -> list[Any] and pyright is satisfied

trim tangle
#

also, why do you want it to work on dicts' keys?

#

In general, you cannot express this function in Python's type system. You can only type a function like list[list[T]] -> list[T] or Iterable[Iterable[T]] -> list[T]

rustic gull
#

I have a new one

# 
Argument of type "list[((...) -> Unknown) | None]" cannot be assigned to parameter "loader" of type "list[(...) -> Unknown] | None" in function "_add"
  Type "list[((...) -> Unknown) | None]" cannot be assigned to type "list[(...) -> Unknown] | None"
    "list[((...) -> Unknown) | None]" is incompatible with "list[(...) -> Unknown]"
      Type parameter "_T@list" is invariant, but "((...) -> Unknown) | None" is not the same as "(...) -> Unknown"
      Consider switching from "list" to "Sequence" which is covariant
    "list[((...) -> Unknown) | None]" is incompatible with "None"PylancereportArgumentType
#

I think I'll just add a little pyright: ignore

#

...

trim tangle
#

I don't think you will be able to explain this function to a type checker in any way
you'll have to add an explicit annotation like def flatten(x: object) -> list[Any]

rustic gull
trim tangle
#

where did you add it? can you show the code?

rustic gull
#

oh if you mean my new error its a different one and I just chose to ignore it

#

its with more complex thing

#

actually I managed to fix it, it was an issue with my annotations

rustic gull
#

is there an opposite of Sequence

trim tangle
#

?

rustic gull
# trim tangle ?

Like I have a recursive flatten function which stops when an element is no longer a Sequence

#

for example [[a, [b, c]]] will call with [a, [b,c]] then stop on a and call with [b, c] and stop on b and c

#

those a b c could be anything but a sequence

trim tangle
#

You can make a bunch of overloads like ```py
@overload
def flatten(x: list[list[list[list[T]]]]) -> list[T]: ...
@overload
def flatten(x: list[list[list[T]]]) -> list[T]: ...
@overload
def flatten(x: list[list[T]]) -> list[T]: ...
@overload
def flatten(x: list[T]) -> list[T]: ...

#

Imagine if you have a function like this: ```py
X = TypeVar("X")

def do_something(items: list[list[X]]):
flattened = flatten(items)
``` what type does flattened have? You can't really know, because X could be a list of something itself, or it could be not a list.

#

and there's no way to express that's something isn't a sequence

rustic gull
rustic gull
#

alright one gripe I have which I spent too much time annotating / adding ignores, is it possible to make it so that when a:list | dict and a.append(1), it will be satisfied because list has append, instead of showing an error because dict doesn't?

tranquil ledge
#

You’d have to narrow the type to list somehow, otherwise the error would be valid. Maybe an isinstance check would suffice for your use case.

rustic gull
tranquil ledge
trim tangle
#

you can do something like assert isinstance(a, list) and the type checker will be convinced that it's a list afterwards

tranquil ledge
#

Alternatively — and this depends on your specific code, obviously — consider if you should be calling append in that place when the validity of that call is uncertain.

viscid spire
trim tangle
trim tangle
viscid spire
#

type RList[T] = list[RList[T]] | T is close for the arg type

trim tangle
#

pyright thinks that the correct answer is list[list[int]] 🙂

image.png
viscid spire
#

What if you remove the | T?

trim tangle
#

that would not work

#

because it would be infinitely nested

viscid spire
#

I wonder if you can do something with a union with Never

trim tangle
#

X | Never is the same as X

viscid spire
#

not inside a list subscript

trim tangle
viscid spire
#

list[Never] means an empty list

trim tangle
#

yes

viscid spire
#

perhaps type RSeq[T] = Sequence[RSeq[T] | Never]

trim tangle
#

That's the same as type RSeq[T] = Sequence[RSeq[T]]

viscid spire
trim tangle
#

"a sequence where each element is either RSeq[T] or Never"

viscid spire
#

oh right 🤔 because the list doesn't have to be the same shape throughout, so it already has that behaviour

trim tangle
#

You can use a type alias like this with a concrete type that you know isn't a list, like type DeepIntList = int | list[DeepIntList]

#

honestly i don't really understand where you can get a list of unknown depth

viscid spire
trim tangle
#

lmao

viscid spire
#

I had an intuition that something would work 😎

trim tangle
#

somehow i got this to work

image.png
viscid spire
#

wdym you got it to work

#

smh

trim tangle
#

oh wait, I had a different bug

#

it showed Type of x is list[list[int]] and then (variable) x: int. Probably just a caching bug

viscid spire
#

however

trim tangle
#

well, it should be RSeq[T] -> list[T]

viscid spire
#

ys

trim tangle
#

actually def flatten[T](xs: Sequence[RSeq[T]]) -> list[T]

viscid spire
#

true I suppose

#

although technically it works for non-sequence arguments

trim tangle
#

I wonder why this works with Sequence but not list

viscid spire
#

which changes the variance

#

Sequences are immutable in typing

rare scarab
#

to the type

#

lists are sequences, but are not immutable

#

Sequence doesn't provide any mutating functions

viscid spire
#

because the way they implemented it, it eventually passes a non-sequence

rare scarab
#

Maybe do *xs: RSeq[T]

trim tangle
#

Yes, if it actually worked like that, flatten("") would loop forever

viscid spire
# 
flatten(1)

is valid with their implementation

#

and returns [1]

#

because of the non-recursive case

trim tangle
#

while the signature might work for concrete types, it is going to give wrong answers if the item is generic

image.png
#

well, it kinda has to

viscid spire
#

🤨

#

the signature doesn't look sus at all

trim tangle
#

well, I can call foo(ys) where ys: list[list[list[int]]]

#

in that case flattened should be int, but it's inferred as list[list[int]]

viscid spire
#

interesting

viscid spire
#

well at least it's good for concrete types 🤷‍♂️ 😄

undone saffron
trim tangle
#

||markdown syntax moment||

#

easy way to remember: on some systems you have to add a space after the link or it won't work in parentheses

errant tulip
#

Why doesn't this work?
OpenAiKwgs = TypedDict("OpenAiKwgs", AsyncOpenAI.__init__.__annotations__)
Error:
Expected dict or keyword parameter as second parameter Pylance(reportArgumentType)

oblique urchin
errant tulip
#

is there any way I can just copy paste AsyncOpenAI types to my function parameter **openai_kwgs?

eg:

def __init__(self, role: str | None = None, **openai_kwgs: ??):
    ...
undone saffron
#

possibly, but I don't know how openai's types exist.

#

if they use **kwargs: Unpack[SomeTypedDict] you can do the same

fierce ridge
#

i feel like it should be possible, but maybe it introduces some kind of soundness problem

rare scarab
#

python types has no concept of this in typescript. ```ts
// defined in stdlib
type ConstructorParameters<T> = T extends (new (...params: infer P) => any) ? P : never;

type OpanAiKwgs = ConstructorParameters<typeof AsyncOpenAI>

fierce ridge
#

didn't know typescript had that. would be wonderful to have it in python.

rare scarab
#

It's not even a special type. It's just syntax.

#

all it needs is extends and infer syntax

undone saffron
#

The question is if that's a good thing, and I think python not having it is a good thing

#

Explicitly having your exposed parameters as a publicly typed typeddict for use with unpack signals whether or not a library is intended to actually work a specific way, and allows the same kind of re-use if you want to allow trivial reuse like that. Allowing any signature to just be copied can end up copying things that are only implementation details, and not part of what's intended to be supported

rare scarab
#

I usually use a custom decorator to copy function arguments

fierce ridge
#

i don't think it's a good idea to design around the 1% of interfaces that expose implementation details as function parameters that aren't meant to be touched, and in doing so obstruct a fairly common pattern in python from being type-checkable

rare scarab
#

sadly no way to extract a paramspec from a concrete function

fierce ridge
#

yup, i've been wanting that since before paramspec existed

rare scarab
#

i think we first need a Callable that supports kwargs

fierce ridge
#

example: i am writing a function that wraps matplotlib.axes.Axes.scatter and passes along kwargs. and matplotlib itself passes along those kwargs through 3 or 4 layers of other stuff. all of that is user-accesible! you really want me to copy-paste that huge signature? and you really want to force the matplotlib devs to maintain that?

rare scarab
#

not including Protocol

fierce ridge
#

@rare scarab why would that be a blocker here?

rare scarab
#

I'm now remembering type checkers already know how to handle functions properly.

undone saffron
#

this, in conjunction with the ongoing intersection work if accepted would be much more composable

fierce ridge
rare scarab
#

"internal" kwargs would be prefixed with _

undone saffron
#

yes, and they still get copied by anything that would copy types for reuse, so they become part of the API that is now potentially breaking if that kind of construct is added

fierce ridge
#

i've literally never seen that in a library

rare scarab
#

but the point of **kwargs: Unpack[...] is to be resilient to runtime changes.

#

besides, you should be ashamed if you use an internal argument and it breaks

fierce ridge
#

i guess, sure. i just feel like this is forcing an even greater divide between "typed python" and "untyped python", instead of making it easy to do the thing that people want to do, using existing code

#

so okay, it works well for matplotlib, which already uses **kwargs extensively. what about wrapping pd.DataFrame? do i need to copy all those overloads? etc.

undone saffron
#

I'm all for making it easy to do the right thing, I don't think blindly copying signatures is the right thing except in cases where the use is blind too, and have been actively involved in helping improve on ways to write the right thing. If the use is blind, you can do it with paramspec

oblique urchin
rare scarab
#

It wouldn't be much of an issue IMO if @override had the side effect of implicitly applying the parent's type hints.

fierce ridge
oblique urchin
undone saffron
#

If your function wraps (blindly) another library, you can already do it with paramspec

fierce ridge
#

i'd probably want to study how typescript does it before i can claim to have any good ideas

oblique urchin
#

Though there's the practical problem that you need some way to deal with positional arguments

#

Python signatures are complicated

rare scarab
undone saffron
#

you dont need to use a decorator

fierce ridge
undone saffron
#

you can do it via generics

rare scarab
#

can you share an example?

#

ParamSpec is generics

#

and for generics to have any meaning in a function, it has to appear twice

undone saffron
undone saffron
# rare scarab can you share an example?

I don't normally wrap libraries in a way where I'd do it quite this way, and I think we can find better ways to spell this kind of behavior, but I'm concerned about the library side of the equation of blind copying with non-blind use where providing types now means that private things are part of an API contract, and adding private things could potentially cause conflicts.

from collections.abc import Callable
from typing import Generic, ParamSpec, TypeVar

P = ParamSpec("P")
R = TypeVar("R")


class _WrappedBehavior(Generic[P, R]):

    def __init__(self, wrapped_function: Callable[P, R]):
        self._wrapped = wrapped_function
    
    def some_deferred_behavior(self, some_other_per_method_arg: int, *args: P.args, **kwargs: P.kwargs) -> R:
        # do something with our other args
        return self._wrapped(*args, **kwargs)


def example_func(x: int, y: int) -> int:
    ...


WrappedExampleFunc = _WrappedBehavior(example_func)
#

This clearly isn't easy to compose (could get easier than this with HKT support)

#

the problem comes in if you allow copying and non-blind usage for changing what could be considered breaking

#

(you can do similar with protocols and partials, and more as well, you don't have to use the generic pattern directly)

fierce ridge
#

@undone saffron i assume you'd also have to implement __call__ on _WrappedBehavior?

#

or is that what some_deferred_behavior represents?

undone saffron
#

up to you or not, it depends on what the purpose of deferring to another library is

fierce ridge
#

i see

undone saffron
#

I agree that there arent easy and ergonomic ways to spell the behavior you want, and I do hope we can find a way to improve that

fierce ridge
#

well that's at least a serviceable workaround. how does that work if example_func has overloads?

rare scarab
undone saffron
#

yes and no, the result is more flexible because you can attach other generic variables to this for more ocmplex use

fierce ridge
#

(like something from a 3rd party library, which is the case i had in mind)

undone saffron
#

it works fine with overloads with pyright, im not sure on mypy, but this is within specified behavior, so if it doesn't, that would be a mypy issue

fierce ridge
#

makes sense. well thanks for the example at least

undone saffron
#

I guess with copying the signature, we could just say "okay, allow copying funtion signatures as is, but it's the copier's responsibility to know they will never ever be broken by this" idk how I feel about that.

#

otherwise things that most people would not consider breaking suddenly become breaking, like even just adding an optional kwarg that's meant to be public, but coinceidentally conflicts with a kwarg of a wrapping library the main library had no knowledge of

fierce ridge
#

that is true

#

personally i still think it would be very useful for smaller and/or ad-hoc programs

#

you'd also have the same problem with @oblique urchin 's proposal of kwargs <-> typeddict, which i think is really useful for doing something like building up a dict of kwargs incrementally

pulsar axle
#

Am I stupid or this isn't correct??

image.png
viscid spire
#

what the func

pulsar axle
#

its the prompt() method from prompt toolkit

#

I'm very confused if they made a mistake or??

undone saffron
#

looks pretty intentional, but not an API design I would have made.

pulsar axle
#

Since I'm trying to typehint list[tuple[str,str]] but mypy screams at me since its not list[tuple[str,str] but that's not valid is it??

viscid spire
pulsar axle
#

ohh 😂 😂

viscid spire
pulsar axle
#

or wait

undone saffron
#

is that the actual typehint, or is that a generated inlay?

pulsar axle
#

is it because of union that this is valid

undone saffron
#

cause that looks fine as a generated inlay, but not a valid typehint

pulsar axle
viscid spire
pulsar axle
viscid spire
#

can you paste that typehint as text?

pulsar axle
#

yup

# 
 message: str | MagicFormattedText | list[tuple[str, str] | tuple[str, str, (MouseEvent) -> None]] | () -> Any | None = None,
           *,
#

sry terrible formatting

undone saffron
#

yeah, so (MouseEvent) -> None is rejected syntax

#

for Callable[[MouseEvent], None] which your other screenshots actually have

undone saffron
pulsar axle
#

yeah

undone saffron
#

ie. this isn't the actual typehint

viscid spire
# 
list[
    Union[
        tuple[str, str],
        tuple[str, str, Callable[[MouseEvent], None]],
    ]
]
pulsar axle
#

anhhhhhhhhhhhhhhhhh

#

now it makes sense lol

#

jesus

tranquil turtle
#

list is invariant, so list[A] is incompatible with list[A | B]

#

but in this case it should be, it can be fixed by replacing list with Sequence in their typehint
but in this case giving it list[tuple[str, str]] is fine, and it can be allowed by replacing list with Sequence in their typehint

viscid spire
#

shouldn't?

tranquil turtle
#

i doubt prompt_toolkit will mutate your list
so giving it list[tuple[str, str]] is perfectly fine

viscid spire
#

yes, so you meant to say "shouldn't" then?

#

it shouldn't be invariant

tranquil turtle
#

oh, right

#

english not englishing

viscid spire
#

actually it's ambiguous on what "it" refers to

#

so both "should" and "shouldn't" could be right...

undone saffron
#

changing the variance may not be the correct fix

#

lifting the union probably is may or may not be either

viscid spire
#

I can't parse that last sentence

undone saffron
#

It may not be correct to change it from list to Sequence, and allowing the equivalent of list[A] | list[B] | list[A | B] rather than list[A | B] might be more correct

#

idk how prompt toolkit is using it

tranquil turtle
#

well, actually they are doing isinstance checks, so replacing list with Sequence is not correct without changing the code: ```py

if value is None:
    result = []
elif isinstance(value, str):
    result = [("", value)]
elif isinstance(value, list):
    result = value  # StyleAndTextTuples
elif hasattr(value, "__pt_formatted_text__"):
    result = cast("MagicFormattedText", value).__pt_formatted_text__()
elif callable(value):
    return to_formatted_text(value(), style=style)
elif auto_convert:
    result = [("", f"{value}")]
else:
    raise ValueError(
        "No formatted text. Expecting a unicode object, "
        f"HTML, ANSI or a FormattedText instance. Got {value!r}"
    )
viscid spire
#

smh

pulsar axle
#

Just a small question, since im really unsure how I should really be using typehints

#

Lets say you have a var that is obviously a string as suhc :

my_var = "hello"
#

People don't do

my_var: str = "hello"

right?

oblique urchin
pulsar axle
#

Or something like this

parts = command.split(" ")
parts2: list[str] = command.split(" ")
#

Because I feel like it becomes "bloated" if I do that

oblique urchin
#

in Python you usually only annotate variables if the type is otherwise ambiguous, e.g. if it's an empty list and you need to tell the type checker what types go into the list

oblique urchin
pulsar axle
#

hmmm gotcha

#

What about functions? I feel like that's useful (even if obvious)

#

(func args and return type)

oblique urchin
# pulsar axle hmmm gotcha

Always annotate function arguments. For return types, one popular type checker (pyright) always infers return types, but other type checkers generally expect you to annotate return types too

pulsar axle
#

yeahh I'm using mypy

oblique urchin
#

I feel it's useful to annotate return types because it makes your code more self-documenting, but not everyone agrees

pulsar axle
#

alright

#

thanks 😄

undone saffron
#

I think it's also useful to annotate them as it also ensures no accidental API changes

pulsar axle
undone saffron
#

yeah. If the rest of your code happens to work with a more lax inferred return type, that isn't always a good thing. Being explicit about it enforces consistency with the return type annotation, which means code you can't see (like users of library code) have something consistent.

pulsar axle
undone saffron
#

I leave some small things to inference, but never anything publically exported

pulsar axle
#

great example is, maybe doesn't matter that much, but I had return statements returning 0 and None within same method

#

and mypy screamed at me lolol

tranquil turtle
#

i find it very useful that pyright is smart enough to infer a lot of stuff
it is very convenient when writing dirty PoC prototyping, mypy usually either screams at you or just infers unannotated args as Any

undone saffron
#

for sure. when prototyping, the inference is great and can prevent you from being more restrictive than you needed to be. you can often later just take what it infered and then add it as an annotation later.

#

very much a case of different levels of explicitness being more appropriate at different stages of development

fair pendant
#

how do I type hint that an argument is a class? not an instance of the class

chrome hinge
fair pendant
#

I tried that, but whence do I import Type then?

chrome hinge
#

it's in typing

#

!docs typing.Type

rough sluiceBOT
#
typing.Type 

class typing.Type(Generic[CT_co])```
Deprecated alias to [`type`](https://docs.python.org/3/library/functions.html#type).

See [The type of class objects](https://docs.python.org/3/library/typing.html#type-of-class-objects) for details on using [`type`](https://docs.python.org/3/library/functions.html#type) or `typing.Type` in type annotations.

New in version 3.5.2.

Deprecated since version 3.9: [`builtins.type`](https://docs.python.org/3/library/functions.html#type) now supports subscripting (`[]`). See [**PEP 585**](https://peps.python.org/pep-0585/) and [Generic Alias Type](https://docs.python.org/3/library/stdtypes.html#types-genericalias).
chrome hinge
#

huh, I didn't realise type replaced it, actually. weird.

fair pendant
#

glad I could help enlighten you today

#

bot removed my 💙 so have two 💙 💙

trim tangle
fair pendant
#

I'm trying to get rid of some code duplication in django. I have two very similar methods that I'm compressing into one

#

I'm not convinced this is a good idea but I have nothing else on my plate so might as well explore it

#

the classes are models and forms

#

the model classes are used as parameters, and the forms are not but they are instantiated differently depending on get/post

granite plover
#

If I dont have access to pydantic/mypy/x validation lib, what are my options?

isinstance and other checks during runtime?

trim tangle
rare scarab
#

Did I just find a pyright bug? ```py
def foo(x: type, y: Any):
if dataclasses.is_dataclass(x):
# Second argument to "isinstance" must be a class or tuple of classes
# Protocol class must be @runtime_checkable to be used with instance and class checks
if isinstance(y, x):
pass

#

I may need to update my python extension...

soft matrix
#

Isn't is dataclass true for instances and classes?

#

Oh actually idk actually

oblique urchin
rough sluiceBOT
#

stdlib/dataclasses.pyi line 219

def is_dataclass(obj: type) -> TypeGuard[type[DataclassInstance]]: ...```
rare scarab
#

Yeah, I just need to update vscode/python

#

it was fixed in pyright 1.1.337

granite plover
trim tangle
granite plover
#

This is mostly about function arguments really

#

Still only have isinstance checks, right?

trim tangle
#

If this is your own code, you should rather use a static type checker like mypy or pyright. It doesn't need anything at runtime, it just looks at your code and points out errors/diagnostics

#

If this is library code and you want to check inputs from other people, then yes, you need to do a bunch of isinstance checks

rare scarab
#

you may also be able to use match

trim tangle
viral glacier
granite plover
oblique urchin
#

use an old version of mypy 😛

granite plover
#

Also we dont have ownership of the images we use or deployment

oblique urchin
#

using mypy wouldn't require any runtime changes

#

but it's been several years since mypy dropped py2 support. It's probably still possible to install the old version but you're very much on your own

granite plover
#

mypy i could use but what i need is something like pydantic really

#

Mypy would be fine if everyone used it but i cant force others to sadly

grave fjord
tranquil turtle
# 
import os
assert not os.system('mypy --strict ' + __file__)
#

paste this in every file

oblique urchin
granite plover
#

Maybe getting out of this hellhole is the play
I'll stick to isinstance spam for now

oblique urchin
#

as well as not being on Python 2

cedar sundial
#

I don't think this is yet possible but I will ask to make sure anyway. I'm looking to dynamically create a list of filenames from a path that can be used in a Literal type. Is there a way of doing that?

chrome hinge
#

That'd require your type checker to execute nontrivial code at typechecking time. Perhaps you can make a mypy plugin like that, but I don't think there's any way in the type system itself.

oblique urchin
terse sky
#

I just upgraded my mypy version

#

running mypy --ignore-missing-imports as before

#

but now I'm getting tons of import-untyped errors

#

when I google how to suppress it, main thing I see people suggest is --ignore-missing-imports

#

Like, I guess this is because it knows there are stubs. Which I appreciate it, but I want to first get this passing, and then be able to add stubs later

oblique urchin
#

we now recommend --disable-error-code=import-untyped

#

instead of --ignore-missing-imports

terse sky
#

thanks, i'll give that a shot

#

fwiw I was looking here

#

didn't see that mentioned

#

man python changing its mind on annotating optional = None stuff 🤕

tranquil ledge
terse sky
#

For a function argument

#

foo would be treated as Optional[int]

#

This was like an officiK thing

#

Then they made it officially not a thing 🙂

tranquil ledge
#

Ah, right. I feel like I have a vague memory of that behaviour undergoing a deprecation period.

terse sky
#

Yeah

#

We have piles of code with that now

tranquil ledge
#

Might be a bit annoying to add Optional to everything, but you could probably get away with using a regex find and replace that's eyeballed by people to determine if it's valid, right?

terse sky
#

A regex isn't a bad idea though I'm pretty terrible at them

#

Maybe I can figure it out though

#

Thanks for the suggestion

trim tangle
# 
(?!.*Optional)[^:]+:.*\s*=\s*None
terse sky
trim tangle
#

nope, that's PCRE (close to what python uses)

#

grep -P should use that

#

I normally use the built-in search in VSCode

terse sky
#

How would you apply this to fix the whole codebase at once thrn? Write a python script?

#

I see

oblique urchin
#

I think Ruff has an autofix for this

terse sky
#

Thanks I'll check out Ruff

bleak imp
#

I'd do it with python since why not, re.sub(r"(?m)^([ ]*\w+: )(\w+)( = None)$", r"\g<1>Optional[\g<2>]\g<3>", file)

terse sky
#

another really weird error, I'm being asked to type annotate a variable that's just

client = pymongo.MongoClient(...)
db = client["..."]  # the only usage of client
#

It's literally just calling a class init here so I'm confused why it would want an annotation

oblique urchin
terse sky
#

ah interesting

pulsar axle
#

Is it normal that mypy isn't screaming at me when I don't use typehinting within func declarations? I read somewhere something about this, but not really sure honestly (I just want to make sure I'm using mypy correctly) (using it via pycharm plugin)

oblique urchin
pulsar axle
#

👍 would using something like --strict flag be recommended ?

oblique urchin
pulsar axle
#

oki thanks

#

another question, I find this quite ugly, I'm unsure if theres a better way to do this? Or is it normal
self.results: list[dict[str, list[dict[str, str]] | str]] = []

oblique urchin
pulsar axle
viscid spire
terse sky
#

Pretty sure it used to be okay

#

A past version of this PEP allowed type checkers to assume an optional type when the default value is None, as in this code:

#

It was the spec until it wasn't 😛

oblique urchin
terse sky
#

Sorry, as Optional

oblique urchin
terse sky
#

Gotcha

#

I mean I'm not against the core idea here

#

But the benefits seem small and backwards compatibility break is pretty annoying

#

If mypy didn't have a flag I'd be more irritated but as it is I guess it's nbd

oblique urchin
#

this was six years ago, for what it's worth. There was a lot less typed Python code at the time

terse sky
#

We probably got the memo late

hallow flint
abstract plaza
#

Hi folks, I'm currently struggling with some type hints, Unions, and get_args. My example basically comes down to this: https://mypy-play.net/?mypy=latest&python=3.12&gist=d565dd4bb160a457c5eaf2097256f06d . In this small test case mypy tells me everything is fine (just as expected), however, in my project it gets stuck at the assert_never stating, that the types in the Union are not covered. The only difference that I could spot so far is, that my Union definition is in a separate module. Any pointers on what's wrong or what I can do to debug this issue?

mypy Playground

The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.

viscid spire
#

so I think maybe you didn't reduce the problem correctly

abstract plaza
#

and suddenly it stops working

#

thanks so far. 🙂 so any ideas how I can solve this issue? isn't this the correct use uf get_args?

viscid spire
#

pylance infers it correctly it seems, or rather differently maybe...

#

hrmm

#

pylance says Argument of type "A" cannot be assigned to parameter "__arg" of type "Never" in function "assert_never"
and mypy says Argument 1 to "assert_never" has incompatible type "Any | D"; expected "NoReturn"

#

I do not know why they give different answers for the error

#

what the heck 🤔

image.png
#

ok very interesting, if I remove any of the code that was below that in the loop, then the type becomes B | C

#

(btw the issue is not tuple vs. union in the isinstace, I checked that it did not make a difference)

#

I think pylance is smoking something

image.png
#

oh I see

image.png
#

pylance being greedy with inferring a narrower type. However this doesn't say much about MyPy's behaviour

viscid spire
#

the revealed type is builtins.tuple[Any, ...]

#

and I think I know why

#

I heard something about this being removed from MyPy (because it's not spec)

Union[*Ts]
#

so get_args has no mechanism

# 
from typing import reveal_type, get_args
reveal_type(get_args)

Revealed type is "def (tp: Any) -> builtins.tuple[Any, ...]"
#

sorry to ramble 🤭

tranquil turtle
viscid spire
abstract plaza
#

Visiably its much more pleasing, but this still causes the same issue :/

viscid spire
#

oh nvm I understood now

#

I can't test any more because I've left my computer for the night

abstract plaza
#

then I ignore the type checks for this for now, thanks for your input folks 😉

viscid spire
#

I'll look again tomorrow unless someone else solves it

undone saffron
#

pyright handles this fine

undone saffron
undone saffron
viscid spire
#

I believe mypy previously had it as Callable[[Union[*Ts]], tuple[*Ts]]

#

Which isn't spec

undone saffron
trim tangle
#

Why was Union[*Ts] not approved btw?

abstract plaza
#

(or, more precisely, the examples)

undone saffron
oblique urchin
terse sky
#

Thoughts on mypy vs pyright: what do folks view as the pros and cons of each at the moment?

undone saffron
undone saffron
#

Yes, but there's a question of whether or not you should. pyright (the type checker vscode's language server leverages) has configuration settings that will prevent implicit any, meaning you can stil use things like json.load, but into a known structure

#

I like pyright, but realistically you should be able to just use any type checker that is working on being conformant to the specification. mypy, pyright, pyre, and pytype all are, but have varying levels of what is and isn't supported currently.

cunning plover
#

it should isolate the scope of any escapings no further than single function

hallow flint
terse sky
#

I'm not a big fan of literals generally speaking; I do recall now some people posting examples of stuff with pyright and literals that I would just avoid completely

#

i also remember pyright getting strict on optional earlier (IIRC) and supporting recursive unions earlier; though it seems like mypy has all the stuff I can think of that I care about fo rnow.

#

the vscode integration is nice

hallow flint
#

mypy has had strict_optional=true as default for longer than pyright has been around. maybe you're thinking of no_implicit_optional, which yeah, i agree that default was bad legacy which is why i changed it 🙂

#

definitely recommend starting your mypy config off with strict=true and then adding options that reduce strictness, rather than other way round...

stone harness
#

How can I make the typehints correct?
if you use generic type

from abc import ABC, abstractmethod
from pydantic import BaseModel


class AbstractUseCase(ABC):
    @abstractmethod
    def execute(self, params: BaseModel) -> BaseModel:
        pass


class Param(BaseModel):
    user_id: str


class Result(BaseModel):
    user_id: str


class UseCase(AbstractUseCase):
    def execute(self, params: Param) -> Result:
       ...
undone saffron
inland radish
#

dd

#

s

#

d

undone saffron
#

Right. What I was getting at is that forbidding every single Any probably isn't productive, and it's better to be sure you don't have Accidental uses of Any. json.load is typed as returning Any because it can't know the type. This isn't a mistake, and if you know the actual type with certainty, you're allowed to use this:

e.g.

some_mapping: dict[str, int] = json.load(some_fp)

But if you don't actually know the type, you'd have no way to statically analyze it in the first place.

Any is a useful part of the type system, and it's usually better to look at it as defining the boundary between what you know about incoming data and not than to try and remove it entirely.

If you have an Expected type that you don't trust, you can instead use a library like msgspec to efficiently parse into a typed struct, but this falls under not removing Any entirely, and it being part of the boundary. This just moves that boundary into a library that handles that for you.

undone saffron
#

Not neccasily. Do you actually know the keys? Some json payloads might be dynanic, and return a mapping between things like strings representing job ids, and some value. Not really important to the point I was getting at there though.

Avoiding Any for the sake of avoiding Any may make your code worse to interact with. If you instead use the places you have Any to handle and signify the boundary between things that come in untyped, and then handle validation as needed, you end up with good results.

Another more obvious example of this comes with sql

row: tuple[str, int] = cursor.execute(
    "SELECT memo, amount FROM deposits ORDER BY post_date DESC LIMIT 1"
).fetchone()

If this returns a tuple[Any, ...], the user's annotations just work and this can be assumed to be fine, because presumably your database has strict typing,

#

Well, you still want to provide a type where you have it, but you don't need to avoid Any to do that. Both of the examples above show using something that gives you something typed As Any, and providing a more detailed type where you know it.

#

In a case where you have potentially hostile input (Such as user facing web apps) you'd want to actually validate it

#

so there's no one size fits all option, but you can view Any as a way to see what hasn't been validated yet, and pick the best option for your use to go from that at the boundary of your application, to typed for use in the rest of it

#

(There are also cases where the type system may not have a way to express the type of what you have, but that doesn't apply to the example you have here)

fierce ridge
#

For JSON specifically you can at least write a "read-only" JSON data type:

JSON_ro: TypeAlias = Mapping[str, "JSON_ro"] | Sequence["JSON_ro"] | str | int | float | bool | None

json.loads of course is annotated to return Any but you can always assign an Any to a variable of any type.

from here, you can take the "parse, don't validate" approach:

from collections.abc import Mapping, Sequence
from enum import StrEnum
from numbers import Real
from typing import TypeAlias

import attrs


JSON_ro: TypeAlias = Mapping[str, "JSON_ro"] | Sequence["JSON_ro"] | str | int | float | bool | None


class Color(StrEnum):
    red = "red"
    blue = "blue"


@attrs.define
class Widget:
    size: float
    color: Color


def parse_widget(txt: str) -> Widget:
    data: JSON_ro = json.loads(txt)
    if not isisntance(data, Mapping):
        raise TypeError("Widget must be a JSON object.")

    try:
        w_size = data["size"]
    except KeyError:
        raise KeyError("'size' is missing from input data.")
    if not isinstance(w_size, Real):
        raise ValueError("'size' is not a real number.")

    try:
        w_color_str = data["color"]
    except KeyError:
        raise KeyError("'color' is missing from input data.")
    w_color = Color(w_color_str)  # Raises ValueError if invalid

    return Widget(
        size=w_size,
        color=w_color,
    )
#

of course that's a lot of boilerplate. this is what pydantic, marshmallow, mashumaro, etc. are good for

#

this is often seen in the OO world with patterns such as alternate constructors. in python you could write this if you like the OO style:

from typing import Self

@attrs.define
class Widget:
    size: float
    color: Color

    @classmethod
    def parse_widget(cls: type[Self], txt: str) -> Self:
        ...  # all the parsing stuff
        return cls(
            size=w_size,
            color=w_color,
        )
terse sky
#

usually better to use a dataclass than either of those, tbh

#

unless you have some already existent code that you know works, that uses dict, that you just want to add type annotations to

#

And yes, you should avoid Any, even if you have "untyped" json, you can still type it better than Any, as salt rock's example shows

#

but most json isn't really untyped; in most cases you ought to be able to simply define a dataclass/attrs class, parse directly into that, and use it

#

and definitely look into a framework as salt rock suggests

fierce ridge
#

i think worrying too much about avoiding "Any" in this case is quibbling over details. if you read from a pickle file for example, the return type really is Any. your job is not to avoid it entirely, but to contain it. the Any should not "escape" from the immediate location where the data is loaded and parsed.

terse sky
#

Sure, for reading a pickle file

#

we were talking about json though

#

in your example above, I agree that JSON_ro adds little, but then that whole parse_widget function is basically, as you said, boilerplate that you should just handle via a deserialization framework

fierce ridge
#

well the point is that JSON_ro is barely any better than Any

#

and you'd write basically the same code in the rest of the function if you wrote Any instead of JSON_ro

#

the key is to contain the wide-open unknown type, and prevent it from leaking out into the rest of your application

#

honestly i'd suggest writing out the manual parsing code by hand, or at least sketching an outline, before starting up with pydantic. it's very easy to get caught up in all the pydantic details (especially in v2, there's a lot of complexity there) and lose sight of what you're actually trying to accomplish

terse sky
#

and you'd write basically the same code in the rest of the function if you wrote Any instead of JSON_ro
and this, frankly, is missing the point of static typing

#

the correct code looks basically the same, the question is how it helps you catch incorrect code

#

with the "proper" json type, if you don't validate that the value you extract is of the correct type to go into Widget, mypy will error out, which is kind of the whole point of using mypy to begin with

#

if you use Any or dict[str, Any] then it will not complain, and you could return a Widget that has all its fields populated, but with some of them being of incorrect type.
If you're fine with a parse_widget than can return an incorrectly typed Widget, then yes, the value is much lower

trim tangle
#

maybe have json.loads return object and not Any

fierce ridge
#

in my particular case if you wrote data: Any = json.loads(txt) instead of data: JSON_ro = json.loads(txt) you'd leave the rest of the function unchanged because it's immediately followed by the isinstance(data, Mapping) check

terse sky
#

I mean, again, this is missing the point of static typing 🤷‍♂️

fierce ridge
#

unless i'm missing some detail here, i'm pretty confident mypy will perform the same narrowing to Mapping[Any, Any]

#

so in this particular example yes, JSON_ro is better, but functionally Any is not substantially more problematic than JSON_ro, because JSON_ro is wide open and provides almost no practical type safety

terse sky
#

If you forgot this

    if not isinstance(w_size, Real):
        raise ValueError("'size' is not a real number.")

then mypy will not be able to help you, if you type the json as Any. If you type it with a recursive union, mypy will raise this as an error.

#

mypy catching errors for you is good

#

Typing this correctly isn't really more work than typing it as Any... so in the absence of any downside, and with a minor upside, you should type it correctly 🤷‍♂️

#

I'm not saying you need to pathologically avoid Any

fierce ridge
#

oh i see what you're saying. sure, yeah, JSON_ro helps a little because you know the keys and values can only be strings or numbers, not functions or compiled regexes etc.

terse sky
#

yes

#

it helps a little and hurts not at all

fierce ridge
#

yes, i agree with that

terse sky
#

All I really mean is that usually there's a better type than Any, even if it's only a little better. not always though, I happily concede that

fierce ridge
#

yeah, agreed there as well

terse sky
#

its' just cool that we can actually do this now :-). I'm upgrading from an old mypy, and generally an old python ecosystem so it's nice to gain access to stuff like this.

terse sky
#

what you'r edescribing is the differnece between JSON_ro and say, dict[str, object]

#

Any is not object though, Any basically disables type checking. So it's not really about narrowing from anything to strings/numbers etc
It's about "don't disable type checking"

fierce ridge
terse sky
#

Yeah, that's not what Any is, for better or worse. I get why it's named ANy and object already existed, but it is pretty confusing

#

especially when multiple other languages have Any that is the same or closer to python's object

#

Maybe a better name for Any would have been Untyped, because it emphasizes that you're disabling static type checking, rather than making people think "it can be any type" (which is really object)

viscid spire
#

Treat object as a union of all types, and Any as an intersection of all types

terse sky
#

what does "intersection of types" mean though

#

i guess the closest thing to that is a bottom type

#

Any isn't really that either, if it was an intersection of all types, you could assign to any type from an Any, but you won't be able to assign to an Any from anything else

#

(so I guess, what you're describing is pretty much exactly a bottom type)

fierce ridge
#

you could write data: object = json.loads(...) instead of : Any

terse sky
#

Any is basically simultaneously a top type and a bottom type, which obviously doesn't really make a lot of sense in terms of type systems

fierce ridge
#

or equivalent when loading from pickle, xml, etc.

terse sky
#

hence why it amounts to "disable type checking"

#

yeah, returning object would be "better" but you could argue that the type system in that case just isn't informative, so it would just make you add boilerplate like typing.cast

#

at least, I guess some python folk would argue that way

fierce ridge
#

yeah, it would be a disruptive change

#

but you could/should specifically annotate with : object most of the time you mean : Any as long as you're willing to cast, or assert/if with isinstance()

terse sky
#

For sure, from a backwards compatibility perspective, from a perspective of, even now, people often write untyped python and migrate it, it makes less sense

#

Yes, certainly

fierce ridge
#

i am very sympathetic to the goal of supporting untyped python and avoiding making it harder to write

terse sky
#

i personally have basically no interest in untyped python, outside of the REPL and interactive data analysis.
For me I kind of see that as the only reason I would ever want dynamic typing in general; I can't think any other work I've done with programming where being dynamically typed is something I prefer.

#

but I do recognize there are real world considerations here

#

(to be clear, it wasn't always this way, in C++03 with verbose typing, I felt differently... but with more modern type systems I've never really felt like the type system is enough of a headache to justify this)

oblique urchin
terse sky
#

Really my main wish is that typing in python felt less tacked on, and that python was more expression oriented, which would gie more opportunities for inference

#

but again, that's nobody's fault, I recognize engineering realities

fierce ridge
#

@terse sky have you tried the Nim language?

terse sky
#

I've done a fair amount of hobby rust lately and I'm really start to get used to its style of type inference where it can combine restrictions from anywhere it's used

#

it's a bit hard to go back

#

when you use that though it's really rare you need any local type annotations

bleak imp
terse sky
#

(beyond ones that actually specify intent)

fierce ridge
#

fwiw mypy is particularly bad at type inference

#

just about any modern language does it better, and some not-modern ones as well

terse sky
#

idk, Go's type inference is probably more basic than mypy 🙂

#

Kotlin is also relatively simple, but still better than mypy

terse sky
# fierce ridge <@512649489300062228> have you tried the Nim language?

I haven't. People do bring it up periodically, but it's still really below the threshold for momentum where I would be interested in a language.
it has a bunch of weird (IMHO) choices you hear about immediately, which aren't like intrinsically a big deal per se but kind of send the message of it not being a serious project

#

sounds harsh but that's my 0.02

#

obviously time is very limited to randomly learn new programming languages

fierce ridge
# terse sky obviously time is very limited to randomly learn new programming languages

agreed on all of the above. if you do want to try it for writing a small script or toy program, you might find it interesting. i agree that there are some "weird" elements which might permanently prevent its adoption in industry beyond a certain point, but it's also a tremendously practical language and even after just writing a few programs i find myself wishing i could use it all the time (for non-data-science work at least)

terse sky
#

(or gradually typed/optionally typed)

fierce ridge
#

but its type inference is excellent and you often don't need to write types at all

terse sky
fierce ridge
#

however it compiles to C, C++, or JS which is kind of cool, and it's meant to have particularly good C interop

#

also the C/C++ compiler target provides several GC options, which i believe JVM and D also have

#

"no GC" i think is even an option

rustic gull
#

I'm having a bit of a weird situation. I want to represent a async function like this Callable[[int, str], float] but is Callable really the right type for that, is there not a built in type for asynchronus functions?

oblique urchin
rustic gull
#

AH

#

I see, that's right

#

Thank you

solemn sapphire
#

How would you write typings for a opaque object?
For example selector.register takes in a opaque object and that can be whatever I want. I'd like it to be some concrete type, I was thinking doing something like

class SocketReady:
    def __call__(self, *args, **kwargs):
        self.cb(*args, **kwargs)
    def __init__(self, callback: Callable[[FileIO, int, Client], None]):
        self.cb = callback

and register this, but its kinda verbose :p (and feels ugly), I was wondering if there was a shortcut.

#

or is this reasonable?

muted iron
#

Generic?

  class SocketReady[**P]:
      def __call__(self, *args: P.args, **kwargs: P.kwargs) -> None:
          self.cb(*args, **kwargs)
      def __init__(self, callback: Callable[P, None]) -> None:
          self.cb = callback
#

Object with init and call only makes me think it should be a function / a functools.partial function.

solemn sapphire
#

for example if I have the following where self.selector is a selector

for key, events in self.selector:
    key.data(ctx, events)

Now, how might I go about statically enforcing at every self.selector.register site to adhere to this signature if they decide to pass a data, anything else should be a type error

Python documentation
selectors — High-level I/O multiplexing

Source code: Lib/selectors.py Introduction: This module allows high-level and efficient I/O multiplexing, built upon the select module primitives. Users are encouraged to use this module instead, u...

#

making it generic makes it that any signature is allowed, I want to allow only a particular signature.

elfin crane
#

is there any way to have a any type but disclude a type. Say for example i want all types except for string

#

im using python 3.12

#

and this has changes which i don't really know how they look

#

something like

#

any | (not 'type') (ofc ik this doesn't work)

chrome hinge
#

is there any wya to have a any type but disclude a type
nope

elfin crane
#

oh

trim tangle
#

why do you want this btw?

chrome hinge
#

i think there's no plans to actually add it; it's a big change for the type system.

elfin crane
# 
@dataclass
class SinglyNode():
    data: any
    point: 'SinglyNode' = None
#

i don't want data to be a SinglyNode

viscid spire
#

Why not?

#

Also that any is wrong

#

That refers to the any function, not typing.Any

elfin crane
#

why does it work then?

viscid spire
#

Because typehints only need a valid expression

#

and a defined variable is a valid expression

elfin crane
elfin crane
viscid spire
#

although that class should probably be generic

elfin crane
#

mhm

tranquil ledge
# solemn sapphire How would you write typings for a opaque object? For example `selector.register`...

This likely isn’t what you’re looking for, but another way of writing that code snippet would be with a function closure, as well as restricting the args and kwargs to a more specific signature. That being said, you could also restrict the call signature on your class there.

from typing import TypeAlias


MySig: TypeAlias = Callable[[FileIO, int, Client], None]

def socket_ready(callback: MySig) -> MySig:
    def call_callback(fileobj: FileIO, events: int, data: Client): -> None:
        return callback(fileobj, events, data)
    return call_callback
storm cloak
#

I'm experimenting with dynamically generating .pyi files for things like URL paths, .etc. Are there any existing project that do this?

chrome hinge
storm cloak
#

I'm making a templating library similar to Jinja, I wanted to provide auto complete for template name files since they're easy to mess up.

#

I tried dynamically generating the python files before, but that was error prone. So I'm experimenting with generating type stubs instead

#

Bunch of typing overloads to specify the template parameters based on the template name.

solemn sapphire
# tranquil ledge This likely isn’t what you’re looking for, but another way of writing that code ...

as well as restricting the args and kwargs to a more specific signature

Looks like doing this raises a type error if I pass a kwarg?

from typing import Callable
from socket import socket

class Client:
    pass

MySig = Callable[[socket, int, Client], None]

class SocketReady:
    def __call__(self, *args, **kwargs) -> None:
        self.cb(*args, **kwargs)

    def __init__(self, cb: Callable[[socket, int, Client], None]) -> None:
        self.cb = cb

def socket_ready(callback: MySig) -> MySig:
    def call_callback(fileobj: socket, events: int, data: Client) -> None:
        return callback(fileobj, events, data)

    return call_callback

def ready_func(fileobj: socket, events: int, data: Client) -> None:
    print(fileobj, events, data)

with socket() as sck:
    # This is fine
    SocketReady(ready_func)(sck, 0, data=Client())
    # but this raises an error on pyright and mypy
    socket_ready(ready_func)(sck, 0, data=Client())
tranquil ledge
#

Huh. I guess a callable protocol would need to replace that Callable annotation, then.

from typing import Protocol

def MySigProto(Protocol):
    def __call__(fileobj: socket, events: int, data: Client) -> None:
        ...

def socket_ready(callback: MySigProto) -> MySigProto:
    ...
solemn sapphire
#

Interesting, that does work

#

actually that does make sense, the callable only types the positional arguments and doesn't give names to those arguments

zenith plaza
#

hi

#

im trying to write a validator for the following yaml:

shared_documents:
    Foobar:
      fields:
        iuiyi:
          type: date
        fgfdgf:
          type: keyword
        gfgdg:
          type: keyword
        foobar:
          type: keyword
          fields:
            partial:
              type: text
              analyzer: standard
    FoobarDocumentType:
      fields:
        blah:
          type: date
        nothing:
          type: keyword
        testfield:
          type: keyword
          fields:
            partial:
              type: text
              analyzer: standard

With the following model I am unable to capture the multifields (the ones that have a nested fields keyword):

class ElasticFieldDetail(BaseModel):
    type: str
    analyzer: Optional[str] = None

class ElasticProperty(BaseModel):
    type: str
    fields: Optional[Dict[str, ElasticFieldDetail]] = None

class SharedDocumentDefinition(BaseModel):
    fields: Dict[str, ElasticProperty]

class Config(BaseModel):
    shared_documents: Dict[str, SharedDocumentDefinition]
#

using latest pydantic stable

#

originally I tried with:


shared_documents:
    Foobar:
        iuiyi:
          type: date
        fgfdgf:
          type: keyword
        gfgdg:
          type: keyword
        foobar:
          type: keyword
          fields:
            partial:
              type: text
              analyzer: standard

(Note the removal of 'fields' at root) With pydantic.RootModel: Dict[str, ElasticProperty] but no luck.

fierce ridge
#

or the otherwise undesirable output

delicate kiln
#

Does anyone use Sphinx for docs on a project with Generic classes? It seems that Sphinx wants you to document your TypeVars, otherwise there is no record of what the bounds and variance are, but that feels really ugly and like I'm saying that the TypeVar's are public.

zealous skiff
#

Hello Folk - anyone here consider themselves an expert in pydantic or dataclasses? i have a question on typehinting and the init function

#

basically - how tf do you dynamically create a kwargs init from just attributes and typehints

trim tangle
zealous skiff
#

kinda both i suppose, its a blury field for me

#

the question is "how does one metaprogram an __init__ method with kwargs mapped to attributes from only the class attributes and their typehints"

#

so both typing and runtime metaprogramming

trim tangle
rough sluiceBOT
#
inspect.get\_annotations 

inspect.get_annotations(obj, *, globals=None, locals=None, eval_str=False)```
Compute the annotations dict for an object.

`obj` may be a callable, class, or module. Passing in an object of any other type raises [`TypeError`](https://docs.python.org/3/library/exceptions.html#TypeError).

Returns a dict. `get_annotations()` returns a new dict every time it’s called; calling it twice on the same object will return two different but equivalent dicts.

This function handles several details for you:
trim tangle
#

You can get annotations through this^

zealous skiff
#

yeah getting the annotations isnt the hard bit, i have tried pulling from instance._attributes and instance.__annotations__

#

its the defining the init based on them im puzzzled by

trim tangle
zealous skiff
#

but the typehinting on that __init__ is just a dict

#

id like to have the **kwargs explicitly named and typed, like dataclasses and pydantic do

trim tangle
#

!d typing.dataclass_transform

rough sluiceBOT
#
typing.dataclass\_transform 

@typing.dataclass_transform(*, eq_default=True, order_default=False, kw_only_default=False, frozen_default=False, field_specifiers=(), **kwargs)```
Decorator to mark an object as providing [`dataclass`](https://docs.python.org/3/library/dataclasses.html#dataclasses.dataclass)-like behavior.

`dataclass_transform` may be used to decorate a class, metaclass, or a function that is itself a decorator. The presence of `@dataclass_transform()` tells a static type checker that the decorated object performs runtime “magic” that transforms a class in a similar way to [`@dataclasses.dataclass`](https://docs.python.org/3/library/dataclasses.html#dataclasses.dataclass).

Example usage with a decorator function...
trim tangle
zealous skiff
#

hmm, looks rather complicated for what i was aiming for, thanks for the pointers tho

#

ill need to go read through this properly to get whats going on

#

final question: do you think something like this could be done using metaclasses?

trim tangle
#

in any case, type checkers have to hard-code this behaviour somehow (in the form of the dataclass_transform decorator)

crisp steppe
#

@zealous skiff what are you trying to do?

#

something like generating data for property testing?

zealous skiff
#

id like to enrich my __init__(key_1: str, record_init:bool=True, **arguments) method, but rather than the **kwargs equivalent i want to type it more strongly

#

there are a few other parameters in the init which control how the class behaves wrt persistance and such but the data parameters all come from the **arguments

#

(the __init__ is in an abc at this point too, the models that inherit just use it and the abc calls setattr naively )

crisp steppe
#

and you want to catch this at typecheck time? pycharm will warn you if you try to use Foo(not_a_field=7) not sure if mypy or pylance will also warn on this but i should hope so!

zealous skiff
#

yeah its really for the vs code annotations - as its rather generic type checkers have a tough time picking it up

# 
for n, v in attributes.items():
            if n not in self.get_attributes():
                raise ValueError(f"Attribute {n} specified does not exist"
            setattr(self, n, v)

so type hinting dosent have much hints

crisp steppe
#

maybe just use pycharm 😁

#

it has even better vim bindings too

zealous skiff
#

tbh i moved away from pycharm as its not as concrete with its types ad pylance

#

for example not highlighting the fact the get() may return None and the split() would then fail

Screenshot_2024-02-19_at_21.45.26.png
#

pylance picks this up and tells me im an idiot

Screenshot_2024-02-19_at_21.46.40.png
#

maybe im missing an option in the pycharm settings

crisp steppe
#

you just need to install the mypy plugin i guess

image.png
zealous skiff
#

i do have it installed - maybe ill give it another go 🤷‍♂️ mypy is pretty decent - even if configuring it to be "just pedantic enough but not too much" is a pain

crisp steppe
#

Back to type level questions: What's the best way to parse a type and determine if it's a Union? Currently I'm checking the type name and seeing it's UnionType or _UnionGenericAlias. But I would prefer a more codey solution like issubclass when checking if list[str] is a list

zealous skiff
#

on a typedef or on an instance?

zenith plaza
#

@fierce ridge empty values for the shared_documents components

lunar dune
crisp steppe
#

oh right. phind.com was telling me to use .get(__origin__) but __origin__ isn't a a member of the type. get_origin just uses

if isinstance(tp, types.UnionType):
        return types.UnionType

so i think i'll try that. 😁

cunning plover
# zealous skiff the question is "how does one metaprogram an `__init__` method with kwargs mappe...

Since python 3.12 we can use TypedDict for more precise **kwargs typing
https://peps.python.org/pep-0692/

https://docs.python.org/3/library/typing.html#typing.TypedDict
Together with features like NotRequired, it should be good enough to declare anything

PEP 692 – Using TypedDict for more precise **kwargs typing | peps.p...

Python Enhancement Proposals (PEPs)

Python documentation
typing — Support for type hints

Source code: Lib/typing.py This module provides runtime support for type hints. For the original specification of the typing system, see PEP 484. For a simplified introduction to type hints, see PE...

#

😅 all is left updating to python 3.12 (my company is all in 3.10, and at most still in 3.11)

rare scarab
#

It should still work on earlier versions via __future__

oblique urchin
cunning plover
oblique urchin
#

True, but it's a very simple dependency and it's so widely used as a dependency that it's fairly likely you already have it installed as a dependency of something else

cunning plover
oblique urchin
undone saffron
#

FWIW, you're also better off reimplementing the minimum interface you need than using boto...

fierce ridge
oblique urchin
undone saffron
fierce ridge
fierce ridge
undone saffron
zenith plaza
#

For Pydantic v2, I would like make the two fields in the model mutually exclusive:

class ProcessorDefinition(BaseModel):
    script: Optional[str] = None
    pipeline: Optional[str] = None
#

literally one of the worst coverage areas

rare scarab
#

Make 2 models with a union

zenith plaza
#

ah neat idea, can you show me an example?

undone saffron
#

nothing about python type hints helps there, pydantic may or may not have a way to mark fields as mutually exclusive, but you would need to do the union for type checkers to handle that.

rare scarab
# 
class ProcessorScript(BaseModel):
  script: str

class ProcessorPipeline(BaseModel):
  pipeline: str

ProcessorDefinition = ProcessorScript | ProcessorPipeline
zenith plaza
#

lemme check

rare scarab
#

same way in typescript

zenith plaza
#

smart/neat

#

@rare scarab thanks a mil 🙂

#

should have figured it out on my own, but hey, we live we learn

rare scarab
#

and now you know to think of that

zenith plaza
#

indeed

#

how would be the ideal way to add a validator for a field, so that if it is missing, we take, for example, a parameter from an environment variable, or optionally, an argument passed to the model

#

@validator with pre? i

fierce ridge
zenith plaza
#

that would work

#

another potentially dumb question

#

suppose i have:

    processors:
      - pipeline: generic_add_indexed_timestamp
      - pipeline: validate_entity_subdocuments

I would like to have a special Pipeline definition that recognizes that all processors' items are pipelines.

#

this would simplify my handling of the validated data later, as I can directly access the chained pipeline objects

#

@rare scarab if you are still around

rare scarab
#

hi

#

You mean like an enum or choices?

zenith plaza
#

hm it's not an enum

rare scarab
#

so a definition to be used later in the model?

zenith plaza
#

so, some pipelines are chains of pipelines.

#

so all the items in processors will be Pipeline defs

#

i would like to automatically group those

rare scarab
#

you may want to play around with generics

zenith plaza
#

ex.

rare scarab
#

maybe add a property that filters the processors?

zenith plaza
#

` def validate_pipelines(self):
known_pipelines = []
for pipeline in self.validated_config.pipelines:
known_pipelines.append(pipeline.name)

        if isinstance(pipeline, PipelineProccesorBasedDefinition):
            for processor in pipeline.processors:
                if isinstance(processor, ProcessorPipelineDefinition):
                    if processor.pipeline not in known_pipelines:
                        self.warn(f"ATTENTION: {pipeline.name} contains a reference to unknown pipeline {processor.pipeline}")`
rare scarab
#

using match/case may be useful here

zenith plaza
#

could do that yeah, but that adds another field, i just wondered if an elegant runtime solution could be done directly with pydantic

rare scarab
# 
match pipeline:
  case PipelineProcessorBasedDefinition(processors):
    for processor in processors:
      match processor:
        case ProcessorPipelineDefinition(pipeline) if pipeline not in known_pipelines:
          ...
#

sadly match doesn't fair well with for loops or list items

fierce ridge
brisk hedge
#

thanks bot

zenith plaza
#

lol

rare scarab
#

How much traffic to pypi would drop if github actions used a pypi mirror?

undone saffron
patent hawk
#

why are matplotlib's types garbage?

tranquil turtle
#

feel free to contribute

patent hawk
#

Fair enough. In reality I'm just venting because I do not like how shaky python typing is.

#

I just want solid strong typing end to end.

#

but in the case of matplotlib, they just don't have typing at all for a lot of their functions, i.e. Axes3D.scatter

patent hawk
crisp steppe
trim tangle
cunning plover
#

i have a problem with typing overrides.
Lets i have a class

class Typelog
    def with_fields(self, *args: LogType) -> "Typelog":
        logger = deepcopy(self)
        logger._with_fields = args  # type: ignore[assignment]
        return logger

that can return its Self copy... Self.

I have it created as global var

logger = get_logger(__name__)

but when i try using it it to override its own value it starts to malfunction 🙈 and saying it is Unbound
in

def main(event:  ProvisionInput, context: Any) -> ProvisionOutput:
    logger = logger.with_fields(typelog.Any("input", event))
Screenshot_20240220_144408.png
#

it works correctly only if i assigned it to a different named value

Screenshot_20240220_144449.png
#

is there a way to allow with typing override of the same type to same type?

#

i override logger: Typelog = with another instance of Typelog. it should be valid override from same type to same type and supposed to work correctly

trim tangle
cunning plover
rough sluiceBOT
#

examples/test_examples.py line 10

logger = get_logger(__name__)```
cunning plover
#

i have this lib published 😊

trim tangle
#

oh wait

#

logger = logger.with_fields(...) will fail at runtime

#

not sure why pylance is not complaining

#

!e

foo = ...

def bar():
    foo = foo.something()

bar()
rough sluiceBOT
#

@trim tangle :x: Your 3.12 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 6, in <module>
003 |     bar()
004 |   File "/home/main.py", line 4, in bar
005 |     foo = foo.something()
006 |           ^^^
007 | UnboundLocalError: cannot access local variable 'foo' where it is not associated with a value
cunning plover
rough sluiceBOT
#

typelog/logcore.py line 92

def with_fields(self, *args: LogType) -> "Typelog":```
cunning plover
#

umm... you know.

trim tangle
cunning plover
#

scratches head in confusion

trim tangle
#

That's how local variables work in Python 🤷

#

Python sees that global was not present and you assign foo, so foo is considered to be a local variable in that function

cunning plover
#

i wished it would have did

(global)foo = smth
def bar():
  (local).foo = (global).foo.smth()

no?

#

but it sees it as this then

(global)foo = smth
def bar():
  (local).foo = (local).foo.smth()
trim tangle
#

Consider this:

def bar():
    for item in some_items:
        if item.is_nice():
            logger = make_logger(item)
            break
    logger.do_something()
``` if `some_items` happens to be empty and `logger` never gets assigned, should python raise an exception or use the global `logger`?
cunning plover
#

pure global to func loc transition problem

trim tangle
#

yes

cunning plover
trim tangle
#

Yeah it might be quirky. Python just decides to be consistent in that a variable is either always local, always nonlocal or always global in the same scope

#

Why not name a module-level logger _logger though? It's not "public"/for export, is it?

cunning plover
cunning plover
#

in any case... 🙈 there is no clear benefit to its usage unless u start to use Sphinx Autodoc that has different observed behaviour for stuff u declare public/private

trim tangle
#

non-underscored symbols will appear higher in autocomplete, and it's clear that it's not for export

#

and you can use logger as a local variable name 🙂

cunning plover
#

i could. i thought just using log as shorter nickname.

#

than shorter then better. ^_^

trim tangle
#

Well, log is a log, not a logger

cunning plover
#

logger serving as regular default everyone got used seeing as global var

#

log as short local used instance with extra fields

trim tangle
#

A log is what a logger produces

cunning plover
trim tangle
#

surprised they didn't call it l

#

naming is bloat and is actually not required

cunning plover
#

log is shorter than _logger and still gives meaning correctly enough 😛

trim tangle
#

If I see a variable named log I'll first think that it's a log, not a logger

cunning plover
#

okay, we can name it as logus then 😅
(what are some other suffixes for noun forming from log are present 🤔 )

trim tangle
#

use cyrillic letters if there's a clash with a global

#

lоgger = logger.with_fields(...)

cunning plover
#

damn.

cunning plover
#

my whole team of other devs are French/English speaking people
I am not sure how it will work with them

#

will they be even able to see correct letter 🙈

#

also... what if I (or they) wrote name like logger manually, then it will not work as intended

trim tangle
#

yeah о should look exactly like o in modern fonts

cunning plover
#

no no no. I am not ready for this 🙈

#

sounds like a crime to me.

#

i'll name my local variable as logus 🎶

trim tangle
#

You get 4 distinct variables

logger
lоgger
loggеr
lоggеr
cunning plover
hardy linden
#

Hey, good morning. I'm using Pylance/Pyright in VS Code; Pyright updated this morning (to 1.1.348), and I believe I've got a regression - I've got a typing error on some code that's previously come up as clean, and seems like it should be okay.

I've read the open and recently-closed Issues for Pyright, and I don't think I see anyone else raising the same problems I'm seeing, but before filing an Issue and getting my hand slapped for "as designed", I wanted to ask here first, to make sure that I'm not just being an idiot.

I've got:

from __future__ import annotations
import typing

StringOrNone = typing.TypeVar("StringOrNone", str, None)
char_replacements: dict[str, str]  # Things like "MS Word quotes" -> regular quotes

def replace_bad_chars(input_string: StringOrNone) -> StringOrNone:
    if input_string is None:
        return input_string

    for bad_char in set(input_string) & set(char_replacements.keys()):
        # The following line now shows a typing error, and did not do so before the update
        input_string = input_string.replace(bad_char, char_replacements[bad_char])
    
    return input_string

Previously, the line beginning with input_string = did not show a typing error. However, I now get the following two errors on this line:

  • Expression of type "str* | Unknown" cannot be assigned to declared type "StringOrNone@_replace_bad_chars"
  • "replace" is not a known member of "None".

This... doesn't seem to make sense to me. By the time that we're doing character replacements, we know that input_string is str, not None; if it was None, we'd have returned immediately. On top of this, we know that we're not turning input_string into something other than a str, because char_replacements is a dict[str, str], and str.replace returns str. I don't understand where Unknown is able to enter the picture here.

Am I wrong to think that this error is incorrect? Or is this something I should file an Issue about?

trim tangle
#

if someone happens to have a None, they can handle it on their own

#

This does sounds like a bug though, I would report it to pyright

#

Anything in pyright can be marked as "as designed" 🙂 so don't let it discourage you

hardy linden
trim tangle
#

ah

hardy linden
#

about to grab the extra information requested

trim tangle
#

I think this is not the right way to use the "enumerated" typevar though. I'd expect it to work like
replace_bad_chars(None) : None
replace_bad_chars(str) : str
replace_bad_chars(str | None) : str | None
but because you're using the comma-separated thing and not the bound kwarg, the third one is not allowed

#

What exact function shape does the parent expect?

oblique urchin
trim tangle
#

Yep that is true

hardy linden
#

The goal is to process an element of an iterable of str | None. I've used a TypeVar here in an attempt to ensure that the output type matches the input type.

trim tangle
oblique urchin
trim tangle
#

Or StringOrNone = typing.TypeVar("StringOrNone", str, None, str | None)

hardy linden
# trim tangle Maybe `replace_bad_chars(input_string: str | None) -> str | None` then?

Neither of those are what I want, though. If input_string is a str, I know the return type will be a str. If input_string is None, I know the return type will be None. For both of those examples, I no longer know what my return type will be; I lose information about the value.

This having been said, with a use-case this narrow, it occurs to me that I can just write it as an overload. I'll do it that way if this doesn't bear fruit or gets complicated.

trim tangle
#

It is a bit silly. But that's how constrained typevars work

atomic idol
#

How can I get rid of pyright errors (from lsp) when at my variable is None at start and initialize that variable with appropariate value before I call the other methods.
For example:

# FILE - view.py
class View:
  def __init__(self, control: Controller):
    self.ctrl = control
    self.ctrl.view = self
  
  def update(self, *args, **kwargs):
    pass

# FILE - controller.py
class Control:
  def __init__(self, model: Model):
    self.model = model
    self.view: Optional[View] = None

  def update_view(self, *args, **kwargs):
    self.view.update(*args, **kwargs) # "update" is not a memeber of "None"

# FILE - main.py
def main():
  model = Model()
  control = Controller(model)
  view = View(control)
  view.mainloop()

Is there any way to get rid of this error thingy?

trim tangle
# atomic idol How can I get rid of pyright errors (from lsp) when at my variable is `None` at ...

Right now it's totally possible ("type safe") to do controller = Controller(model) and call controller.update_view(foo=1, var=4). That's why a type checker will not let you do this.
You have 3 options:

  1. lazily initialize view
class Controller:
    def __init__(self, model: Model, make_view: Callable[[], View]) -> None:
        self._model = model
        self._cached_view: View | None = None
        self._make_view = make_view

    @property
    def _view(self) -> View:
        if self._cached_view is None:
            self._cached_view = self._make_view()
        return self._cached_view

You create the controller as controller = Controller(model, lambda: view) use self._view inside the controller
2. Keep the multi-step initialization, but put all the checks in one place:

class Controller:
    def __init__(self, model: Model) -> None:
        self.model = model
        self.view = None

    @property
    def _view(self) -> View:
        if self.view is None:
            raise RuntimeError("You should initialize the view with a controller before using the controller")
        return self.view
  1. Put # type: ignore with the specific error on every use
#

Note that options 1 and 2 are not just for the sake of typing -- I think they will actually make it easier to write correct code and/or understand the error if you use it incorrectly

eternal surge
# oblique urchin That's not something that the type system can express. However, presumably `Mess...

Apparently this isn't correct at least on Pyright

from abc import ABC


class A(ABC):
    pass

class B(A):
    pass

class C(A):
    pass

def f(x: A):
    g(x)
#    ^^^
#     Argument of type "A" cannot be assigned to parameter "x" of type "B | C" in function "g"
#       Type "A" cannot be assigned to type "B | C"
#         "A" is incompatible with "B"
#         "A" is incompatible with "C"

def g(x: B | C):
    pass```
Correct me if I'm wrong, but a value of type A can either be a value of type A, a value of type B, or a value of type C.
It cannot be a value of type A - on account that A is an abstract class. Therefore the value can only be either of type B or of type C. Meaning type "A" should be able to be assigned to type "B | C".
rare scarab
#

Because it could be D(A)

eternal surge
rare scarab
#

But there could be in another module

#

A isn't sealed, so you can always extend it

eternal surge
rare scarab
#

That's not a feature of python

chrome hinge
#

You could use a structure like

class B:
    pass

class C:
    pass

A = B|C

instead. That sometimes makes more sense than inheritance (if you want to be able to use these variants differently).

rare scarab
#

A protocol could also be an option

eternal surge
eternal surge
undone saffron
#

If you mean B | C and actually need something from B or C, just use that as the annotation.

#

no need to complicate this further

brisk hedge
chrome hinge
eternal surge
undone saffron
#

type alias syntax is deferred in evaluation with the annotation future import, but isn't available pre 3.12

crisp steppe
#

At a type level there's no different but if you were to use something like Pydantic or dacite it might mean it tries to make B before C or C before B when reading a dict

spare minnow
#

Hi what is the best (at least most common) practice of return type for a function that returns a django queryset ? ty (ping me)

frigid jolt
#

Hi, i have this code here:

orders_total: dict[str, list[tuple[str, int]]] = {...}
total = await self.calculate_total_income(orders_total.values())

and pyright is giving me this error:

Argument of type "dict_values[str, list[tuple[str, int]]]" cannot be assigned to parameter "stocks" of type "list[list[tuple[str, int]]]" in function "calculate_total_income"
  "dict_values[str, list[tuple[str, int]]]" is incompatible with "list[list[tuple[str, int]]]"

now i know that list != dict_values but why the dict_values needs the type of the keys??? same question for .keys() but switched, why dict_keys needs the type of the values?

image.png
frigid jolt
trim tangle
#

a) total = await self.calculate_total_income( list(orders_total.values()) )
b) If the function doesn't need to mutate the list, change the signature to ```py
from collections.abc import Iterable

async def calculate_total_income(self, stocks: Iterable[Iterable[tuple[str, int]]]) -> int:
``` (or Sequence instead of Iterable if you need indexing)

frigid jolt
#

now i know that list != dict_values but why the dict_values needs the type of the keys???

viscid spire
trim tangle
rough sluiceBOT
#

stdlib/_collections_abc.pyi lines 70 to 75

@final
class dict_keys(KeysView[_KT_co], Generic[_KT_co, _VT_co]):  # undocumented
    def __eq__(self, __value: object) -> bool: ...
    if sys.version_info >= (3, 10):
        @property
        def mapping(self) -> MappingProxyType[_KT_co, _VT_co]: ...```
frigid jolt
#

huh

#

what pep introduced it?

trim tangle
frigid jolt
#

alright ty

signal brook
#

chat i just discovered something p game changing and figure other ppl need to know too
https://peps.python.org/pep-0655/#motivation
tldr:

""" before python 3.11 """
class _MovieBase(TypedDict):
    title: str

class Movie(_MovieBase, total=False):
    year: int

""" after python 3.11 """
class Movie(TypedDict):
    title: str
    year: NotRequired[int]
oblique urchin
signal brook
#

damn lol i had no idea NotRequired existed at all

#

usually when deep diving other libraries i see people making their own implementation of it

rare scarab
#

Can't you also do ```py
class Movie(TypedDict, total=False):
title: Required[str]
year: int

civic grotto
#

Just saw this line in our codebase. What does it mean? T = TypeVar("T", bound=Mapping[str, object])

trim tangle
civic grotto
#

I've never seen TypeVar before

trim tangle
civic grotto
#

thanks!

trim tangle
#

bound means that the type variable can only refer to values of that type. For example:

class Widget:
    def is_hidden(self) -> bool: ...

W = TypeVar("W", bound=Widget)

def only_visible(widgets: Iterable[W]) -> list[W]:
    return [w for w in widgets if not w.is_hidden()]

buttons: list[Button] = ...  # Button is a subclass of Widget
label: list[Label] = ...  # Label is a subclass of Widget

visible_buttons: list[Button] = only_visible(buttons)
visible_labels: list[Label] = only_visible(labels)
only_visible([1, 2, 3])  # error
#

I somehow didn't write anything about bound in the tutorial, maybe I'll fix it later

civic grotto
#

so this: T = TypeVar("T", bound=Mapping[str, object]) means that T should be an object where keys are strings and values are also objects?

trim tangle
viscid spire
patent hawk
#

How do i notat this type (using typescript notation for the example):

ImgType
ColorType

DictSpecialKeys = {
    [img: number]: ImgType
    color: ColorType
}```
trim tangle
#

that's a very strange structure in Python tbh

#

Just have a dict[int, Image] and a Color as separate fields

soft matrix
#

you can technically do this i think if subclass dict[int | Literal["color"], ImgType | ColorType] and then adding a bunch of overloads

cunning plover
#

fascinating moment.
TypedDicts with same keys are type equal!

Screenshot_20240223_171649.png
#

😅 that means all is enough to make such assignment in a test file, in order to validate type equality of some dicts

oblique urchin
rare scarab
#

Has anyone thought of a tuple-like iterator? ```py
def iter(self) -> TupleLike[int, int, str]: ... # yields a int, int, then str

viscid spire
tranquil turtle
#

but i like the idea

rare scarab
#

Still wish we had dict unpacking...

#

spreading?

#

spread dict to variable

#

like {x, y, z} = d

tranquil turtle
#

i found flaky pyright false-positive: https://pyright-play.net/?code=GYJw9gtgBA%2BjwFcAuCQFM5QJYQA5hCSgEMA7UsJYpLMUgZwFgAoFgYwBtj76oAVdGgDafALoAuFlGlQ2ACywcAJuP6CRoqAB8oAOTpopMgG7EOCNKr7a9BlkA
try removing value: T | None line and typing it manually again

sometimes i get the Type variable "T" has no meaning in this context error that references previous line
i also can reproduce it in my local setup that uses pyright LSP

if i touch code somewhere to force re-checking, error usually disappears

image.png image.png image.png image.png
#

not sure what is causing it
i guess some incorrect cache not-invalidation or something like that

cunning plover
eternal surge
#

How can I define a Protocol for a class with a method that is decorated with @asynccontextmanager?

I tried this


class Processor(Protocol):
    @asynccontextmanager
    async def meow(self, x: int) -> AsyncIterator[str]:
        ...

But I got this:

Argument of type "(self: Self@Processoar, x: int) -> Coroutine[Any, Any, Unknown]" cannot be assigned to parameter "func" of type "(**_P@asynccontextmanager) -> AsyncIterator[_T_co@asynccontextmanager]" in function "asynccontextmanager"
  Type "(self: Self@Processoar, x: int) -> Coroutine[Any, Any, Unknown]" cannot be assigned to type "(**_P@asynccontextmanager) -> AsyncIterator[_T_co@asynccontextmanager]"
    Function return type "Coroutine[Any, Any, Unknown]" is incompatible with type "AsyncIterator[_T_co@asynccontextmanager]"
      "Coroutine[Any, Any, Unknown]" is incompatible with protocol "AsyncIterator[_T_co@asynccontextmanager]"
        "__anext__" is not present
        "__aiter__" is not present

What am I supposed to put instead of AsyncIterator? That's what I've seen on stackoverflow....

oblique urchin
eternal surge
oblique urchin
#

you can still use async def in an implementation

#

unfortunately the interactions between type annotations are a bit weird here, since the presence of yield inside the function affects the type

eternal surge
#

I see
So for the concrete implementation I'd use async and it would work then?

oblique urchin
#

what you have says that if you call it, you get an awaitable that when awaited produces an async iterator

#

instead what you want is that if you call it, you get an async iterator

eternal surge
rare scarab
#

Why not annotate it with AbstractAsyncContextManager?

eternal surge
oblique urchin
#

implementations of the protocol may use @asynccontextmanager, but it should be possible to implement the protocol some other way

eternal surge
oblique urchin
#

or you'd have to write async with (await proc.meow(1)):

eternal surge
#

ah right

#

That does make sense
I like it better

patent hawk
#

so not completely true

trim tangle
patent hawk
#

btw you're probably right, but in the end the ergonomgics were nice and i got full type saftey

#

one sec

trim tangle
#

well, you could make an overloaded __getitem__

patent hawk
# 

class AssocaitedPoint(Protocol):
    @overload
    def __getitem__(self, i: int) -> ImagePoint: ...
    @overload
    def __getitem__(self, i: Literal["color"]) -> Color: ...
    @overload
    def __getitem__(self, i: Literal["has_wp"]) -> bool: ...
    @overload
    def __setitem__(self, i: int, v: ImagePoint): ...
    @overload
    def __setitem__(self, i: Literal["color"], v: Color): ...
    @overload
    def __setitem__(self, i: Literal["has_wp"], v: bool): ...
    def __contains__(self, i: int | Literal["color"] | Literal["has_wp"]) -> bool: ...
class PointsDict(dict, AssocaitedPoint):
    def __getitem__(self, i):
        return dict.__getitem__(self, i)

    def __setitem__(self, i, v):
        dict.__setitem__(self, i, v)
patent hawk
trim tangle
#

That is... mildly cursed 😛

patent hawk
#

yeah, works excatly as expected

#

defintely doesn't really make much sense do to unnecessary hashing of the literal, etc

#

but it's what i did

soft matrix
#

no get or delitem though :(

trim tangle
#

also not very type safe, because typing.overload doesn't check that your implementation follows the overloads

soft matrix
#

or pop or setdefault

patent hawk
patent hawk
#

it think

trim tangle
#

It can't

patent hawk
#

oh yeah, your right

#

but it gives you all the type hinting

#

but yes, you could viloate the Protocol in the impl

#

it's terrible

#

overall, in all capacities

#

but i needed a retroactive type for how i coded it in the first place

#

i had the residuals of JS every object being a dict in my system

#

hence the awefulness you see above

#

won't do it again fs

trim tangle
#

I think this structure is just not easy to work with, even in JS, compared to two separate fields

patent hawk
#

but i learned a lot about typing in python in the mean time

patent hawk
#

it just neans you avoid needing to do object.dict[1], object.color

trim tangle
#

Enumerating all the (img, ImgType) pairs is pretty awkward

patent hawk
#

you just do object[1], object.color

patent hawk
#

idk, this is not the right use case for that pattern in TS

#

like at all

#

but there are valid and solid use cases

trim tangle
#

Yeah, Python's type system doesn't have good tools to compose and remix existing types, sadly

#

I wish zod from TypeScript was possible in Python

patent hawk
viscid spire
patent hawk
#

lmao

#

im leaving it

solemn sapphire
# 
from typing import Protocol, TypeAlias, NamedTuple

class HasFileno(Protocol):
    def fileno(self) -> int:
        ...
FileDescriptorLike: TypeAlias = int | HasFileno

class SelectorKey(NamedTuple):
    fileobj: FileDescriptorLike

class Client:
    def fileno(self) -> int:
        return 0

    def handle_request(self) -> None:
        ...

def get_key() -> SelectorKey:
    return SelectorKey(Client())

# Error, but shouldn't this be safe?
client: Client = get_key().fileobj
#

Client is a FileDescriptorLike, so I should be able to assign get_key().fileobj to a variable of type Client?

grave fjord
solemn sapphire
#

but Client is an object with fileno()

grave fjord
#

Right but an object with fileno() might not be a Client

solemn sapphire
#

oh, its like an Animal and Dog thing isn't it?

grave fjord
#

And you have the | int to deal with

solemn sapphire
#

do I have to put a assert isinstance(key, Client) then

#

but I literally put a client in the selector!

grave fjord
#

But you could have put an int in there, the type system doesn't know

solemn sapphire
#

I see, would the Selector benefit from being typed as a generic thing then in typeshed?

grave fjord
#

It's not Generic at runtime so it's problematic to be Generic in the stub

#

Also I don't think it helps you as Selector would need to be generic

solemn sapphire
#

ye, I'll just put a # type: ignore then :(

grave fjord
#

Feels like the opaque data should be generic

solemn sapphire
#

but there I can do the_data: SomeType = key.data