type-hinting

[brittle socket]

Yup, better

[trim tangle]

Yeah it's annoying, I don't know any good solution

[gusty kelp]

I'm curious, would it make sense to simply use a class in that case?

[paper salmon]

seems to work for me, using pyright and mypy 1.1.1

oh whoops forgot to type it as a function argument

[paper salmon]

they might not want to lose the built-in enum features thoguh

[paper salmon]

then yeah you might be fine using a regular class as a namespace, and typehinting env with Searcher instead (or an abc/protocol, and/or a dataclass containing the searcher and any other attributes if that's desired)

knowing the type of .value would presumably require enum to be generic, and currently it isnt

i guess subclassing (Searcher, Enum) would be an option?

this seems to work, but doesnt actually assert that the enum's members all match said type

[trim tangle]

What does search do?

(maybe you can show some code?)

[trim tangle]

What I would probably do is not attach any behavior to actual enum members, and instead have a function like ```py
def search(env: Environment, entry: str) -> Value | None:
...

or ```py
search_in_env: Mapping[Environment, Searcher]

[trim tangle]

and this is also how you'd do it in other languages with enums

[sour tundra]

I have a log files, it might contain an error lines or not, i want to make a code that can understand each error line and print just a unique from each no need to duplicate

Example

Input file :
Leakage value 1.2 for circuit 1 is greater than the standard

1)Leakage value 0.9 for circuit 2 is greater than standard

2)Capacitance is huge in circuit 3

3)Capacitance is huge in circuit 4

4)Capacitance is huge in circuit 5

5)Capacitance is huge in circuit 6

6)High delay in circuit 7

Output: must be the unique ignoring instance information like circuit number or certain value

1. Leakage value 1.2 for circuit 1 is greater than the standard

2. Capacitance is huge in circuit 3

3. High delay in circuit 7

The log file may contain over than 10000 errors but not ,however its might be just 10 unique errors as shown in the output ,,,

Anyone can suggest a library, or a place to start from?is it possible to make code clever enough to determine these things ?

[hearty shell]

Question of type "#python-help" cannot be assigned to parameter "chat" of type "#type-hinting" in server "Python"
  "#python-help" is incompatible with "#type-hinting"

[stuck oyster]

hello,everyone,I am confused about mypy and typing,what major difference in devlopments'

[eager vessel]

typing is a built-in python package, providing you with objects to use in type annotations

For example Self

mypy is a type checker, they're to be used together

[regal summit]

def test() -> None:
    return 123``` why is my type checker not complaining im confused

im on vscode with it set to strict

[soft matrix]

you sure?

do you have any configs for this workspace?

[reef yacht]

A guess might be that you're not running a type checker :P

[paper salmon]

^ if there's a pyproject.toml you might have a [tool.pyright] section which will take precedence over your IDE settings (also pyrightconfig.json https://microsoft.github.io/pyright/#/configuration)

[trim tangle]

I kinda wish we had a language-agnostic place to store all the tool configs...

Well, we do have environment variables!

but like

ew

[rare scarab]

what languages are you mixing?

[trim tangle]

For example:

1. Frontend (e.g. JavaScript, TypeScript, Elm)
2. Documentation (e.g. markdown)
3. Backend (e.g. F#, Python)
4. Build and auxiliary scripts (e.g. Bash, Python)

[rare scarab]

but what tools?

[trim tangle]

linters, formatters, build tools, package managers

[rare scarab]

so like prettier, black, shellcheck, etc?

something like .editorconfig, but for everything else?

[trim tangle]

yes

[rare scarab]

Well good luck getting everyone to agree on a format. It took a while for pyright to consider supporting pyproject.toml

[reef yacht]

I started a project like 5 years ago do make a unified command line interface on top of All The Tools. But I kinda god sick of trying to deal with just python honestly.

[rare scarab]

A lot of tools like that are usually written in ruby

[reef yacht]

My vision was to be able to clone a project from github, and go into it and do p run, p test and p repl and that working for all programming languages

[reef yacht]

kill me now

[buoyant swift]

i seem to recall a highly relevant xkcd

[rare scarab]

You could probably also do it in bash

[reef yacht]

yea, it's a thing heh. BUT I don't think a truly unified system is a thing people have attempted no?

[rare scarab]

I've seen one before, but I forget the name.

[reef yacht]

https://github.com/replit/upm maybe?

It's.. close

[buoyant swift]

i'm sure it's been tried, but probably not very successful

[hearty shell]

If only everybody used it

[oblique mantle]

how can this be fixed

[pastel egret]

The precise value of default values isn't really used by type checkers, so these two overloads are basically the same as each other. What you need to do is use Literal[True/False] as the type itself, and make 3 overloads (one each for a literal true/false, then another for the case where a bool is passed and the specific value is not known).

@overload
def hello(age: Literal[False] = False) -> Optional[str]: ...
@overload
def hello(age: Literal[True]) -> str: ...
@overload
def hello(age: bool) -> Optional[str]: ...

In this case you could do just two overloads - putting Literal[True] first, then age: bool = False) second. But usually you'll need to cover all three cases separately.

All that = ... does here is indicate that the first overload matches if you call the function with no parameters. Since that case is handled there, there's no need to specify it for the other two.

[oblique mantle]

Thank you

[trim tangle]

Why not make two functions?

[oblique mantle]

i was testing in something else, this is just made to show

[trim tangle]

Still 🙂

[oblique mantle]

ah two functions

[signal oak]

Hey, just want to check whether I have annotated this lambda correctly?(MyPy doesn't argue, though I am concerned about my Class implementation that I took from builtins module)

Class: TypeAlias = Type[Any] | types.UnionType | Tuple[Type[Any], ...]
predicate: Callable[[Class], bool] = lambda attr: not(inspect.isroutine(attr))

get_class_attrs: Callable[[Class, Class], 
                          List[Tuple[str, Class]]] = lambda _class, _type: [attr for attr in inspect.getmembers(_class, predicate) if isinstance(getattr(_class, attr[0]), _type)]

[dull lance]

is there a reason why you are using lambdas?

imo it is far less readable than a regular type hinted function

[trim tangle]

Yeah don't use a lambda here

[signal oak]

Reasonable, thanks. What about typing, it's all fine?

[dull lance]

I guess the typing is correct

[signal oak]

That's better?

Class: TypeAlias = Type[Any] | types.UnionType | Tuple[Type[Any], ...]

def get_class_attrs(_class: Class, _type: Class) -> List[Tuple[str, Class]]:
    predicate: Callable[[Class], bool] = lambda attr: not(inspect.isroutine(attr))

    members = inspect.getmembers(_class, predicate)

    return [attr for attr in members if isinstance(getattr(_class, attr[0]), _type)]

[dull lance]

I think you could just pass the predicate into inspect.getmembers directly to avoid having to type hint it explicitly

if you have to separate it out, at least give it a meaningful name

[signal oak]

Oh, yeah, I have just moved it into parameters. Thanks!

[dull lance]

and in your list comprehension, you can use unpacking to directly get the elements of the tuple instead of having to index it

i.e. [(name, value) for name, value in members if isinstance(getattr(_class, name), _type)]

[signal oak]

What are the advantages of doing so?

[dull lance]

more readable (attr[0] vs name)

[signal oak]

Indeed

[brittle socket]

from typing import TypeAlias

class ABCD:
  A: TypeAlias = ABCD

I want to typealias ABCD as A and use this alias within ABCD. This doesn't work because ABCD is not yet defined. Is there a way for it to work?

[soft matrix]

whats the point lol?

[brittle socket]

Because ABCD stands for a long name and I'm typehinting with it all over the place, as well as calling static methods

It's too much visual noise

[soft matrix]

im personally not a fan of shortening names for classes but whats stopping you just importing it as A?

also you are already typing out the name anyway right?

[brittle socket]

How can I import it as A? I'm within the class's definition

The typealias is for internal use within the class

[soft matrix]

maybe just consider renaming your class then?

whats its current name?

[brittle socket]

Its current name is explicit, as it should be

I only wanted to alleviate the noise of its use internally, but if it's not feasible nvm

[soft matrix]

you definitely can but i dont see the point, use a classproperty and just return cls

and just make the sig (type[Self]) -> type[Self]

[brittle socket]

You'd see the point if you saw how many static method calls there are

[soft matrix]

i dont really get the point of static methods but maybe thats just me

[brittle socket]

Moral encapsulation 🤷‍♂️

[soft matrix]

theres already encapsulation from them (if they were all free functions) all being in the same module no?

[brittle socket]

Maybe there are benefits to importing a self-contained class as opposed to its whole module

[tranquil turtle]

= 'ABCD' should work

[soft matrix]

not at runtime though

[brittle socket]

class ABCD:
  A: TypeAlias = "ABCD"

  def __init__(self):
    A.static_method()

doesn't work :/

It's ok, I can live with it

[paper salmon]

if your usage is inside instance/class methods, there's no convention saying you can't use self/cls to reference them

or alternatively you can define the alias outside of the class, assuming no function/method that references the alias will get called beforehand: ```py
class AwfullyBoundlessCognomenDesignation:
def init(self):
A.static_method()

A = AwfullyBoundlessCognomenDesignation```

though worth noting that they are not equivalent solutions when dealing with subclasses (self/cls can call an overridden static method while the explicitly named class will always refer to that one static method)

[brittle socket]

I'll try out your solution, thank you 🙏 (and yes, the hierarchy ambiguity is why I generally prefer calling class/static methods by the class name rather than self)

[low wraith]

hello guys, can someone explain to me why one would use the @rustic gullmethod decorator for a function in a protocol ?

[paper salmon]

in a typing.Protocol specifically? or do you mean an Abstract Base Class, because afaik @abstractmethod doesnt have any meaning outside of an ABC

[low wraith]

in typing.protocol, yes

[paper salmon]

oh huh, apparently it is demonstrated in pep 544
https://peps.python.org/pep-0544/#explicitly-declaring-implementation

though the behaviour appears to be different between pyright and mypy ```py
class PColor(Protocol):
@abstractmethod
def draw(self) -> str:
...

def complex_method(self) -> int:
    ...  # some complex code here

class BadColor(PColor): # pyright: missing complex_method member
def draw(self) -> str:
return super().draw() # mypy: unsafe to use super

class NiceColor(PColor):
def draw(self) -> str:
return "deep blue"

def complex_method(self) -> int:     # no complaints from pyright
    return super().complex_method()  # mypy: still unsafe to use super?``` well regardless it seems that the pep's intention is to disallow calling certain methods from the protocol with super()

[warm knoll]

I'm starting to use type hinting for the first time and i am curious on how is the best way to do this:

MENU = {
    "espresso": {
        "ingredients": {
            "water": 50,
            "coffee": 18,
        },
        "cost": 1.5,
    },
    "latte": {
        "ingredients": {
            "water": 200,
            "milk": 150,
            "coffee": 24,
        },
        "cost": 2.5,
    },
    "cappuccino": {
        "ingredients": {
            "water": 250,
            "milk": 100,
            "coffee": 24,
        },
        "cost": 3.0,
    },
}

[paper salmon]

i would probably use dataclasses or pydantic to represent each menu item instead, but just on an isolated scenario for typing that specific structure, you can use TypedDict to statically assert the existence of certain keys and values:
https://docs.python.org/3/library/typing.html#typing.TypedDict ```py
from typing import TypedDict

class MenuItem(TypedDict):
ingredients: dict[str, int]
cost: float

MENU: dict[str, MenuItem] = ...```

[warm knoll]

ooooh, that make senses, thank you so much

[paper salmon]

and taking it further if you wanted to assert specific ingredients for coffee items: ```py
T = TypeVar("T")

if you only expect your menu items to have CoffeeIngredients,

you're free to typehint ingredients: CoffeeIngredients directly

class MenuItem(TypedDict, Generic[T]):
ingredients: T
cost: float

class CoffeeIngredients(TypedDict, total=False):
water: int
milk: int
coffee: int

COFFEE_MENU: dict[str, MenuItem[CoffeeIngredients]] = ...```

[warm knoll]

You the best, thanks

[paper salmon]

derivative question, it'd be nice to make T bound to dict[str, int] for all ingredients, but can CoffeeIngredients be declared as a subtype of dict[str, int]?

[warm knoll]

I think so, yea

[paper salmon]

i mean like how would you type it, because by itself pyright and mypy dont seem to understand that CoffeeIngredients is compatible with dict[str, int]

[warm knoll]

Honestly I don't know, just first time doing typing 😅

[signal oak]

Not sure what you are trying to do. Maybe this would help?

from typing_extensions import Self

class ABCD:
    A: Self

[tranquil turtle]

You can use __class__ too. But it also doesn't work with subclasses

[paper salmon]

ah i forgot, dictionaries are mutable collections which makes them invariant
https://blog.daftcode.pl/covariance-contravariance-and-invariance-the-ultimate-python-guide-8fabc0c24278#2bd6

[reef yacht]

But.... it's not? A dict of str to int can have any str key. But your typed dict cannot.

[paper salmon]

right thats why i mentioned their mutability

[brittle socket]

I'm on phone rn, but would Self.static_method() work?

[soft matrix]

No

We thought about having it do something at one point but that was after it was submitted and it'd be way too hard to implement

[reef yacht]

That's not the thing here I'm pretty sure. A typed dict is also mutable.

[soft matrix]

class.static_method would though

[signal oak]

Don't think so

[paper salmon]

i meant to theorize that its a mutable collection and those need to tolerate arbitrary elements, which TypedDict cannot (so the same thing you said)

pep 589 follows that same reasoning, but they also disallow Mapping[str, int] which i wasnt expecting (that was the second thing i tried before reading the pep)
https://peps.python.org/pep-0589/#type-consistency

A TypedDict isn’t consistent with any Dict[...] type, since dictionary types allow destructive operations, including clear(). They also allow arbitrary keys to be set, which would compromise type safety.

A TypedDict with all int values is not consistent with Mapping[str, int], since there may be additional non-int values not visible through the type, due to structural subtyping.

[reef yacht]

but they also disallow Mapping[str, int]
You misread. It actually says:
First, any TypedDict type is consistent with Mapping[str, object].

[paper salmon]

i know that but what i referenced is near the bottom of that section

[reef yacht]

ah, my bad

yea, sorry, I missed the specific int there

[paper salmon]

i guess that restriction is necessary to maintain the substitution principle

[reef yacht]

yea

[tranquil turtle]

from __future__ import annotations
import typing as t

class Func(t.Protocol):
    def __call__(_, self: X, msg: str) -> None:
        ...

TFunc = t.TypeVar('TFunc', bound=Func)

def cmd(f: TFunc) -> TFunc:
    return f

class X:
    pass

class Y(X):
    @cmd # Value of type variable "TFunc" of "cmd" cannot be "Callable[[Y, str], None]"
    def stop(self, msg: str) -> None:
        pass

I dont understand. What is the problem there?
How to fix it?

[brisk hedge]

variance

Func requires the call method to accept X and its subclasses, but stop only accepts Y and its own subclasses which doesn't include X

In general you'd want to replace X with a contravariant typevar bound to X I believe

But you obviously can't parameterize TFunc if you do that

so you'd need to restructure

[slender timber]

I am using this decorator (supports_slice):

class ModelBase(abc.ABC): ...
MT_co = TypeVar("MT_co", bound=ModelBase, covariant=True)

@runtime_checkable
class ModelCollection(Iterable[MT_co], Protocol[MT_co]):
    @overload
    def __getitem__(self, i: int | str) -> MT_co: ...
    @overload
    def __getitem__(self, i: slice) -> Sequence[MT_co]: ...

def supports_slice(func: Callable[[ModelCollection[MT_co], str | int | slice], MT_co]):
    """Wraps a :meth:`ModelCollection.__getitem__` to return a sequence if required."""

    @overload
    def wrapper(self: ModelCollection[MT_co], i: int | str) -> MT_co: ...
    @overload
    def wrapper(self: ModelCollection[MT_co], i: slice) -> Sequence[MT_co]: ...

    @functools.wraps(func)
    def wrapper(self: Any, i: Any) -> MT_co | Sequence[MT_co]:
        if isinstance(i, slice):
            return [model for idx, model in enumerate(self) if idx in range(i.start, i.stop)]
        return func(self, i)

    return wrapper

as a helper for implementing __getitem__ methods:

class Arrangements(EventModel, ModelCollection[Arrangement]):
    @supports_slice
    def __getitem__(self, i: int | str | slice) -> Arrangement: ...

I get an error:

"__getitem__" overrides method of same name in class "ModelCollection" with incompatible type "_Wrapped[(ModelCollection[Arrangement], str | int | slice), Arrangement, (self: Any, i: Any), Arrangement | Sequence[Arrangement]]"

and

Argument of type "(self: Self@Arrangements, i: int | str | slice) -> Arrangement" cannot be assigned to parameter "func" of type "(ModelCollection[MT_co@supports_slice], str | int | slice) -> MT_co@supports_slice" in function "supports_slice"
  Type "(self: Self@Arrangements, i: int | str | slice) -> Arrangement" cannot be assigned to type "(ModelCollection[MT_co@supports_slice], str | int | slice) -> MT_co@supports_slice"
    Parameter 1: type "ModelCollection[MT_co@supports_slice]" cannot be assigned to type "Self@Arrangements"
      "ModelCollection[Arrangement]" is incompatible with "Arrangements"

So the most noticeable thing is when I do Arrangements_instance[0] it says that I miss the self argument

[tranquil turtle]

You should specify return type of supports_slice at least

Why are you explicitly subclassing protocol?
Also, your __getitem__ signature is incorrect. It shouldn't accept slice.

[slender timber]

so where should i begin then?

why do you say __getitem__ shouldn't accept slice?

[soft matrix]

i agree that seems fine

[signal oak]

flake8 gives the following error: WPS421 Found wrong function call: __import__
What's wrong with using __import__() here?

modules: Callable[[str], Iterator[pkgutil.ModuleInfo]] = lambda package: pkgutil.walk_packages(
   [__import__(package, fromlist=['dummy']).__path__[0]],
   )

[tranquil turtle]

Because the whole point of your decorator is to not care about slices, so your function should care only about int and str indices

[slender timber]

ok but how will that resolve the other errors i get?

[sinful briar]

Why do you need to import classes that you only use for type hints? Arent type hints just for human reader and your IDE?

I keep getting circular imports bc of type hints and needing to do the thing where you put the class name in quotes

[soft matrix]

!d typing.TYPE_CHECKING

[rough sluice]

typing.TYPE\_CHECKING

typing.TYPE_CHECKING```
A special constant that is assumed to be `True` by 3rd party static type checkers. It is `False` at runtime. Usage:

```py
if TYPE_CHECKING:
    import expensive_mod

def fun(arg: 'expensive_mod.SomeType') -> None:
    local_var: expensive_mod.AnotherType = other_fun()
```  The first type annotation must be enclosed in quotes, making it a “forward reference”, to hide the `expensive_mod` reference from the interpreter runtime. Type annotations for local variables are not evaluated, so the second annotation does not need to be enclosed in quotes.

[soft matrix]

and

future.annotations might help

[inland mist]

Hi everyone,
my project has the following structure:

./mypy_
├── app
│   ├── __init__.py
│   └── main.py
└── packages
    └── my_package
        ├── my_package
        │   ├── __init__.py
        │   └── lib.py
        └── pyproject.toml

in main.py I import from my_package.lib import hello
when I run mypy ./app/main.py, I get
Cannot find implementation or library stub for module named "packages.my_package.lib"
which is understandable - the way we use the app is build and install the packages in the venv, so app can use them.
However, I would like to be able to run mypy without venv and point it where local packages reside.
I've read and tried https://mypy.readthedocs.io/en/latest/running\_mypy.html#mapping-paths-to-modules and https://mypy.readthedocs.io/en/latest/running\_mypy.html#finding-imports, without success

[inland mist]

Alright, when I run mypy . it works as expected, although it falls flat when I specify concrete file as above. Any idea how to achieve working mypy ./app? I need to incrementally introduce mypy per project, and I need to selectively run it per folder

One of workarounds: mypy_path = $MYPY_CONFIG_FILE_DIR/packages/my_package in mypy.ini. Not happy with that, but it will work

[celest iron]

when using coroutines via generators, how can u type the yield?

def my_gen():
  x: ??? = yield

(i'm using pyright, but i don't know if mypy would have something different).
if I put in something Ik like int or str, pyright tells me x: Unknown | int

[oblique urchin]

the thing returned from the yield is the second type argument to Generator

so if you have def my_gen() -> Generator[A, B, C]:, yield returns B

[celest iron]

ah, i thought i did that, let me chk again

i think i got it. i'm using a decorator to turn a generator into a normal function (similar to contextlib.contextmanager) and I think I need to forward some types back to my wrapper function 🤔

[trim tangle]

FYI pyright-playground will be down for a few days

[stray summit]

is there a correct way to type a function that wraps around cast

taking a input of Type[T] and returning T appears not to work

https://mypy-play.net/?mypy=latest&python=3.12&gist=2a9662095098d413fffcb09a2ee39dc0

[trim tangle]

I'd use a # type: ignore

also maybe use ```py
ep = next(iter(entry_points(name=name, group=group)), None)

[tranquil turtle]

Is it possible to declare TypedDict with some keys and somehow express that this dict can contain arbitrary set of other keys?

[trim tangle]

no

[tranquil turtle]

:-(

[trim tangle]

https://github.com/python/mypy/issues/4617 💀

[buoyant swift]

it seems there has been some progress, at least

now you can just ignore the error it causes 😬

[trim tangle]

Anyone knows how to tell Pylance to auto-complete imports from Starlette?

Starlette has a pretty nested structure, and pylance straight up refuses to complete stuff like from starlette.staticfiles import StaticFiles

[rare scarab]

You need to set the search depth in the vscode config. Let me fetch mine real quick.

[trim tangle]

    "python.analysis.packageIndexDepths": [
        {
            "name": "starlette",
            "depth": 16,
            "includeAllSymbols": true
        }
    ]
``` You mean this? Yeah this does not help

[rare scarab]

Looks like the configuration I use is on my other pc at home

Do you have python.analysis.indexing set to true?

[trim tangle]

yep

[paper salmon]

huh, a depth of 2 seems to work fine for me

[signal oak]

_T = TypeVar('_T')
def get_classes(module_name: str, class_type: Generic[_T]) -> list[Generic[_T]]:

Why does MyPy argue: Variable "typing.Generic" is not valid as a type?

[oblique urchin]

Generic is a special form that can only be used in a base class

I think you want type[_T]

[signal oak]

Is it possible to make a TypeAlias for it to not write type[_T] every time? Or does it sound ridiculous?

[oblique urchin]

probably but you'd still need to write the [_T] part

so at best you save three characters

[signal oak]

Meaning there is no way to have something like that:

Class: TypeAlias = type[_T]
def foo(a: Class): ...

?

[soft matrix]

no because the default for Class's parameter will be Any

[oblique urchin]

writing just Class would be equivalent to Class[Any]

[signal oak]

And so I am guessing there is no way to make a "shorthand" and not specify the parameters each time?

[trim tangle]

@soft matrix @acoustic thicket https://pyright-playground.decorator-factory.su/

It should be a bit faster now

Also I would really appreciate help on decorator-factory/pyright-playground#4

[mighty linden]

GitHub

[pyright-playground] #4 rollup hangs for no apparent reason

[buoyant swift]

new tld?

[trim tangle]

no, I just moved everything into serverless ™️

to save myself some headaches and money

why 😩?

[buoyant swift]

idk it just felt right

[frank glade]

hey guys, quick question: what's the best way to automatically generate annotation files from .py files?

with .py files having annotations themselves

[soft matrix]

.py -> .pyi?

[frank glade]

yes

[soft matrix]

why lol?

that just seems like a step backwards

[frank glade]

i'm tinkering with justuse. i really like having my shared modules on github and just pull them on the fly. problem is, vsc doesn't know what to do with things coming out of thin air, so i'd generate a pyi into the local project when it's first loaded

then i have the pyi for reference on subsequent runs and still the benefit of loading stuff from github directly

so, how to .py -> .pyi?

[tranquil turtle]

Stubgen can do that, i think

[frank glade]

stubgen?

[tranquil turtle]

It is a tool that generates stubs from python files for you

[frank glade]

https://pypi.org/project/pystubber/ ?

[tranquil turtle]

No

[frank glade]

oh, mypy stubgen?

[tranquil turtle]

https://mypy.readthedocs.io/en/stable/stubgen.html

Yes

[frank glade]

excellent, thanks.

[rough kettle]

Pyright is supposed to support recursive types. I'm getting warnings on this test function:

# A tree where nodes can be strings or ints
KeyStrOrInt = dict[str | int, "KeyStrOrInt" | None]
# A tree where the first nodes are strings
FirstLevelKeyStr = dict[str, KeyStrOrInt]

def test(targs: FirstLevelKeyStr, **kwargs: FirstLevelKeyStr) -> KeyStrOrInt:
    # kwargs is a tree where the first and second nodes are strings
    # We should be able to merge these.
    return targs | kwargs

Error is on the return statement:

Expression of type "dict[str, KeyStrOrInt | FirstLevelKeyStr]" cannot be assigned to return type "KeyStrOrInt"
  "dict[str, KeyStrOrInt | FirstLevelKeyStr]" is incompatible with "KeyStrOrInt"
    TypeVar "_KT@dict" is invariant
      Type "str" cannot be assigned to type "str | int"
        "str" is incompatible with "int"

I'm pretty sure this is probably false: Type "str" cannot be assigned to type "str | int"

[trim tangle]

This probably has to do with variance, but in this case I don't think there can be any issues.

If you want you can make an issue

[rough kettle]

how would you title the issue?

[trim tangle]

no idea

something like "False positive when returning a union of dicts"

[rough kettle]

https://github.com/microsoft/pyright/issues/4881 in case anyone is interested

Any idea of how I can work around this besides a cast?

[trim tangle]

yes, with a # type: ignore 🤡

[rough kettle]

lol

no runtime cost tho

[trim tangle]

yes, removing a call to an identity function is the key to improving the performance of a system /s

[rough kettle]

• ceo of programming

[rough kettle]

Pyright's behavior is correct here. The resulting type from the | operation is dict[str, dict[str | int, KeyStrOrInt] | dict[str, dict[str | int, KeyStrOrInt]] which is not compatible with dict[str | int, KeyStrOrInt]. The type parameters for dict are invariant, so they must match exactly.

shouldn't this also apply here:

def test2(a: dict[int | str, int], b: dict[int, int]) -> dict[int | str, int]:
    return a | b

[rough kettle]

pyright doesn't complain here

[rough sluice]

We have an exciting announcement to make!

First typing talk ⌨️ on Python DIscord!

⭐ **Embracing Python: How to remove type annotations from your code ** ⭐

We will cover:
👉 Tools to remove type annotations completely from your code
👉 Switching to better Python versions (such as Python 2.7) to make it harder to annotate your code
👉 Tweaking your tools and CI/CD pipelines to reduce or completely remove type checking
👉 Advanced techniques like the Mapping[str, Any] pattern to make type checkers close to useless

The talk will start on <t:1680299880> and will be hosted by @trim tangle! The stream 📹 will start here: 👉 YouTube stream 👈

[brazen jolt]

a | b creates a new dict that is a combination of the two, so there isn't an issue.

The reason dict is invariant is to prevent functions that take in say a dict like dict[str, Animal] being passed a dict[str, Dog], and mutating it, by setting some value to a Cat.

But when a new dict is created, this isn't an issue, there's no mutation, so invariance wasn't broken, you just created a new dict that contains both, that's why pyright doesn't complain in there

[rough kettle]

@brazen jolt shouldn't that also be the case for the recursive type case?

Is it just that pyright applies these "it's not mutating so it's ok" exceptions, but in the recursive case it was too complicated for it to figure it out?

[brazen jolt]

well in this case it's actually because your KeyStrOrInt type has keys specified as a union of str or int

[flint garnet]

hello 🙂 can someone help me with my assigement ? please 😄 it is "easy" ,but i am in programming school for like 2 weeks only.. please 🙂

[brazen jolt]

but the return type from targs | kwargs is just dict[str, "KeyStrOrInt"]

so the return type doesn't match

[flint garnet]

i tried chat gpt help and i tried my own logic and quest doesnt work 😦

[brazen jolt]

this probably isn't the right channel, check out #❓｜how-to-get-help

[flint garnet]

i tried that , only 1 respond 😦

[brazen jolt]

well, maybe try to make your question more appealing, provide more context, shrink it down to only the small part that you don't understand, either way, asking in channels that are for something else is not a good way to get an answer

@rough kettle perhaps consider using collections.abc.Mapping instead, those are immutable, and hence covariant

the issue with using **kwargs is also that pyright will infer it to be a dict with key type of str, and value type of the given type, so you'd be getting dict[str, dict[str, KeyStrOrInt]] which you're then merging with targs, that has a type of dict[str, KeyStrOrInt].

So, the merge will simply give you a union of keys, and union of values, hence you get dict[str, KeyStrOrInt | dict[str, KeyStrOrInt]], which is then not the same as dict[str| int, KeyStrOrInt], hence pyright screams at you.

as an example with dicts, this would work: ```py
KeyStrOrInt = dict[str | int, "KeyStrOrInt"]
FirstLevelKeyStr = dict[str, KeyStrOrInt]
FirstLevelKeyInt = dict[int, KeyStrOrInt]

def test(a: FirstLevelKeyStr, b: FirstLevelKeyInt) -> KeyStrOrInt:
return a | b

since in both cases, the value type is the same, and the key results in a union of the two, matching the KeyStrOrInt.

[carmine phoenix]

pretty cool stuff

[vital shore]

when type hinting, are you supposed to add a -> None to functions that don't return anything?

[trim tangle]

Yes

[vital shore]

grazie

should it be green in vscode?

[trim tangle]

no idea

[vital shore]

ok

[trim tangle]

(I prefer fewer colors so YMMV)

[vital shore]

im doing unit testing with pytest, and I am using pytest.raises(ValueError) and contextlib.nocontext going into a function through the parameter expectation. What type should expectation be?

I looked at https://docs.python.org/3/library/stdtypes.html#context-manager-types but it says there isnt really a specific type for this

[tranquil turtle]

You always can declare SupportsWith protocol

[vital shore]

im sorry i dont know what that means

[soft matrix]

!d contextlib.AbstractContextManager

[rough sluice]

contextlib.AbstractContextManager

class contextlib.AbstractContextManager```
An [abstract base class](https://docs.python.org/3/glossary.html#term-abstract-base-class) for classes that implement [`object.__enter__()`](https://docs.python.org/3/reference/datamodel.html#object.__enter__ "object.__enter__") and [`object.__exit__()`](https://docs.python.org/3/reference/datamodel.html#object.__exit__ "object.__exit__"). A default implementation for [`object.__enter__()`](https://docs.python.org/3/reference/datamodel.html#object.__enter__ "object.__enter__") is provided which returns `self` while [`object.__exit__()`](https://docs.python.org/3/reference/datamodel.html#object.__exit__ "object.__exit__") is an abstract method which by default returns `None`. See also the definition of [Context Manager Types](https://docs.python.org/3/library/stdtypes.html#typecontextmanager).

New in version 3.6.

[soft matrix]

this is i think what denball is referring to

[vital shore]

hmm i think i see what's going on here

i haven't done anything with contextmanagers before

so would I have to define my own abstract base class in order to define my type hint?

[soft matrix]

no

just annotate the parameter as this and it should just work

[vital shore]

ohhhh

yeah that works

AbstractContextManager ftw

[frigid jolt]

no

however colors don't have much to do with Type hinting, they depends on the theme that you're using

they're just colors

[vital shore]

what does the colour represent in this case?

[frigid jolt]

? it doesn't represent anything, it's just a color

[sinful wadi]

following docs for fastapi, in particular type hinting with annotations

this is what i have:
async def get_info(token: Annotated[str | None, Header()] = None):

same format as in the docs yet i get this:

AssertionError: "Header" default value cannot be set in "Annotated" for 'token'. Set the default value with "=" instead.

thoughts?

[trim tangle]

@sinful wadi What version of FastAPI and Python are you using?

[sinful wadi]

python 3.10.8

fastapi 0.95

everything newest i believe

[trim tangle]

can you also share the code?

[sinful wadi]

from fastapi import APIRouter, Cookie, Request, Response, Header

@router.get("/")
async def get_info(token: Annotated[str | None, Header()] = None):
  userInfo = await authorize(token)
  if not userInfo:
    return JSONResponse({ "message": "Not logged in"}, 403)

  return userInfo

router is used instead of app but it's just for separation of project into multiple files, directly from the docs

[trim tangle]

from typing import Annotated

from fastapi import FastAPI, Header

app = FastAPI()

@app.get("/")
async def get_info(token: Annotated[str | None, Header()] = None):
  return {}
``` ![pithink](https://cdn.discordapp.com/emojis/652247559909277706.webp?size=128 "pithink") this works for me

[sinful wadi]

literally what i have

what version of

fastapi

and python

are you using

[trim tangle]

oh, I think I did a brain mistake and I used an earlier version, sorry

[sinful wadi]

i dont mind using an earlier version

just tell me which one

[trim tangle]

0.88.0

[sinful wadi]

python 3.10?

[trim tangle]

Yep

Hm, works for me on 3.10.8 and fastapi 0.95.0

[sinful wadi]

now error is not being thrown

but actual application logic changed

its so over

i think more of a #web-development issue anyways at this point

thanks

[cerulean latch]

what even is class SomeClass(typing.Generic[T])? T being TypeVar("T")

ive seen this used somewhat a bit and ive always wondered what the point is

[trim tangle]

There's a tutorial 🙂 https://decorator-factory.github.io/typing-tips/tutorials/generics/

[pliant vapor]

how do I type hint the arguments for __exit__?

[soft matrix]

i generally wouldnt

just use *args: Any

assuming you arent planning to use them

[oblique urchin]

*args: object is better in that case

if you do want to annotate them the types are type[BaseException | None, BaseException | None, types.TracebackType | None

[pliant vapor]

gotcha

idk why I'm type hinting considering I'm about to override int and break stuff but

[dapper sierra]

yo wasgud gang

can yall help me with this line of code?

[trim tangle]

Please see #❓｜how-to-get-help and open a help post. Make sure to include:

1. What you're trying to do
2. The code as text
3. The error you're getting, or anything else that you don't expect to happen

[trim tangle]

It's always such a pain to type-hint __exit__ tbh

[hasty jungle]

where on earth isn't the semantic explicit enough, that people actually have to type-hint a dunder? 😄

[soft matrix]

abstract class property might work

[rare scarab]

I hate how I have to type hint an overridden method.

[frigid jolt]

smh just type ignore everything

[deep saddle]

probably because you're allowed to change the method signature in subclasses

although it seems reasonable that a typechecker could just assume that it's unchanged if no annotations are provided 🤷

[trim tangle]

the reason probably is -- it's not how it's defined in PEPs 🙂

[livid cypress]

was there any status update on the PEP?

[soft matrix]

!pep 612

[rough sluice]

**PEP 612 - Parameter Specification Variables**

Link

Status

Accepted

Python-Version

3.10

Created

18-Dec-2019

Type

Standards Track

[soft matrix]

It's been accepted for a long while

[livid cypress]

can something like the angle bracket syntax in typescript be done for functions in python?

say i do funcType which would make the return type to be Type

[soft matrix]

!d collections.abc.Callable

[livid cypress]

basically what i want is to be the return type of a fucntion to be generic, independent of types of arguments.

[rough sluice]

collections.abc.Callable

class collections.abc.Callable```
ABC for classes that provide the `__call__()` method.

[hearty shell]

How can a function be generic independently of type arguments?

[soft matrix]

i have no idea what syntax this is meant to be but im assuming youre looking for

from typing import TypeVar
T = TypeVar("T")
def identity(x: T) -> T:
    return x

[livid cypress]

i dont understand generics very well, i was just asking if there is a possibility of such a case.

say i want to make the return type generic but the function has no arguments, what do i do then?

[trim tangle]

that's pretty much the same as returning Any or object

[livid cypress]

def identity() -> T:
    return x

export function generic<T>(): T {
    return {} as T;
}

const test = generic<string>();

test has the type string in this context

[livid cypress]

so i need to type cast it or annotate the variable it stores the data in? thats what i have been doing

h6_labels: ResultSet[Tag] = h6s[0].find_all("span", recursive=False) this works because i get the result in a variable

span.text.strip() for span in li.find_all("span", recursive=False)

something like this doesnt since i cannot explicityl provide types for return of find_all, hence span is just Any and no type checks for .text and anything else

[hearty shell]

That isn't the identity function (it would be def identity(x: T) -> T: ...), and if you wanted an identity for a type (assuming that type has an identity), that would be observable runtime behaviour

[hearty shell]

That doesn't really make any sense though, it is just a cast, it isnt sound

function generic<T>(): T {
    return {} as T;
}

const test: string = generic<string>();
console.log(test.charAt)  /* returns underfined */

it still inst a string

[solemn sapphire]

how to get around mypy's inability to infer lambda types?
I used to use pyright, but something's very wrong with it and the node process it starts has started to go rogue. (memory keeps climbing past 1 gig and cpu usage is pinned at 100% usage)

I switched to py-lsp with the mypy plugin but mypy is sadly unable to infer lambda types. I did see one github issue but it didn't have much discussion in it.

[soft matrix]

pyright shouldnt leak memory like that, did you file an issue

!d collections.abc.Callable to infer lambas though you need to use this

[rough sluice]

collections.abc.Callable

class collections.abc.Callable```
ABC for classes that provide the `__call__()` method.

[solemn sapphire]

I am not sure if I bunged something on my end or if it is really pyright
Do you use pyright?

[soft matrix]

yes

[solemn sapphire]

using?

[soft matrix]

yes

[solemn sapphire]

sorry, I meant to ask using what editor

[soft matrix]

ah, vsc

i have in the past used it with st4 though

[solemn sapphire]

ah I don't use either hence asked

so pyright is indeed fine, its probably my doing
If it gives you some hint, I was recently trying to use multiprocessing and I <ctrl-c>'d the program. I thought it might have left a bunch of orphan processes so manually kill'd all the python processes. But One of them was pyright. Oops, I tried to re-launch my editor but pyright never came back to life

everytime my editor calls pyright-langserver --stdio it only starts a single node process which keeps hogging memory

[solemn sapphire]

what would be the difference between this and typing.Callable?

[soft matrix]

typing.Callable is deprecated

[soft matrix]

https://github.com/microsoft/pyright the maintainers might be able to help

[solemn sapphire]

Oh? didn't know that

so collections.abc is the way to go?

[solemn sapphire]

ok! I'll give it a shot

[soft matrix]

yes

[solemn sapphire]

thanks, will keep that in mind

[solemn sapphire]

Ok looks like something about my codebase is freaking out pyright

[solemn sapphire]

https://github.com/microsoft/pyright/issues/4900
filed an issue

[twin field]

It's an old message, but I thought you might get a kick out of a recent PR of mine: https://github.com/matplotlib/matplotlib/pull/24976

[terse sky]

Very impressive work!

Look forward to benefitting from it

[solemn sapphire]

Woah that's quite impressive!

[tranquil turtle]

Is there a way to get more info about why two types are incompatible?
I get this error: Incompatible types in assignment (expression has type "Point", variable has type "Sequence[float]"), i added all methods from Sequence[float] protocol and from all of its bases, but i still get this error

[soft matrix]

you might be able to tell if you make Point subclass Sequence[float] directly for a bit

hmm actually do you have the full definition of point?

all you should technically need according to cpython is len and getitem

[tranquil turtle]

stubs says i need this (pic 1) + __len__ from Container + __reversed__ from Reversible + __iter__ from Iterable
actually i have this (pic 2)

i dont want to subclass Sequence[float] explicitly

[trim tangle]

@tranquil turtle cold you post some runnable example maybe?

[tranquil turtle]

ok, give me a minute

[tranquil turtle]

https://mypy-play.net/?mypy=latest&python=3.11&gist=af8f28c108428a91df00c0d95cbccd2a

[soft matrix]

hmm its weird that it doesnt complain if you add Sequence to the base class

and there are no incompatible overrides

[trim tangle]

yeah

oh

hm

[tranquil turtle]

print(isinstance(Point(), Sequence)) says False

[trim tangle]

Is Sequence even assignable "implicitly"?

like Iterable

[tranquil turtle]

oh, it is not a Protocol...

[trim tangle]

oh yeah

[soft matrix]

ignore that then

i thought Sequence was implicitly a protocol

[trim tangle]

same tbh

[soft matrix]

https://github.com/python/typeshed/pull/1220

i cant work out why it wasnt included here

oh wait https://github.com/python/typeshed/pull/1220#issuecomment-345236629

so it looks like it just doesnt have the stuff at runtime to check if something meets sequence

thats a bit odd

[oblique urchin]

Generally only the very simple ABCs (one or two methods) were converted to Protocols

Sequence and Mapping and co were judged too complex

[hasty phoenix]

Given fileobj: Optional[IO]. I see hasattr(fileobj, "read") is commonly used to test for file-like behavior, but mypy doesn't follow that logic and thus gives warning. Is this a case where it's actually useful to cast?

[soft matrix]

just check if fileobj is not None

IO always implements read afaik

[hasty phoenix]

Yeah, sorry, I shortened my example a bit too much, fileobj: Union[None, IO, str} is more accurate. The baseline question is how to reliably test for file-like objects. Runtime hasattr() seems to be the most common way, but the type checker doesn't

[soft matrix]

just check if fileobj is not None and not isinstance(fileobj, str) or something to that effect

[hasty phoenix]

Ref this https://github.com/python/typing/discussions/1293#discussioncomment-4168023

[soft matrix]

does my suggestion not work?

[hasty phoenix]

It's inverting the group. It assumes that everything not None or str is file-like, which is not precisely the same as testing if something is file-like.

[soft matrix]

That's a very safe bet imo

If someone implements a subclass of str that implements io methods it's kinda on them for doing strange things like that

[hasty phoenix]

Given a: Dict[int, Any]; b: Dict[str, Any] how can I type this var = a.get(x) or b.get(x) when x: Union[int, str]. Mypy doesn't like x being int in b lookup and x being str in a lookup.

[acoustic thicket]

dont think you can, type: ignore looks like the only option to me

[soft matrix]

could also cast to Any

[trim tangle]

yeah it's unfortunate

theoretically, get and __getitem__ accept anything hashable, but for practical purposes, I assume, it was decided to make them accept only the key type

[hasty phoenix]

Yeah, there are at least two views: The type hint is to indicate what type to expect, or the type hint indicates what type hint to accept.

Assuming homogenous types in dict d, d.get(a) is most probably wrong if a is of different type than the key type of d. But the statement isn't incorrect thou, it'll just return None.

[soft matrix]

can anyone explain this error?

  No overloaded function matches type "(StrPath) -> _S@map"```or is pyright just having a bad time?

StrPath is str | PathLike[str] fwiw

it doesnt complain if i write out the tuple comp in full

[runic sleet]

Is there a way to type hint injected arguments by a decorator?

@some_deco
def func(ctx):
    reveal_type(ctx) # ContextClass

[soft matrix]

nope

[trim tangle]

@some_deco
def func(ctx: ContextClass):

[tranquil turtle]

Sentinel = Newtype('Sentinel', object)
some_deco = Callable[[Callable[[Sentinel],Sentinel], Callable[[ContextClass], None]]
@some_deco
def func(ctx: _) -> _:
      ...

From outside this will work as expected but it will cause problems inside this function

[reef yacht]

*args? That seems weird. That's clearly a LIST. It can't possibly ever be str

[trim tangle]

wdym

[soft matrix]

i assumed they wanted the typechecker to know without annotation that ctx would be Context

[tranquil turtle]

Yeah, it is not possible

[tranquil turtle]

*args is always a tuple, it cannot be a list. *args: str means that every item is a str, so args has type tuple[str, ...]

[reef yacht]

well.. ok, sure, it's a tuple, but is the annotation really like that? Looks quite weird to me. And with kwargs? Does **kwargs: int mean Dict[str, int]? Seems pretty special case:y to me

[buoyant swift]

https://peps.python.org/pep-0484/#arbitrary-argument-lists-and-default-argument-values

i also don't really like this, but it is what it is

[reef yacht]

huh. weird

The title of the section is wrong too 🥴 Kwargs isn't the same as "default arguments"

or at least it's incomplete

[soft matrix]

its the first bit

"Arbitrary argument lists"

and i think it makes a lot of sense

[reef yacht]

kwargs isn't lists, and, technically args isn't either as it's tuple :P

[soft matrix]

its an argument list not a list

[reef yacht]

"arbitrary arguments" would be correct

[dire bobcat]

In this pr here https://github.com/surrealdb/surrealdb.py/pull/50 i implemented a missing sentinel which in my opinion makes the (None, ...) tuple (MISSING, ...). In the case of the pr, when checking against None its impossible for the operand to be None accidentally, so by checking against MISSING it just makes the code more explicit that the None is a missing variable. Does it make sense what I've done or is it a useless implementation?

[soft matrix]

im personally against using a sentinel that is cast to Any in situations like this, youre losing sanity checks for no gain

[dire bobcat]

hm okay

[soft matrix]

None normally at least to me means something is missing

i would agree that changing the check to is not None is good though

cause the request_id could be ""

[dire bobcat]

okay thanks

[dire bobcat]

where would you use a missing sentinel?

[soft matrix]

where None is a valid value for something

first thing i can think of for this in practice is TypeVar's defaults https://github.com/python/typing_extensions/blob/main/src/typing_extensions.py#L1185

[rough sluice]

src/typing_extensions.py line 1185

default=_marker, infer_variance=False):```

[dire bobcat]

Okay, thanks

[soft matrix]

another is Messageable.send's content arg or dict.get's default argument

[hasty phoenix]

How do I annotate file-like IO classes?

from typing import IO
import io
class MyStream(io.RawIOBase):
    ...
def myopen() -> IO:
    return MyStream()  # mypy: Incompatible return value type (got "MyStream", expected "IO[Any]")  

However, adding IO mixin like this class MyStream(io.RawIOBase, IO) will result in mypy complaining about conflicting definitions for write and __enter__. Finally, using other IO classes, e.g. io.BufferedReader(raw), requires raw to be a io.RawIOBase so doing class MyStream(IO) is not an option either. Is there a right way to do that is eluding me?

[soft matrix]

generally i think the recommended way to deal with io (cause its a bit of a mess) is to just define a protocol describing the methods you need and then use that

and id return the concrete class MyStream

[hasty phoenix]

Yes. Problem is that myopen()'s output is used in a var that might store regular open(). I had hoped to use type IO here as a common type, but one might need to use Union[IO, MyStream] for the above reasons.

[hasty phoenix]

Indeed, IO is a mess!

I'm trying to figure out the precise typing for each IO method in my IO class, but even basic methods like read() has different prototypes in different io classes in stdlib. E.g. if read() returns None or just an empty object.

Using None is a "no-value" indicator in py is very commonplace. But combined with type hints, it becomes tedious to deal with everywhere. Especially when adding annotations to an existing library/package.

[hasty phoenix]

Is there any ways to inherit a typing annotation of a method parameter? def readinto(self, b) -> int defines bas Any. The super class is io.RawIOBase and it has a rich definition for b found in typeshed as def readinto(self, __buffer: WriteableBuffer) -> int | None: .... However WriteableBuffer is as far as I can see buried in typeshed.

[soft matrix]

no, but WriteableBuffer should be importable if you import it in an if TYPE_CHECKING block and then use stringised quotes for it

[trim tangle]

you can do ```py
if TYPE_CHECKING:
from dummy import Foo
else:
Foo = object

[hasty phoenix]

Or use from __future__ import annotations

[trim tangle]

won't help with type aliases

[hasty phoenix]

About that: I read in the typing guide that one should use TYPE_CHECKING sparingly, as it creates a different path through the code for the static code analyzer as for the runtime python. However, I find it really difficult to tackle circular dependencies when many additional imports are needed for type annotation. In one project I decided to try out that when an import exists exclusively to load typing definitions for a module, it is imported under TYPE_CHECKING. Is this considered very smelly?

[soft matrix]

its just how things work at this point aint much that can be done about it

until your code breaks cause of pep 649 ;)

[grave fjord]

It breaks sphinx autodoc unfortunately so yeah use it sparingly

You can use from __future__ import annotations and import the module rather than the type for circular imports

[soft matrix]

if you want a way around this via an extension upvote this https://github.com/microsoft/pylance-release/discussions/3822 😉

[grave fjord]

Oh you're adding a sphinx plugin that calls out to pylance?

[soft matrix]

yeah

[hasty phoenix]

I'm in a similar situation. I'm adding type hints to an existing and established py package and it has multiple classes named Variable that have different API (so they are unrelated). The difference in naming in the path sdo.Variable, pdo.Variable and so on. When hovering it sais "Variable" which helps nothing in resolving the actual source.

(its a bit of a job to type annotate something which seems to be intended as some sort of duck typing, but only goes quack in some cases.)

[runic sleet]

I mean like type hints in the decorator and somehow be inferable without type hints in the actual function

pycharm does this for pytest decorators that provide arguments, not sure if this is a standard typing feature

[trim tangle]

ah yeah no that's not a thing

[warped lagoon]

def caller(*args: P.args, **kwargs: P.kwargs) -> Unary[Callable[P, R], R]:
    def call(function: Callable[P, R]) -> R:
        return function(*args, **kwargs)

    return call
``` wish this worked :(

def caller(*args: Any, **kwargs: Any) -> Unary[DynamicCallable[R], R]:
    def call(function: DynamicCallable[R]) -> R:
        return function(*args, **kwargs)

    return call``` this one is much sadder

[hasty phoenix]

What is the way to declare that a function accepts an object and all derived from it? fn(a: A) ? And it works with class B(A) too?

[trim tangle]

yes

[soft matrix]

PEP 695 got accepted 🎉 🎉 🎉

[green gale]

YAYYY

[plain dock]

soon the brackets will change to the angle ones, and python and java will unite

[trim tangle]

eeeehh

not sure I like it

[green gale]

angle bracket generics suck

[trim tangle]

Extra syntax with no clear benefit

[green gale]

hm, i guess this doesn't change function call syntax.

[soft matrix]

(thats my next pep :))

[green gale]

but the fact that rust has angle bracket generics which can be used at call site is trivially ambiguous

[green gale]

yesss

[trim tangle]

isn't that exactly what turbofish solves?

[trim tangle]

and extra typing stuff to learn, given the fact that there's still no single tutorial/reference on typing stuff!

[green gale]

yes, and introducing the need for turbofish-equivalent would be annoying

[trim tangle]

you literally have to do a reduce operation on all the PEPs to figure out how to do stuff

and now there's one more step in this reduction

[rustic gull]

What's the pep about

The site doesn't load for me

[oblique urchin]

!pep 695

[rough sluice]

**PEP 695 - Type Parameter Syntax**

Link

Status

Draft

Python-Version

3.12

Created

15-Jun-2022

Type

Standards Track

[rustic gull]

Interesting

[tranquil turtle]

is this a valid syntax in pep695: x = add[int](1, 2) (considering add is a function)?

[soft matrix]

no it will raise a typeerror

[soft matrix]

hence ^

[tranquil turtle]

GenericAlias exists, so it can be extended to handle functions too

[soft matrix]

yeah thats what id propose

[tranquil turtle]

myfunc[args, ...] is handled by builtins.function.__getitem__, not by builtins.function.__class_getitem__ because functions are instances of builtins.function

[soft matrix]

oh yeah whoops

[tranquil turtle]

__class_getitem__ vs __getitem__ is very unobvious to me
i spent several minutes figuring out if i am right or wrong

[soft matrix]

i do actually want to add both

the other one would be a version of callable that supports the descriptor protocol so typing MethodType is possible

[tranquil turtle]

what do you think builtins.function[...] means?
builtins.function[[params...], rettype] is equal to Callable[[params...], rettype]?

[soft matrix]

p. much

but in a class it would bind so that it could mean the thing as a method

[tranquil wave]

is something like this possible

class Foo:
    a: str
    b: str

class FooProtocol(Protocol):
    def do_something(self) -> int:
        ...

def transform_foo(foo: T) -> MergeTheseTypesSomehow[T, FooProtocol]:
    ...

Foo.a  # OK
Foo.do_something()  # type checker error!
Foo = transform_foo(Foo)  # this would ideally be a decorator
Foo.do_something()  # OK

and please dont tell me the obvious and say "just subclass FooProtocol"

[acoustic thicket]

damn

thats great

[hearty shell]

https://github.com/python/typing/issues/213

Not possible

[slender timber]

scala syntax

[slender timber]

[brisk hedge]

So glad to hear that!

The current generics system results in a lot of boilerplate for what should be a simple generic class

The extra syntax is a lot more intuitive and arguably easier to teach

[pliant vapor]

looks like a pretty big change

I'm up for it tho

looks neat

[carmine phoenix]

perhaps you want to make the TypeVar T bound to FooProtocol ?

[vital sandal]

I'm using mypy for type checking and it's saying that Callable[[Set[T]], ...] (actual type) is incompatible with Callable[[Iterable[T], ...] (expected type)

Based on this table, that isn't true
https://docs.python.org/3/library/collections.abc.html#collections-abstract-base-classes

Why does mypy view these types as incompatible, and how can I fix it?

[oblique urchin]

you're expecting a callable that can take any iterable, and giving a callable that only takes sets

that's incompatible

callable parameters behave contravariantly

[vital sandal]

Should I just change the actual type to the expected type and pass in a set when I use the callable?

I don't understand what the fix is :/

[oblique urchin]

it depends on what your code is doing

[vital sandal]

I'll make a contrived version, hold on

class ListQueue(Generic[T]):
    def __init__(self, consumer: Callable[[Iterable[T]], ...]):
        self.queue = []


class SetQueue(ListQueue):
    def __init__(self, consumer: Callable[[set[T]], ...]):
        self.queue = set()```

I'd like to have a subclass of `ListQueue` that uses a set internally. When the queue reaches capacity, it's consumed by `consumer`

[oblique urchin]

you'd probably have to type the __init__ arg as still taking an Iterable[T]

which seems better anyway, that way users of the class don't need to care about what the class uses internally

[vital sandal]

Okay

Thanks

[young tundra]

Should I use typehint when getting input?

[soft matrix]

no

input is already fully typed

[compact mountain]

@young tundra in python2, yes! python3 not really

[oblique urchin]

Not sure what makes you say that

[compact mountain]

@oblique urchin python2's input cannot be type checked by the checker is what I assumed

been a while since I wrote any, but doesn't the return type change based on user input?

[oblique urchin]

you mean the input() that evals whatever the user gives it?

I guess type checking is the least of you worries if you use that

[compact mountain]

(:

[cedar sundial]

Are there any videos that explain and demonstrate type variance?

[leaden oak]

I don't know of any videos, but fix error has a page on about variance in his typing tips website: https://decorator-factory.github.io/typing-tips/tutorials/generics/variance/

wait no, I do know one video. https://www.youtube.com/watch?v=LoeEstgMXQs <-- this is a great primer on why variance even exists

[rare scarab]

A list of dogs is a collection of animals though

[oblique urchin]

it's even a sequence of animals

[leaden oak]

yes, the video does go into how immutable structures aren't invariant, don't worry

[dreamy cove]

hi, relatively new to python and type hinting. could anyone point me in the right direction here?

T = TypeVar("T")
P = ParamSpec("P")

def derive(*fs: Callable[P, T]) -> Callable[P, T]:
    """Add the functions `fs` to `T` as methods."""

    def inner(x: T) -> T:
        for f in fs:
            if getattr(callable, "__name__", None) is None:
                raise Exception("derived method missing name")
            setattr(x, f.__name__, f)
        return x

    return inner

Expression of type "(x: T@derive) -> T@derive" cannot be assigned to return type "(**P@derive) -> T@derive"
  Type "(x: T@derive) -> T@derive" cannot be assigned to type "(**P@derive) -> T@derive"

[tranquil turtle]

->Callable[[T],T]

[dreamy cove]

ah, gotcha

this seems valid to me? am i missing some kind of secret handshake?

class AsDict(Protocol):
    """Allows converting `T` into a `Dict[str, Any]`."""

    def asdict(self) -> Dict[K, V]:
        """Convert a `T` into a `Dict[str, Any]`."""
        raise NotImplementedError

TypeVar "K" appears only once in generic function signature

[trim tangle]

did you mean -> Dict[str, Any]?

[dreamy cove]

hmm, yeah i did

but as a general statement even if you're not using the type parameter in the body that doesn't mean it's pointless

[trim tangle]

It does, because the type variables aren't "linked" to anything

[dreamy cove]

i'm not used to python type hint quirks but in a compiled language like c++ or rust, those type parameters have meaning

[trim tangle]

Type variables exist to connect two types together:

def identity(x: T) -> T:
    return x

But this is not valid:

def foo() -> T:
    ...

[dreamy cove]

just realised i should have used @abstractmethod, whoops

apparently i'm not the first to bump into this: https://github.com/google/pytype/issues/704

[trim tangle]

yes, in Rust you can do something like ```rs
fn empty_dict<K, V>() -> HashMap<K, V> {
//...
}

But in Python it's just kinda... quirky yeah

[dreamy cove]

i'm very used to rust so it's been a strange learning experience

it influences the codegen, sure, but i mean the function declaration establishes a clear contract with the generics

python can't really do anything with the codegen but i was hoping to still get some benefits from the type checking

[trim tangle]

Well, in the way you wrote it, the only thing asdict can return is an empty dictionary

[dreamy cove]

ah yeah, i goofed it

class AsDict(Protocol):
    """Allows converting `T` into a `Dict[str, Any]`."""

    @abstractmethod
    def asdict(self) -> Dict[str, Any]:
        """Convert a `T` into a `Dict[str, Any]`."""

this is what i actually wanted

probably should update the comments lol

it's just a trait for dataclasses.asdict

[grave fjord]

Do you need abstractmethod on a Protocol?

[trim tangle]

you don't

[dreamy cove]

not sure, i'm the noobie here

https://peps.python.org/pep-0544/#protocol-members

it's mentioned in the pep though

[trim tangle]

huh, TIL

[soft matrix]

It just will give a runtime error when subclassing if you don't implement it

[dreamy cove]

how should i do it then?

[soft matrix]

i think your code is fine as is

this is something i do myself and i think is helpful

[compact mountain]

@dreamy cove you can remove the @abstractmethod. since Protocol is something that only mypy understands, and mypy will make sure that you can't make a class of a Protocol base class itself, it doesn't really serve any purpose

[soft matrix]

there is nothing stopping you inheriting from a Protocol subclass lol

[rare scarab]

mypy has no say about what happens at runtime

[trim tangle]

what happens at runtime stays at runtime

[hearty shell]

I never really got why you would inherit from a Protocol, shouldn't the decoupling be an advantage?

I guess it gives other users the option to inherit or not, so in that case it is better then ABC, I still find it quite odd

[soft matrix]

its the optional decoupling yeah

[rare scarab]

The whole benefit of protocols is that you can type a third party module

[soft matrix]

thats not the whole benefit lol

[rare scarab]

*one of the benefits

[dreamy cove]

if you want my spicy hot take: ||OO was a mistake but inheritance set the industry back 20 years||

i prefer interfaces and composition

[hearty shell]

I think at this point the spicy hot take would be saying "actually inheritance is great and we should use it more", poor OO, always getting the blame for bad design x)

[mellow drum]

Spiciest hot take: Python needs ||goto||.

[slender timber]

I do feel the need for goto from time to time

If a property returns Self when unbound i.e. instance arg in __get__ is None why does it not error when I do self.some_prop.some_attr but it does when I do the same for my custom descriptor?

Or is this something which an overload can solve?

i am curious if overload will allow a variant of Any and None

[soft matrix]

i cant reproduce this

[soft matrix]

an overload could however make it so this would work though, id recommend just defining __getattr__ on your descriptor so you can keep some of your typing

[compact mountain]

https://github.com/tushar-deepsource/python-goto
a fork of the original to make it work on 3.8+

[slender timber]

this is nice

I came across this code

class FieldsFromTypeHints(type(ctypes.Structure)):
    def __new__(cls, name, bases, namespace):
        from typing import get_type_hints
        class AnnotationDummy:
            __annotations__ = namespace.get('__annotations__', {})
        annotations = get_type_hints(AnnotationDummy, include_extras=True)
        namespace['_fields_'] = list(annotations.items())
        return type(ctypes.Structure).__new__(cls, name, bases, namespace)

can I remove the creation of AnnotationDummy and directly use get_type_hints(cls, include_extras=True)?

[oblique urchin]

I don't think so because cls is the metaclass

[mellow drum]

Those are nerfed gotos like C has. They can only jump within a function. Real gotos are like jmp instructions in assembler: They will transfer control flow anywhere, no questions asked. You might wonder: How does that interact with the interpreter's stack? goto's answer is: You're the programmer, so the stack is your responsibility. goto just does what you tell it to do, like a knife blade with no handle.

[slender timber]

do i ask a question related to metaclass implementation here or in #esoteric-python, its heavily related to type hints too

it involves ctypes too, idk where to ask

[compact mountain]

i really wonder if it's impossible to do that with this package

assuming that its opcode rewriting logic is flawless and handles nested functions/classes etc., you could write an encoding or something that wraps the entire file in a function def. now all the gotos are inside a function.

[trim tangle]

Nah it's not really possible

[quaint hedge]

hello, so i am new to python and like wanna create something like this

python file.py download "song"```

how can we do this

[trim tangle]

Hello, please see #❓｜how-to-get-help and open a help post

[slender timber]

what purpose does Annotated really serve in comparison with say descriptors?
I just imagine code like this

class Proxy:
    name = StringDesc(...)

getting transformed to

class Proxy(metaclass=AnnotatedMeta):
    name: StringAnno[...]

The latter doesn't allow any call expressions either right?

I have a lot of code like the first in my codebase, I was wondering if it did be semantically (and type hinting wise) better to shift to annotations in some form

[rare scarab]

Annotated is only useful for type checkers and any library that uses inspect

It means name won't be inferred to be a StringDesc instead of str at runtime

And descriptors require a bit of extra logic where normally you wouldn't need it.

[soft matrix]

also not everything uses Annotated as just descriptors

[compact mountain]

Annotated isn't even used by type checkers right? I thought it's meant for additional metadata that can be used by other type based libraries, usually at runtime

[rustic gull]

pyright supports Annotated iirc, mypy doesn't

[oblique urchin]

what do you mean by "support"?

pretty sure mypy ignores it just fine

[short dawn]

Hi all, (how) is it possible to define a typing.Protocol for primitives like int, str,...?

e.g. while I can easily specify the ExProtocol for Animal, how would I do it for int?

class ExProtocol(Protocol):
    def lala(self): ...

class Animal:
    def lala(self):
        return "lala"
    name: str

[frigid jolt]

Why would you want to implement a protocol for a primitive data type

[short dawn]

@frigid jolt I am thinking extension methods for primitives. Also not thinking about some specific use case, but just about the possibility

meaning: just for fun

[frigid jolt]

and what typing.Protocol has to do with it?

You just have to subclass the primitive data types and extend them with what you want

[short dawn]

I was asking about the possibility with typing.Protocols. Not about subclassing. Thx anyways interesting!

[short dawn]

If there are some other ideas on how to do that via typing.Protocols I am happy to hear and learn about those

[soft matrix]

I think you just need to manually define them all yourself

[short dawn]

@soft matrix sorry I can not quite follow. What do you mean define them all yourself? Coming back to the above example

a = Animal("hmm")
a.lala

Should be possible. But how (if possible) could I do that for primitives (int,...) via typing.Protocols?
meaning

1.lala

Basically I want primitives or also types from other libraries extend their capabilities

[soft matrix]

Oh lol yeah you can't do that, 1 it's a syntax error 2 it's not safe and 3 it doesn't play with typing at all

[short dawn]

2,3) at least for python I guess. There are plenty languages out there which can that safely & totally part of their type system

thank you! this was really helpful. Because I just wanted to know if it is possible. It seems like this is not possible (via typing.Protocol). Especially when comparing other languages I was not sure if it is possible in python or not

[soft matrix]

It is possible, you just shouldn't use it for anything serious

[short dawn]

@soft matrix At this point I would be very thankful for concrete code snippets what you are talking about. So far I only got some more or less helpful pointers in this channel 😅

[soft matrix]

!e ```py
import fishhook

@fishhook.hook_cls(int)
class IntExtensionMethods:
def lala(self):
print("This is bad")

1 .lala()

[rough sluice]

@soft matrix :white_check_mark: Your 3.11 eval job has completed with return code 0.

This is bad

[short dawn]

@soft matrix 🙏 . Interesting! Alright, so looks like it is not possible with Protocol in that case, thanks for clarifying

[frigid jolt]

I think that you misunderstood what typing.Protocol is used for

[short dawn]

For what is typing.Protocol used for @frigid jolt ?

[paper salmon]

a rust trait is the closest thing i know of to what you're trying to do, but protocols are only to facilitate structural subtyping, for example if you wanted to define a function that takes any object with a .open() and .close() method

!pep 544

[rough sluice]

**PEP 544 - Protocols: Structural subtyping (static duck typing)**

Link

Status

Accepted

Python-Version

3.8

Created

05-Mar-2017

Type

Standards Track

[short dawn]

👍yeah was thinking on these traits exactly bc they can also do it for primitives. Alright thx

Now the thing is from the mechanics they are very similar. This is the reason why I at first did not understand why the basic types and also other lib types are not supported

[obsidian solar]

How do people hint data types like PIL Image or numpy arrays?

I always try to hint as much as makes sense but sometimes it's difficult to google everything. Is there a standard method to hint for non standard types that I'm missing?

[frigid jolt]

do you mean if you need to type hint like this?

from PIL import Image

x: Image.Image = Image.new(...)

this is not required since Image.new(...) returns an Image object so x will never be unknown, it'll be Image

not sure if this was answer your question

[obsidian solar]

Should've tried out before asking but I couldn't think of it obviously. You definitely have answered my question because this works:

from PIL import Image
x = Image.open("...")
def fn(image: Image.Image = x) -> None:
    pass

[pliant vapor]

from typing import Protocol, TypeVar, overload, Any

L = TypeVar('L', contravariant=True)
R = TypeVar('R', contravariant=True)
LR = TypeVar('LR', covariant=True)
RR = TypeVar('RR', covariant=True)

class LeftAddable(Protocol[L, LR]):
    def __add__(self, other: L) -> LR:
        ...

class RightAddable(Protocol[R, RR]):
    def __radd__(self, other: R) -> RR:
        ...

@overload
def add(a: LeftAddable[L, LR], b: L) -> LR:
    ...

@overload
def add(a: R, b: RightAddable[R, RR]) -> RR:
    ...

def add(a, b):
    return a + b

is this a valid way to type hint add?

[mellow drum]

For NumPy arrays, use numpy.typing.NDArray. E.g.,

def f() -> NDArray[np.float64]:
    return np.empty((3, 3), dtype=np.float64)

[rustic lagoon]

yeah, you just also need to somehow type hind in the actual implementation

[hearty shell]

Here is a doozy for you

https://github.com/microsoft/pyright/issues/3368

[trim tangle]

cursed...

[pliant vapor]

actually cursed

[zealous skiff]

anyone worked with type hinting decorator return objects? my ide isnt recognising the return type by the decorator 🙃

[slender timber]

Send code

[zealous skiff]

Sure, its pretty complex - basically typehint annotations only work in some cases (i think when its at the "root" of the file, not a nested function.. The decorator model() takes the function, wraps it in a callable class and returns said class: https://github.com/GitToby/framelink/blob/add_tests_and_fix_pre_commit/src/framelink/core.py#L186

When i write a test for these functions though (in pycharm) the following shows an error on ctx.ref(...)

def test_model_link_dag(initial_framelink):
    pipeline, src_frame = initial_framelink

    @pipeline.model()
    def src_model_2(_: FramelinkPipeline) -> pd.DataFrame:
        n = 10
        data = {
            "id_col": list(range(n)),
            "colour_col": [random.choice(["red", "green", "blue"]) for _ in range(n)],
            "uuid_col": [str(uuid.uuid4()) for _ in range(n)],
            "int_col": [random.randint(0, 100) for _ in range(n)],
        }
        return pd.DataFrame(data)

    @pipeline.model()
    def only_blue_records(ctx: FramelinkPipeline) -> pd.DataFrame:
        src_2 = ctx.ref(src_model_2)
        src_2_blue = src_2.loc[src_2["colour_col"] == "blue", :]
        return src_2_blue

[rough sluice]

src/framelink/core.py line 186

def model(self, *, persist_after_run=False, cache_result=True) -> Callable[[F[T]], FramelinkModel[T]]:```

[zealous skiff]

when the code is the same, but at the root of the file it seems to work fine 🤷‍♂️ any thoughts?

This is the nested screenshot again with ide hints

same thing in vs code too

[trim tangle]

Well, initial_framelink is not typed, I wouldn't expect any guesses to its type from the editor

You should add a type annotation to initial_framelink

On the top level it works because pipeline's type is known

[zealous skiff]

its part of the pytest fixtures, its got implicit typing for both those params

[trim tangle]

ah, I suppose it's some magic from PyCharm

[zealous skiff]

adding the tuple type explicitly still has the issue 😦

its strange as pylance also has the same type warning but mypy is ok with it

seems odd that type hinting is not a fan of this pattern when inside a function - its not really a showstopper; just frustrating when writing tests

bit of a hack inside tests, but it works

def test_model_link_dag(initial_framelink):
    pipeline, src_frame = initial_framelink

    @pipeline.model()
    def src_model_2(_: FramelinkPipeline) -> pd.DataFrame:
        n = 10
        data = {
            "id_col": list(range(n)),
            "colour_col": [random.choice(["red", "green", "blue"]) for _ in range(n)],
            "uuid_col": [str(uuid.uuid4()) for _ in range(n)],
            "int_col": [random.randint(0, 100) for _ in range(n)],
        }
        return pd.DataFrame(data)

    src_model_2: FramelinkModel[pd.DataFrame]

    @pipeline.model()
    def only_blue_records(ctx: FramelinkPipeline) -> pd.DataFrame:
        src_2 = ctx.ref(src_model_2)
        src_2_blue = src_2.loc[src_2["colour_col"] == "blue", :]
        return src_2_blue

[carmine phoenix]

what is a py.typed file?

[fathom river]

It's a marker file that tells type checkers that the type hinting is distributed in your code (*.py files) and not in stub files (*.pyi files). https://peps.python.org/pep-0561/

[carmine phoenix]

wait so why does this directory have stub files and also the py.typed file?

https://github.com/explosion/spaCy/tree/master/spacy

[fathom river]

Not all python files have their respective stubs. Some have the typing in the code

[carmine phoenix]

oh so you can use both

interesting

[dense plank]

Can someone tell me why my server is rejecting hashlib??

[tranquil turtle]

no you cant
if you have file.py and file.pyi it is not ok
in this particular repo all type information is written in .py files, so there is a py.typed file to indicate that. But it also has stubs for modules that have no python source code (such as .pxd) so there are .pyi files for these files.
IIRC typechecker first tries to look into .py files (if they exist and py.typed also exists), and then if it has not found type information it looks into stubs

[elfin oyster]

ParamSpec cannot be used in lambda expressions - is this correct? pyright flags this usage as not allowed in this context.

E.g.,

from collections.abc import Callable
from typing import ParamSpec

P = ParamSpec("P")

f: Callable[P, int] = lambda *args: len(args)

[heady flicker]

Paramspec doesn't make any sense in this scenario. What you want is ...

See the usage of paramspec and typevar in docs for more details

[tranquil turtle]

... also accepts kwargs, isnt it?

[pastel egret]

Yes, it's saying you don't care what parameters the function has.

[elfin oyster]

Got it, thanks!

[rare scarab]

If you want to explicitly only accept *args, I think you can do tuple[Any, ...]

[tranquil turtle]

i dont think you can express f(*args,) signature with Callable
to do so you should define protocol with def __call__(self, *args)

[rare scarab]

callable (without paramspec) doesn't support **kwargs

[fading temple]

if i want to typehint a list of any type on numbers i should do this?

list[int | float]```
or just this
```py
list[float]```

[trim tangle]

list[float]

because int is assignable to float:

x: float = 42

which is... a bit questionable IMO, but that's what it is

[fading temple]

okay, thanks

[rustic gull]

in theory an int of like 1 is 1.0 imo

[rare scarab]

That means True is also a float.

[trim tangle]

Until you try to call .hex() or .is_integer() on it

[rustic gull]

I meant for type checking otherwise it would be cursed

[fading temple]

!e print(float(True))

[rare scarab]

!e print(float(True))

[fading temple]

bruh💀

[rustic gull]

Bots still typing tf

[rough sluice]

Sorry, an unexpected error occurred. Please let us know!

TimeoutError:

[hearty shell]

Wasn't True an alias to 1 at some point? I don't find the bool <: int too weird, the float thing yeah

[hearty shell]

Nvm, perhaps not

[rustic gull]

anyone know of a project that uses user defined mypy protocols for something cool?

[pastel egret]

bool was added in Python 2.3. There was already a lot of code that had defined False = 0, TRUE = 1 etc and it needed to not break. So bool inherits from int and until Python 3 True/False were just builtin globals that could still be shadowed.

See https://peps.python.org/pep-0285/

[hearty shell]

Yeah I saw the commit where they were added, I just miss remembered the False = 0 stuff, I thought for a second it was builtin aliases

[elfin oyster]

Thanks for the suggestion!

My query was mainly to get to know what’s possible with Callable as I was working on to fix an issue (https://github.com/charliermarsh/ruff/pull/3983) in ruff.

[young tundra]

Should I make typehint for every variable im creating?

[elfin roost]

short answer: No

[trim tangle]

The long answer is also no 😛

[restive warren]

Has anyone had luck type-hinting starlette/fastapi's request state? https://www.starlette.io/requests/#other-state

I can't find a good way to do it, but this might not be a typing issue but rather a usage error. Maybe there's a better way to share state throughout the transaction

Best I've come up with so far is to generate stubs for the entire module and manually add the properties to the State class

[viral bramble]

Depends on how you're using it, I guess, but if I remember correctly, I put my own object there, something like a dataclass. Then you only have one attribute on the state, and you can have a fastapi dependency that depends on the request, accesses the attribute, and forwards it into the handler. The dependency would basically just do return request.state.my_state. If you need, you can cast it right there for mypy etc. (or assert its presence / raise if its not there, or is the wrong type) and not worry about the typing elsewhere, as everything else can receive your object that is properly typed.

[restive warren]

That is pretty clever

I wonder if that behavior - or something similar - should just be a part of starlette's interface instead of having to work around it with other approaches

[stray summit]

anyone aware of a tool that exports the public type annotations of a package in a way that allows to see differences/compat issues between versions

[fading temple]

this typehint is correct?

def borrar_error(function: Callable):
    def wrapper(self):
        if self.pantalla.get() == "ERROR":
            self.pantalla.set("")

        function(self)
    
    return wrapper```

[fathom river]

Callable should receive some generic arguments

[stray summit]

its missing both the argument types for the callable as well as the return types for the wrapper or the wrapper type + it seems to be a lossy wrapper

[fading temple]

wdym with lossy wrapper?

and not sure about what arguments i have to typehint

the methods that use that decorator has no arguments

only self arg

[hearty shell]

How is this being used? Is the wrapper just supposed to always return None?

[fading temple]

yup

[hearty shell]

Wouldn't that just be looking at a git diff of stubs?

[fading temple]

usage example:

    @borrar_error
    def esborrar(self) -> None:
        if len(self.pantalla.get()) != 0:
            self.pantalla.set(self.pantalla.get()[:-1])```

[hearty shell]

Type hint it

def borrar_error(function: Callable[[T], object]) -> Callable[[T], None]: ...

or instead of object also use None

This way you don't lose the original type hint -> None

[fading temple]

what means T?

[hearty shell]

It means TypeVar, but here you would be fine using Any. TypeVar is just a way to have a hold in for a type, so it is just saying "Gets callable with argument of type (T) and return of type object and return callable which also takes type (T) and returns None". But any tutorial you see only will give you a better understanding

Also for that you need to declare it

T = TypeVar('T')  # This somewhere in global scope, preferablly at the top

def borrar_error(function: Callable[[T], object]) -> Callable[[T], None]: ...

It is not really important here, you could just use Any, using T just allows you to subclass

[fading temple]

i saw a lot of times that thing in the typing docs but never understood, so mindblowing for me

OH

i think i got it

thanks

[fading temple]

but just to clarify, Callable typehint works like this Callable[[argument type], return type]?

[soft matrix]

Yes

[fading temple]

then for my case it would be more correct to use this? since it take self as argument, and doesnt return nothing

def borrar_error(funcion: Callable[[object], None]) -> Callable[[object], None]:...```

[soft matrix]

Unfortunately since it's contravariant no that's not gonna work (probably)

[fading temple]

💀

[hearty shell]

even if it wasnt contravariant, and was instead covariant, which is what you are implying (Callalbe[[ClassWithMethodEsborrar], None] is compatible with Callable[[object], None]) that would also be "poor" typing because it is too generic, you could give a a callable X and get back a Callable[[str], None] because str is also an object.

If you dont plan on the decorator being used on anything other then ClassWithMethodEsborrar, then you would go for

def borrar_error(funcion: Callable[[ClassWithMethodEsborrar], None]) -> Callable[[ClassWithMethodEsborrar], None]:...

It is the most specific one, however you can't subclass due to the contravariance

With a variable T to correct, you would also have to bind it to the ClassWithMethodEsborrar, via T = TypeVar('T', bound=ClassWithMethodEsborrar), so that you have access to the self.pantalla attribute

But all of this is, in my opinion, so overkill for such a decorator, the main thing was just maintaining the -> None part of the public signature, what argument it is called with is not too relevant

[fading temple]

okay

[stray summit]

Stubs don't tell the compatibility story in particular when using metaprogramming for a compat layer

[rare scarab]

I suddenly got the idea to copy type casting from typescript (not as, <X>)

current cast: ```py
from typing import cast

x = cast(X, foo)

With diamond casting ```py

x = <X>foo

too bad it doesn't look like python.

[soft matrix]

This got denied as well afaik

[trim tangle]

I don't think it's a good idea

It makes an unsafe operation look ok

[rare scarab]

Yeah, I always forget I can do ```py
x: X = foo # type: ignore

[soft matrix]

I hate that

[rare scarab]

https://tenor.com/view/i-hate-this-data-tng-star-trek-the-next-generation-gif-19976541

[plain dock]

could be worse. could be Any

[rare scarab]

you don't need to type ignore that.

[soft matrix]

Sorry forgot to explain why

It's a rather massive footgun to have in your ide, I get a lot of false sense of security if there's nothing wrong in seeing no issues with my code

I find that I accidentally introduce a lot of bugs on lines with type ignores

That could be avoided using cast

[hallow flint]

if you're using mypy, using type ignore with the error code usually avoids that problem

[soft matrix]

Yeah but unfortunately I use Pyright

[carmine phoenix]

why do some libraries still use List, Dict, etc. from the typing module when you can use the builtins from like python 3.9?

[pastel egret]

They might want to still support Python 3.8 (which I have to do because of Win7), or it hasn't been important enough to go through and change them to the builtins.

[carmine phoenix]

in that case why dont they just use future imports?

[pastel egret]

That doesn't work in a bunch of places still - type aliases, base classes, etc. Easier to use the imports instead of trying to decide whether it'll work on this specific location or not.

[carmine phoenix]

you could put that kind of stuff under if TYPE_CHECKING:, that should do the trick

[pastel egret]

Not really.

[oblique urchin]

Some runtime typing libraries (e.g. Pydantic) won't work well with the future import

[rare scarab]

Is there an easy way to type this? mypy wants to infer it to tuple[object, ...]

def decorator(that: T, *args: P.args, **kwargs: P.kwargs):
  p_args = (that, *args) # <- type this

[trim tangle]

I think you'll have to use a TypeVarTuple

though you won't be able to use kwargs, yeah

[rare scarab]

Would implementing this as a descriptor be simpler type-wise?

that way I don't have to worry about self

[soft matrix]

Probably not considering how much of a pain they are to type if you want them to bind

[rare scarab]

Typing a descriptor is easy with overloads

[soft matrix]

oh is this not meant to be a method-like decorator?

[rare scarab]

It is.

[soft matrix]

how do you not run into https://github.com/python/typing/discussions/946?

[rare scarab]

I haven't actually tried it

[cedar sundial]

How would I go about typing a return value that depends on the value of a parameter? For this example, when upload is True, I want the return value to be None:
def create(self, merged_df: pd.DataFrame, upload: bool = True) -> None | list[pd.DataFrame]:

I've looked at overloads, but I realised that only works when a parameter has multiple types rather than values. When upload is False, I want to avoid having to check if the value is not None when I know it can't be

Nvm, using overloads was correct. Forgot about the Literal[True/False] that can be used in place of bool 🙂

[trim tangle]

I would make two methods instead

what does this method do?

[cedar sundial]

It's in a Tables class so I'm creating different tables based on a variable schema. I initially had it to always upload them to S3

[cedar sundial]

For this, we already have a function which uploads files to S3 so would it not be redundant to do something like this where we are just calling another function?:

class Tables:
    ...
    def upload(self, table: pd.DataFrame) -> None:
        save_to_s3(table, ...)

[slender timber]

is it possible to re-annotate an existing type hinted instance method?

[soft matrix]

This is a rather vague question, I'm assuming you don't just want to change a few characters manually

[slender timber]

its actually a function from pyqt, not my code. Basically the function is like this

findChildren(type: type, ...) -> QObject

I would like to use a typevar T here for the type and QObject here, else I need to cast the result everywhere

if there was a way to just "patch" in a fix?

[rare scarab]

Something like this? ```py
def find(typ: type[T]) -> T: ...

[oblique urchin]

you can provide your own stub file, but it has to duplicate the whole file

the pytype folks are developing an idea to allow you to override just some parts of a stub

[slender timber]

yes

[slender timber]

I'd rather use a few more type: ignores

[median tangle]

Is there some built-in protocol for objects that support all comparisons? (==, <, >)?

[oblique urchin]

no. Protocols for binary operations are generally difficult

[median tangle]

oh, so is there no way to do this then?

[void panther]

they have fallbacks that are annoying to type, or are just object in eq's case

[median tangle]

so, what should I do if I need 2 comparable types? Should I just type-hint them as any?

[rare scarab]

from __future__ import annotations

from typing import TYPE_CHECKING

if TYPE_CHECKING:
  from _typeshed import SupportsRichComparison

def foo() -> SupportsRichComparison:
  return ()

https://github.com/python/typeshed/blob/main/stdlib/_typeshed/__init__.pyi#L57-L73

Gotta put it in TYPE_CHECKING because _typeshed doesn't exist at runtime

[median tangle]

hmm, doesn't seem to work though

oh nvm, I just used it wrong, I was doing tuple[SupportsRichComparison, SupportsRichComparison], but doing tuple[T, T], with T = TypeVar("T", bound=SupportsRichComparison) did the trick

[rare scarab]

Make your own protocol that extends all the protocols listed in the linked pyi file section

[carmine phoenix]

is this the correct way to specify that if the 2nd param is provided it will return the tuple: tuple[pd.DataFrame, np.array], else: tuple[pd.DataFrame, None] ?

@overload
def get_smth_from_db(param1, param2: Literal[None]) -> tuple[pd.DataFrame, None]:
    ...

@overload
def get_smth_from_db(param1, param2: str) -> tuple[pd.DataFrame, np.array]:
    ...

def get_smth_from_db(param1, param2: str = None) -> tuple[pd.DataFrame, np.array | None]:
  ...

[oblique urchin]

pretty much! two small things are that you can just write None (no need for Literal)

and the type annotation in the implementation should be param2: str | None = None, because None is not a str

[carmine phoenix]

I removed the Literal but Pycharm isnt understanding the return type:

[carmine phoenix]

I always wondered why this is the case in Python because in C you just specify str and NULL is still a valid arguement

[soft matrix]

because passing NULL to a function that expects a string is honestly stupid

causes so many errors

[carmine phoenix]

fair enough, then why is that allowed in C?

[acoustic thicket]

C has very few restrictions in general

[soft matrix]

C is bad :)?

[carmine phoenix]

does anybody know how to fix this?

[carmine phoenix]

hmm I thought quite the opposite, perhaps its because I'm not used to (enforced) static typing

[soft matrix]

use a better linter lol

pycharm's is not great to say the least when it comes to type checking

id recommend either pyright or mypy

[rare scarab]

C: Here's a pistol. I don't know if it's loaded, so you should check before shooting yourself in the foot.

Also, the pistol still has the shell in there from the last time you shot yourself in the foot.

[celest iron]

Is it possible to type a function that returns a pandas DataFrame with column details (with their types)?

Also for polars since that's the new kid on the block.

I'm wanting to write some functions that returns strictly typed dataframes so you can move w/ ease as you write visualizations, etc.

---

Found this: https://github.com/CedricFR/dataenforce

[chrome hinge]

C: to shoot a bullet, please create a new bullet full of dummy data and pass a pointer to it to the gun() function to turn it into an actual one

[rare scarab]

C programmer: Made a typo, so instead of shot themselves, they shat themselves

[errant spire]

I really like beartype

[celest iron]

but this is runtime, i'm looking for mypy / pyright (for pylance integration)

[trim tangle]

beartype is really weird

namely the container-checking strategy

(it only checks one random element of a nested list/dict)

[celest iron]

i'm mostly want to write functions with schema'd dataframes so that the user can rely on certain columns existing

I found this: https://github.com/nanne-aben/strictly_typed_pandas

exactly what I need, though I'd like something more general so it works w/ polars, might considering PRing or something

[slender timber]

NULL is a valid part of a string, its used as a terminator. A better way to serialise a string would be to prefix it with size.

[buoyant swift]

no? NULL is conceptually different from the null terminator, even if both might have the same value

NULL is a pointer value, but '\0' is a char

[buoyant swift]

in Python, None is its own separate type. but in C, NULL is a valid value for strings, because a string in C is just an array of chars (which decays to a pointer pretty much whenever you do anything with it), and null is just a null pointer

[errant spire]

I use both mypy/beartype for runtime enforcement

[celest iron]

that doesn't answer my question?

[errant spire]

Pandera?

https://pandera.readthedocs.io/en/stable/index.html

[buoyant swift]

mypy at runtime?

[errant spire]

Beartype at runtime. Mypy pre-commit

Beartype also does some dbc (design by contract, pre and post conditions)

Deal looks interesting as well. https://deal.readthedocs.io/

[celest iron]

i want lang server support via pylance lol

[buoyant swift]

just use pyright?

[celest iron]

yea, but how do I type the schema of a dataframe

[celest iron]

https://github.com/nanne-aben/strictly_typed_pandas
is this the best? basically

[soft matrix]

is there a way to have mutually dependant generics e.g. Channel is generic over the type of Message thats contained in it but Messages store a reference to their channel?

currently what im doing is annihilating pyright

[tranquil turtle]

does this make sense? ```py
class Channel(Generic[T]):
message: Message[Channel[T]]

class Message(Generic[T]):
channel: Channel[Message[T]]

[soft matrix]

leaving it unspecialised doesnt work i dont think

what i have currently is something along the lines of ```py
from typing import Any, Protocol, Self, runtime_checkable
import abc

@runtime_checkable
class MessageableMessageT: "Message[Any, Any]" = "Message[Any, Any]":
...

class ChannelMessageT: "Message[Any, Any]" = "Message[Any, Any]":
...

class User: ...

class Message[UserT: User, ChannelT: Channel[Self]]:
...

which pins pyright

[tranquil turtle]

class Channel(Generic[MessageT]):
    message: MessageT[Self] # specialized typevar, it is not possible now

class Message(Generic[ChannelT]):
    channel: ChannelT

[soft matrix]

yeah you need something along the lines of hkt then

right well thats a pain in the ass

[soft matrix]

@oblique urchin this why i wanted typing.get_type_parameters

[median tangle]

I've been trying to use the _typeshed.SupportsRichComparison, however the way it's defined doesn't allow me to specify what type the comparison is supported with.

I could define my own type with SupportsComparison: TypeAlias = _typeshed.SupportsDunderGT[int] | _typeshed.SupportsDunderLT[int], which would work if I knew the type this comparison should be supported for.

However I need this to be a generic case, where my fucntion takes 2 values, which can be compared with each other, and returns one of those values. So, as a non-working example, I want something like this:

def pick_bigger(x1: ComparableWith[U], x2: ComparableWith[T]) -> T | U:  # where T gets bound to x1, and U gets bound to x2
   if x1 > x2:
     return x1
   return x2

I thought of something like: ```py
from typing import TypeVar

from _typeshed import SupportsDunderGT, SupportsDunderLT

TComparableWithU = TypeVar("TComparableWithU", bound=SupportsDunderLT["UComparableWithT"] | SupportsDunderGT["UComparableWithT"])
UComparableWithT = TypeVar("UComparableWithT", bound=SupportsDunderLT["TComparableWithU"] | SupportsDunderGT["TComparableWithU"])
``` but pyright gives TypeVar bound type cannot be generic here

If this is completely impossible, would it at least be possible to do this with a single typevar, that's comparable with the same type? So that the function is generic over any same 2 types, which support comparisons against itself?

[soft matrix]

i remember someone did write this for something in this channel in a gist

.

might be a bit more helpful, i cba finding the original

[wraith linden]

Just curious: how would you go about annotating the following protocol? ```py
class Mappable(Protocol):
def map(self, fn):
...

[oblique urchin]

>>> class Mappable:
...     def map[T, U](self, fn: Callable[[T], U]) -> U: ...
... 

[wraith linden]

Sorry to be clear, an implementation of the protocol should be some kind of generic container that allows mapping over its elements.

[oblique urchin]

so more like this ```>>> class Mappable[T]:
... def map[U](self, fn: Callable[[T], U]) -> U: ...
...

!pep 695

[rough sluice]

**PEP 695 - Type Parameter Syntax**

Link

Status

Accepted

Python-Version

3.12

Created

15-Jun-2022

Type

Standards Track

[oblique urchin]

(for this syntax)

[wraith linden]

I thought about doing something like py class Mappable(Protocol[A]): def map(self: Self[A], fn: Callable[[A], B]) -> Self[B]: ... but the way Self works it doesn't allow type arguments.

And this doesn't work because type vars can't be bound by generics with free parameters: ```py
class Mappable(Protocol[A]):
def map(self: MappableT[A], fn: Callable[[A], B]) -> MappableT[B]:
...

MappableT = TypeVar('MappableT', bound=Mappable)

[oblique urchin]

Oh I see the issue now. Yeah you need higher-kinded types, which aren't a thing in the Python type system. Best you can do is return Mappable[B]

[wraith linden]

Ah right I see

[wraith linden]

Ah yeah, I was actually reading this pep, then when I got to this section where it says "the specified upper bound type must be concrete" it got me thinking: https://peps.python.org/pep-0695/#upper-bound-specification

[oblique urchin]

That part isn't new (it's already how type checkers work), but this PEP spells it out more explicitly

[wraith linden]

I'd just never really thought about it before tbh

[mystic harness]

wouldnt it be better if Optional made the union from the default? something more like:

MySentinel = object()
def foo(arg: Optional[int] = MySentinel) -> None:
  ...

# ^ would be this instead of `int| None`
def foo(arg: int | MySentinel = MySentinel) -> None:
  ...

ah i can just use Any:

MySentinel: Any = object()
def foo(arg: int = MySentinel) -> None:
  ...

[rare scarab]

Mypy has an option to imply union types from default args

Though sentinels are usually its own class. ```py
class Sentinal: pass
sentinal = Sentinal()

[mystic harness]

i use pyright rn

[rare scarab]

Pyright only does this for None then

[eager vessel]

None now should be explicit 🤔

[median ledge]

Context: Using VSCode with the pylance language server.

Is it possible to "merge" the type information from a .py and its .pyi?
More concretely, I have a few subclasses of Foo which all use Foo's __init__, but I also create the stub at runtime, which has a __init__for each subclass (with different args).
However, if I have a stub file, the linter doesn't consider the source file for type hinting.

I'll run a setup method for generating the stubs, then I can work with the proper autocompletion.
However, I don't want to have to update the stub for things that are already annotated in the source anyway.

[slender timber]

Type annotations are programmatically accessible, in multiple ways. You can use them to decide what you want to annotate

[median ledge]

So I should programmatically fetch the necessary imports and annotations of other methods, then generate the init myself?

Any pointers on how to do that btw?

[rare scarab]

You can either use inspect.get_annotations(func) or typing.get_type_hints(func)

though .pyi files are not read at runtime

Maybe this will help. https://github.com/google/merge_pyi

[median ledge]

This seems to be what I need, I'll try it later

[median ledge]

Oh, I may have been a bit unclear in my original question.
By "merge", I mean get the linter to use type hints from both the source and the stubs. Rn, if I define a stub, the source is ignored and it doesn't show everything in the autocompletion.

[rare scarab]

maybe try pytype?

Why would you need both?

pyi is for when you're otherwise unable to provide type information inside the .py file

e.g. using python <3.5 or using native bindings

[oblique urchin]

There's a mypy feature request for this but I think we're unlikely to implement it

[median ledge]

The init is this (in the baseclass and it isn't overridden)

class Foo():
    def __init__(self, **kwargs):
        for k, v in kwargs.items():
            setattr(self, k, v)

[rare scarab]

You could assign an Unpack to kwargs

[median ledge]

I want proper autocompletion but don't want to write all the inits

[median ledge]

Idt I've seen that before

[rare scarab]

from typing import TypedDict, Unpack

class MyKwargs(TypedDict, total=False):
  attr1: str
  attr2: str
  attr3: int

def foobar(**kwargs: Unpack[MyKwargs]): pass # or **kwargs: **MyKwargs if 3.11+

Have you considered using a dataclass?

[median ledge]

Well, this is for a custom ORM I'm making

[rare scarab]

then dataclass_transform?

[oblique urchin]

The **kwargs syntax isn't in 3.11, only *args. We proposed it for 3.12, but the Steering Council didn't want the new syntax

[rare scarab]

ah.

so it's just Unpack

[median ledge]

I just skimmed the pep and that seems perfect, thanks.

[carmine phoenix]

hey guys, how do I make typing.Callable take **kwargs?

[oblique urchin]

you can't do that with Callable directly. Use a callback protocol instead (a Protocl with a __call__ method)

[carmine phoenix]

I see

can I just ignore the part that takes **kwargs or would that be considered bad practice?

[oblique urchin]

It might break type checking depending on how you're using it

[median ledge]

Shit, that doesn't work as I use descriptors, so I have
param = Text(...) instead of param : str

[rare scarab]

I've done it.
ORM bases: https://github.com/killjoy1221/ValhallaSkinServer/blob/3fde61e914c8102d45d94bca13aca3b456ca9ebf/valhalla/database.py#L111-L116
Model classes: https://github.com/killjoy1221/ValhallaSkinServer/blob/3fde61e914c8102d45d94bca13aca3b456ca9ebf/valhalla/models.py#L19-L26

It should be simpler for you since you're not wrapping sqlalchemy (right?)

note that annotations are required.

If you don't rely on it being a classvar, you can use Annotated

e.g. param: Annotated[str, Text(...)]

[median ledge]

Nope, fully my doing (needed an async ORM and I didn't like the syntax of any I found)

[rare scarab]

sqlalchemy is async

better support coming in 2.0

[median ledge]

Takes a bit more typing, but that's ok.
One thing I don't like is that it shows the descriptor call as the default value, i.e. (*, param: str = Text(nullable=False, default="spam")) -> None
Is it possible to have it show (*, param: str = "spam")) -> None?

[median ledge]

Didn't notice that at first glance + too lazy to learn it + didn't quite like the syntax (although it seems to have ended up similar 😅)

[rare scarab]

Which is why I wrote a fake descriptor type with __get__ overloads

[median ledge]

That looks like what I did, except it's a real descriptor

[median ledge]

OH, that's what field_identifiers does.

[rare scarab]

It refers to dataclasses.field

[fading temple]

it is correct to use
Callable[...]

[oblique urchin]

no, that has no defined meaning

Callable[..., int] means a callable that takes unspecified arguments and returns an int

[fading temple]

then if it takes no args and returns None the type hint will be
Callable[[None], None]

im right?

[oblique urchin]

that would mean it takes a single argument that is always None

no args would be Callable[[], None]

[fading temple]

oh, okay

and if the Callable is a method, i mean takes self as argument

it is necessary to typehint or not?

[oblique urchin]

The type hint should reflect how you call it

So an unbound method would need an extra arg entry for the self argument, but a bound method would not

[fading temple]

my case is a tkinter app and im passing a class method as button command

and im doing a child class that needs to take that command and inicialitze the tkinter button

[acoustic thicket]

that username seems familiar 👀

[merry current]

from typing import Self

class Shape:
    @classmethod
    def from_config(cls, config: dict[str, float]) -> Self:
        return cls(config["scale"])```cls should be left unannotated here?

[trim tangle]

yes

[merry current]

https://cdn.discordapp.com/emojis/626460111509913630.webp?size=64&quality=lossless

Thnx

[merry current]

from typing import overload

@overload
def foo(num: A) -> A:
    ...

@overload
def foo(num: B) -> B:
    ...

def foo(num: A | B) -> A | B:
    """actual code here"""```Any shorter way to write this?

[spiral fjord]

a typevar?

[oblique urchin]

it's not exactly the same, but def foo[T: A | B](num: A | B) -> A | B: ... is pretty close

(or T = TypeVar("T", bound=A | B) on old versions of Python)

where "old" is any released version