type-hinting

[jade viper]

Btw, should return types be None or NoReturn?

Type hints I mean

[oblique urchin]

almost certainly None

NoReturn is used in the rare case where a function is expected to never return

like sys.exit()

[jade viper]

Ooooh I see

Ty 🙂

[rare scarab]

Or the only way it returns is via an exception

[cosmic cave]

is NoReturn also for functions that always raise an exception?

EDIT beat me to it

[oblique urchin]

sys.exit() always raises an exception 🙂

[rare scarab]

def main() -> NoReturn:
  while True:
    ...

[soft matrix]

NoReturn is also the bottom type so it is valid to annotate variables as it which is weird

[oblique urchin]

however, I would recommend using it only for cases where users expect that it never returns, not for e.g. abstract methods that raise NotImplementedError

[soft matrix]

Especially before Never was a thing

[oblique urchin]

that's why we added Never

[rare scarab]

It's comparable to typescript never

Oh wait

[oblique urchin]

Never and NoReturn mean the exact same thing in the Python type system

[rare scarab]

Where is our Unknown type?

[soft matrix]

It's object ;)

[jade viper]

Any?

[rare scarab]

Any is not unknown.

Any is an escape hatch

[jade viper]

Wdym?

[soft matrix]

Early drafts of pep 696 had Unknown

[rare scarab]

Unknown would be for when you don't know the type, but you don't want to be able to assign it to anything

[oblique urchin]

sounds like object

[rare scarab]

I guess it makes sense that typescript needs an unknown. Not everything is an object

https://stackoverflow.com/questions/51439843/unknown-vs-any#51439876

[cosmic cave]

Is there a web-based sandbox for pyright? I'm almost positive someone linked to it a while ago, but every time I try to find it, I come up empty

just to type some code and see pyright's diagnostics, I don't need to run the code

[oblique urchin]

https://pyright-playground.decorator-factory.su/ but I think the SSL cert needs fixing

[rare scarab]

There's one for mypy too that works

Does pyright work on vscode.dev?

[oblique urchin]

probably, haven't tried it though

[rare scarab]

When you install the python plugin, it warns that some auto completion is limited

You also can't install python packages

Because no terminal

[oblique urchin]

pyright is written in TypeScript

[rare scarab]

Just listing other limitations of vscode.dev

[young zealot]

Is there a way to make a class variable which is an instance of the class while keeping mypy happy?

[tranquil turtle]

class X:
    x: X

[viscid spire]

Thought that required from __future__ import annotations

[leaden oak]

It does:

>>> class X:
...     x: X
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in X
NameError: name 'X' is not defined

[rare scarab]

Quote it.

[tranquil turtle]

or use python3.13

[oblique urchin]

we haven't implemented PEP 649 yet 🙂

[hasty jungle]

that's more subtle than that, and has nothing to do with the any/unknown dichotomy;

unknown tells the type checker to be rigid. you basically say to it: "you do not have enough information to infer the type here"
any means pragmatically the opposite: "whatever you think it can be, consider it is"

definitly not related to who is an object and who isn't, it's types . and I see no reason why Python couldn't have one like this 🤔 (theoretically, I mean)
the typescript object is not related to being an object at runtime or not. according to the doc, it is the type for "everything that is not a primitive"; however, the situation is a bit confusing in javascript, as the notion of primitive is not the same as in langs like java

[fluid jay]

so i'm writing into a database with sql and one column has the date type, which is iirc YYYY-MM-DD.
what should the type hint be with pydantic? is it str?

[acoustic thicket]

ideally it should be parsed into datetime.date i guess?

[blazing nest]

How do I type a union based off of a boolean? I have a enabled field in my dictionary, if it is set to True all other fields will be included, otherwise that is the only one passed:

class Disabled(TypedDict):
    enabled: False

class Enabled(TypedDict):
    enabled: True
    other_field: str

Alias = Enabled | Disabled

This of course fails at enabled: False, but is there anything I can do which would accomplish this?

[hasty jungle]

Literal[False]?

[blazing nest]

Ah! Yes that seems correct, great!

[hasty jungle]

btw, why pattern-matching on a boolean, if you already subtype ? why not just isinstance pattern-matching?

[blazing nest]

These are TypedDict's, so I am typing data I receive/produce from an API

[hasty jungle]

hm, not sure I understand, but okay I guess you know

[trim tangle]

TypedDicts are just dicts at runtime

[hasty jungle]

but also their subclasses?

I mean, still don't get why instance-pattern mathcing couldn't be applied 🤔 , instead of keeping a redundant boolean

[trim tangle]

"a TypedDict" means a subclass of TypedDict, yeah

[hasty jungle]

yeah but you loose the subclassing?

to clarify my question: instead of:

def bla(d: Alias):
  if bla['enabled']:
    ...
  else:
    ...

do

def bla(d: Alias):
  assert isinstance(d, Enabled) or isinstance(d, Disabled)
  assert not (isinstance(d, Enabled) and isinstance(d, Disabled))
  if isinstance(d, Enabled):
    ...
  else:
    ...

but I guess the enabled comes from the API too? (I guess?)

[dull lance]

!e

from typing import Literal, TypedDict

class MyDict1(TypedDict):
    foo: Literal[True]

class MyDict2(TypedDict):
    foo: Literal[False]

d1 = MyDict1(foo=True)
d2 = MyDict2(foo=False)
print(isinstance(d1, dict), isinstance(d1, MyDict1), isinstance(d1, MyDict2))
print(isinstance(d1, dict), isinstance(d2, MyDict1), isinstance(d2, MyDict2))

[rough sluice]

@dull lance :x: Your 3.11 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 11, in <module>
003 |     print(isinstance(d1, dict), isinstance(d1, MyDict1), isinstance(d1, MyDict2))
004 |                                 ^^^^^^^^^^^^^^^^^^^^^^^
005 |   File "/usr/local/lib/python3.11/typing.py", line 2994, in __subclasscheck__
006 |     raise TypeError('TypedDict does not support instance and class checks')
007 | TypeError: TypedDict does not support instance and class checks

[dull lance]

Well, that answers your question

so the best thing you can do is a discriminated union

[hasty jungle]

okay clear to me!

[quiet vortex]

Is there a built in type hint that suggests Success/Failure instead of just using bool true/false?

[tranquil turtle]

Success: TypeAlias = Literal[True]
Failure: TypeAlias = Literal[False]

def f(a: int) -> Success | Failure: return bool(a)

x = f()
if x == Success:
    ...
else:
    ...
``` ?

[viscid spire]

I don't think that should exist

a well-named function makes it clear what True/False signifies on return

[celest iron]

Success = True
Failure = False

yea? oh nvm, u'd need Literal

[quiet vortex]

Eh. I understand that for somethings. But it also makes naming cleaner in languages like Rust or Go where it's clear Result etc.

[hasty jungle]

applying patterns from one language to another, without taking the whole picture into account, might really well be one of the less fruitful exercise you could make ;D python and rust have very different exceptions philosophies, for example

[quiet vortex]

I never said I'm trying to force it into python. It was a question so I could use it if it exists. I'm not trying to create it.

[hasty jungle]

in that case, if it was me, I'd do a NewType('Success', Literal[True]) (if pytohn allows that), so that not every bool can accidently be mistaken with a success/error dichotomy

[rustic gull]

!e

[rough sluice]

Missing required argument

code

Command Help

!eval [python_version] <code, ...>
Can also use: e

Run Python code and get the results.

This command supports multiple lines of code, including formatted code blocks. Code can be re-evaluated by editing the original message within 10 seconds and clicking the reaction that subsequently appears.

The starting working directory /home, is a writeable temporary file system. Files created, excluding names with leading underscores, will be uploaded in the response.

If multiple codeblocks are in a message, all of them will be joined and evaluated, ignoring the text outside them.

By default, your code is run on Python 3.11. A python_version arg of 3.10 can also be specified.

We've done our best to make this sandboxed, but do let us know if you manage to find an issue with it!

[rustic gull]

!e 3.9 print("Hello There!")

[oblique urchin]

consider using #bot-commands

[rustic gull]

okay

thanks

[rare scarab]

Ugh, this is gross.
https://github.com/agronholm/anyio/blob/3.x/src/anyio/abc/_tasks.py#L89-L95

[rough sluice]

src/anyio/abc/_tasks.py lines 89 to 95

@abstractmethod
async def start(
    self,
    func: Callable[..., Awaitable[Any]],
    *args: object,
    name: object = None,
) -> Any:```

[rare scarab]

argonholm/anyio#511 seems promising

agronholm/anyio#511

[mighty linden]

GitHub

[anyio] #511 Add opt-in fuller typing for callables accepting (func, *args, ...)

[trim tangle]

You can make an enum

!d enum.Enum

[rough sluice]

enum.Enum

class enum.Enum```
*Enum* is the base class for all *enum* enumerations.

[trim tangle]

If you want a Rust enum aka discriminated union aka sum type, you can use a Union of dataclasses

[tranquil turtle]

class X:
    def f(self, x: int) -> int: ...
class Y(X):
    def f(self, x: int, y: int) -> int: ...
# Method "f" overrides class "X" in an incompatible manner
# Positional parameter count mismatch; base method has 2, but override has 3
``` why this is not correct?
i dont think it violates LSP

opensr.py:133: error: Signature of "f" incompatible with supertype "X"  [override]
opensr.py:133: note:      Superclass:
opensr.py:133: note:          def f(self, x: int) -> int
opensr.py:133: note:      Subclass:
opensr.py:133: note:          def f(self, x: int, y: int) -> int
``` mypy is also not happy with it

[oblique urchin]

it does violate LSP. Given an object x of type X, you can do x.f(1)

but if it's actually an instance of Y, that will break

[tranquil turtle]

hmm

that makes sense

[young hawk]

I have a classmethod on a mixin that returns an instance of of the subclass class. I'm trying to figure out how to make mypy happy with the typings though.

def get_config(cls: type[_Config]) -> _Config:

But when I call it like this

config = Config.get_config()

the type I get back from that is not Config. reveal_type(config) is telling me

note: Revealed type is "_Config`-1"

Any suggestions?

[tranquil turtle]

works for me (pyright): ```py
from typing import TypeVar

T = TypeVar('T', bound='X')

class X:
@classmethod
def get(cls: type[T]) -> T: ...

class Y(X): ...

x = X.get()
reveal_type(x) # Type of "x" is "X"

y = Y.get()
reveal_type(y) # Type of "y" is "Y"

[young hawk]

I think that's basically identical to what I'm doing but I'm using mypy

huh, something is different though, your code works for me too, going to dig more. really appreciate it 🙂

[oblique urchin]

maybe you didn't annotate cls?

[young hawk]

oh interesting, I had added @lru_cache to my class method and it seems to have corrupted the typing

[tranquil turtle]

yeah, it is removing type information from function completely

[young hawk]

seems like that decorator has some pain points, to say the least

[tranquil turtle]

you can do something like this: ```py
if TYPE_CHECKING:
def lru_cache(f: T) -> T: ...
else:
from functools import lru_cache

[young hawk]

it looks like cachetools with the corresponding typeshed stub package doesn't have this issue

[trim tangle]

@oblique urchin I wanted to take a stab at making an experimental static analysis tools to explore some alternative typing things. Do you have any advice on building such a thing?

I think the biggest first hurdle will be stuff like resolving scopes, modules, etc; and doing control flow graph stuff. Is there some way I can avoid that maybe?

[oblique urchin]

nice! Yes, you'll likely need to implement scoping rules. I suppose the only alternative is reusing the implementation of some existing type checker or linter. My pyanalyze has an ast_annotator mode (https://github.com/quora/pyanalyze/blob/master/pyanalyze/ast_annotator.py) that adds inferred types to an AST. I think pylint uses an underlying AST analysis module (astroid) that might do something similar, but I'm not familiar with it.

[trim tangle]

Hmm, astroid seems interesting

Honestly I'm not sure I have the energy to start such a project now. Maybe I'll stick to thought experiments for now

I wanted to take a stab at a sort of an alternative implementation of types in Python. LIke TypeScript for Python.

[oblique urchin]

that sounds ambitious

what would you want to change?

[trim tangle]

I kinda just want to fix the humongous complexity of the type system... There are a lot of rules, and you need to read a lot of PEPs to understand it.

Last idea I had was unifying TypeVarLikes into a single concept, kinda

Like they are in TypeScript

So in other words, try to make it more uniform

[rare scarab]

At least we have this. https://typing.readthedocs.io

[trim tangle]

yeah I suppose

[rare scarab]

that's published from python/typing/docs btw

[cosmic cave]

I think there's a pip proposing a proper generics syntax to get rid of the TypeVar hack

[viscid spire]

PEP

[cosmic cave]

Hah yeah my bad

[viscid spire]

Already added to 3.12

[cosmic cave]

In your experiment, are you thinking of improving tuple types so they can type non-tuple sequences, like in TS?

[trim tangle]

That's just syntactic sugar though.

It doesn't unify them really

[cosmic cave]

Yeah I get what you're saying. paramspec is limited to params, it's not unified with more general tuple type concepts like in TS

[rare scarab]

Do you mean like type Foo = (str, int)?

[trim tangle]

Yeah, e.g. in TypeScript there's no need for a separate TypeVarTuple, it's just a type variable holding a tuple type

TypeScript doesn't have keyword arguments, so it's a bit different

[cosmic cave]

Still could work, the kwargs type would be a typed dictionary

[rare scarab]

there's hacks for it, but it isn't a builtin language feature

[trim tangle]

Well, ParamSpec has to account for the fact that some arguments can be passed positionally or by keyword

[cosmic cave]

There's also the weirdness with typing rest params, where the type annotation is not a sequence type, so you don't get the option of typing it as either a sequence of T or a tuple type

[trim tangle]

You can do it with Unpack I think

[viscid spire]

can you do either * or ** on a TypedDict to substitute Unpack for kwargs in 3.12?

[oblique urchin]

no, that syntactic change was rejected

you have to use Unpack

[viscid spire]

damn...

[rare scarab]

Did they ever decide on the dict[{"foo": str}] TypedDict shorthand?

I don't think it was ever in a formal pep

but pyright has experimental support for it

[oblique urchin]

there's no PEP yet

Nikita is working on it

[viscid spire]

using a dictionary literal in a typehint could be useful for Callable

[rare scarab]

It's no different to me than tuple[x, y]

[viscid spire]

actually thinking about that lol

pass a tuple of types to the tuple class getitem to get a tuple type 😅

[rare scarab]

wasn't there a pep before to support kwargs on getitem?

e.g. dict[foo=str]

[viscid spire]

and everything to the left would be packed into a tuple?

[rare scarab]

actually, who says we can't use a slice? dict["x":str]

[viscid spire]

slice won't take non-int args I thought

[rare scarab]

!e ```py
class Foo:
def class_getitem(self, arg):
print(arg)

Foo['x': str]

[rough sluice]

@rare scarab :white_check_mark: Your 3.11 eval job has completed with return code 0.

slice('x', <class 'str'>, None)

[viscid spire]

hrm

[oblique urchin]

it was rejected, and it didn't propose any specific uses for that feature

[rare scarab]

that's fair

[tranquil turtle]

slices accept anything:

!e ```py
class X(slice('hi', 123)): ...
print(X)

[rough sluice]

@tranquil turtle :white_check_mark: Your 3.11 eval job has completed with return code 0.

slice('X', (slice('hi', 123, None),), {'__module__': '__main__', '__qualname__': 'X'})

[viscid spire]

unalivejoy also showed yes

I don't like it

😅

[cosmic cave]

Is a slice a view onto part of a sequence?

can't believe I've never seen them before

ohhh I've only seen them via the [x:y] syntax

[oblique urchin]

no, a slice is just a glorified tuple of (start, end, stop)

the [x:y] syntax creates a slice object internally, which is then passed to e.g. the implementation of list.__getitem__ which returns a new list

[cosmic cave]

oh bummer. so a[x:y] is equivalent to a[slice(x, y)]?

[oblique urchin]

yes

!e

class X:
    def __getitem__(self, o): return o
print(X()[1:2])

[rough sluice]

@oblique urchin :white_check_mark: Your 3.11 eval job has completed with return code 0.

slice(1, 2, None)

[chrome hinge]

(and slicing a list, note, copies that part, it's not any kind of view)

[cosmic cave]

yeah i was hoping for a slick view onto an underlying sequence, like could do:

packed_rgb_colors = list()
red = slice(packed_rgb_colors, 0, len(packed_rgb_colors), 3)
green = slice(packed_rgb_colors, 1, len(packed_rgb_colors), 3)
blue = slice(packed_rgb_colors, 2, len(packed_rgb_colors), 3)

red[10] = 255

[chrome hinge]

Slicing of numpy arrays works like that.

[rare scarab]

you can always make a custom class

[chrome hinge]

I think I saw some memoryview tricks to do it with a list, but unsure.

[cosmic cave]

native stuff's faster for tight loops. extra python fn calls and dictionary lookups add up

but yeah otherwise custom class works

[still bridge]

Getting a weird issue with enum.Enum (https://github.com/t94j0/sddl_py/blob/master/sddl_parser/types.py#L6). I'm trying to make this function actually check with mypy. Rewrote it like this:

def rights_to_type(rights: int, access_map: IntEnum) -> Set[AllRightsT]:
    rs = set()
    for r in access_map:
        if (rights & r) == r:
            rs.add(access_map(r))
    return rs

where AllRightsT is a sum type of all items in rights_enums.py. Mypy is claiming "IntEnum" has no attribute "__iter__" (not iterable) [attr-defined] while enum.IntEnum is a subclass of enum.EnumType which has __iter__ defined. Can't get the __members__ property either even though it's defined.

[rough sluice]

sddl_parser/types.py line 6

def rights_to_type(rights: int, access_map: IntEnum) -> List[int]:```

[oblique urchin]

do you mean type[IntEnum]?

[still bridge]

Oh maybe. I've never seen type[*] before. Where could I learn more about what that's doing?

[oblique urchin]

https://docs.python.org/3.12/library/typing.html#the-type-of-class-objects

[still bridge]

Yep, this worked perfectly. Thanks so much

[still bridge]

Ahh, that makes sense. That's exactly what I was looking for

[dull lance]

Python now has Never as the bottom type... when will we get Unknown?

[tranquil turtle]

pyright thinks that Unknown is the same thing as Any

top type in python is object

[dull lance]

Even in strict mode? Interesting

[heady flicker]

Not exactly the same. I believe it stores them differently.

[tranquil turtle]

mypy stores different types of Any differently

[nocturne dew]

a Expression of type "None" cannot be assigned to parameter of type "(...) -> List[int]" Type "None" cannot be assigned to type "(...) -> List[int] I need to give default value of None to this callable while passing it in a function

[trim tangle]

can you show the code?

[nocturne dew]

def func(
    data: List[int],
    func_: Callable[..., List[int]] =None,
) -> List[int]:
    if func_ is None:
        func_ = lambda x: x 
    ...

[trim tangle]

a) func: Callable[..., List[int]] = lambda x: x
b) func: Callable[..., List[int]] | None = None

also, why do you accept ...? can it be called with any arguments whatsoever?

[nocturne dew]

lambda x: x this can be any custom lambda but it will be fixed, thanks man, i will change ...

[trim tangle]

I meant that you can use the function directly as a default value

without None

[nocturne dew]

exactly, silly me

i kept myself thinking about (...) -> List[int] to None instead of revisiting my code

[dull lance]

I am having problems using AbstractContextManager as a base class. Pylance always shows the warning that the class is missing generic type arguments. However, in contextlib, it isn't actually a generic class - the generics are only available through vscode's type stubs.

When I run pyright through the command line, no error is shown since it doesn't use vscode's type stubs

Is there any way to make the two consistent with each other?

[oblique urchin]

it's actually generic at runtime (in 3.9+)

but see https://mypy.readthedocs.io/en/stable/runtime_troubles.html#using-classes-that-are-generic-in-stubs-but-not-at-runtime for the general case

[dull lance]

I see, thanks!

On another note, can I get vscode to only use the typeshed stubs instead of its own ones?

Because this sort of problem also occurs for external libraries such as numpy where vscode's stubs diverge from the ones on typeshed

I don't want devs to be surprised when they "fix" an error flagged in vscode, only for CI to fail because pyright runs on typeshed's stubs

[trim tangle]

I think Pylance uses typeshed as well. Just might be a different version

[dull lance]

not sure why I'm getting this error on vscode

pyright is fine with it

and when I right-click on vscode to show the source, it can locate the file just fine

[slender timber]

Is it okay to use _typeshed imports in normal code (not typeshed stubs)?

[soft matrix]

Yes

[rare scarab]

Make a conditional type alias. ```py
if TYPE_CHECKING:
MyASM = AbstractContextManager[Foo]
else:
MyASM = AbstractContextManager

class MyContext(MyASM): ...

[rare scarab]

Restart the pylance server.

[wicked scarab]

class Foo:
    def __init__(self):
        self.arg: int | None = None

a: tuple[Foo, int, Foo] = (Foo(), 1, Foo())

if (var := a[0]).arg is None:
    raise TypeError

reveal_type(var.arg)
``` This gives me 
```py
information: Type of "var.arg" is "int | None"
``` It should be only `int`, right?

[soft matrix]

Yeah it should be

[fair forge]

yep, what type chekcer are you using?

[wicked scarab]

pyright

[fair forge]

umm

in mypy works correctly

[wicked scarab]

I think it's a bug from pyright's end? (I'm on the latest 1.1.316)

[fair forge]

could be

I don't use pyright so idk

[slender timber]

Does it have protocols for descriptors? Which ones?

[soft matrix]

Idk I don't think it has any

[lunar dune]

The source code is right here if you want to take a look at what we currently do and don't have 🙂 https://github.com/python/typeshed/tree/main/stdlib/_typeshed

[slender timber]

Ik, but scrolling is an issue on github mobile 😄

eh nvm its only 10kb

I thought it would be much much bigger, since vscode always redirects to somewhere in typeshed stubs

[lethal shard]

Hello, is there a way to make a generic function in Python, such as: def generic_function[T](a: int) -> T

[slender timber]

A repo or package containing just protocols for all the dunder methods would be nice, is there one already anyone knows of?

[echo knot]

Hello! I posted my opinion on the chat but it was going so quickly. Before Python 3.12, from my experience I would say there are two options:

1. By passing a "dummy" type:

T_ = TypeVar("T_")
def generic_function(generic_type: T_, ...) -> T_:

Problem of this approach is that you would need to create an instance of that type. So I dont like it at all

2. Wrapping it into a class

class Generic_(Generic[T_])
    @staticmethod
    def func(args) -> T_:
        ...

So you would call it like

Generic_[int].func(args)

Maybe there are better ideas

Written from the phone, so there might be small typos

[slender timber]

Not yet, but you can do

def generic_function(typ: type[T], a: int) -> T:

[echo knot]

Right, that's much better and simpler 😅

[oblique urchin]

typeshed is much bigger than this. that file is just a library of helpers used across typeshed

[lunar dune]

We're writing something like this but it's currently WIP and we haven't cut a release yet

(And when I see "we" I mean "mostly @hallow flint" 🙂 )

[slender timber]

Is there a repo for this?

[lunar dune]

https://github.com/hauntsaninja/useful_types but as I say, WIP

[lethal shard]

type[T] < - this allows me to pass in a type instead of value ?

sorry if the question is silly, just making sure

[echo knot]

type[T] is the object that you have used to construct instance T. For example:

MyClass() # This is type T
MyClass # Then this would be type[T]

[slender timber]

yep

[viscid spire]

int and bool are type[int] right?

14 and False would fit under int

[slender timber]

bool is an int subclass

[lunar dune]

yes, type[T] is covariant

[viscid spire]

🙏

[lethal shard]

Thank you soooo much. Thank you Thank you Thank you

I was getting desperate and hopeless

[lethal shard]

Sorry for not replying to you all this time. Like you said the messages were moving so fast and for some reason your message was not added to the unread messages counter

[viscid spire]

3.12 won't have function subscripting as of current tho

[lethal shard]

Sorry, what do you mean?

[viscid spire]

I wish they would add it because it would be nice to specialize generic functions like you can with classes

[echo knot]

From PEP695:

Here is an example of a generic function today.

from typing import TypeVar

_T = TypeVar("_T")

def func(a: _T, b: _T) -> _T:
    ...

And the new syntax.

def func[T](a: T, b: T) -> T:
    ...

At least that's what I meant

[viscid spire]

def f[T](a: T) -> T: ...
f[int](1) # error

[echo knot]

It is okay! 🙂

[lethal shard]

The proposed solution made me wonder, because my IDE doesn't highlight the function argument as being used. This is what I have atm:

def get_resources(self, docs_type: type[T], url: str, **kwargs) -> PaginatedResources[T]:

but docs_type is greyed out by IDE (as in nothing is using it). Does Python take it into account somehow? Does it get passed into the PaginatedResources[T]?

[viscid spire]

Well yeah, if you don't use it in your function body it's unused

[lethal shard]

but does it really pass in the generic into the PaginatedResources[T] ?

[viscid spire]

it doesn't pass anything

[lethal shard]

😦

[viscid spire]

that is handled by a 3rd party checker such as mypy

[lethal shard]

ah

[viscid spire]

they are hints to type checkers/readers of your code (including you)

[echo knot]

In the end when you call that function, mypy will just look at your function signature and it will assume the return type based on the arguments you provided. But you are responsible for making that happen within the function body

[lethal shard]

I am such a scrub, I probably don't even have mypy installed

[echo knot]

Are you using and IDE? They usually integrate type checkers

[lethal shard]

Yeah, Pycharm

[echo knot]

I have never used PyCharm. But as far as I know, PyCharm uses its own LSP with type checker integrated. Not 100% sure tho

[lethal shard]

I have just gotten back to Python after having spent 1.5 years in Typescript

I see that Python allows most things that Typescript does, just trying to figure out where and how everything is

[viscid spire]

typescript is different in that it transpiles to JS

[lethal shard]

After checking, I have found already have mypy. But different from TS compiler, it seems very mild. I dont see any messages, nobody is screaming at me... Is this normal?

[echo knot]

Usually the most used one is mypy as it is from python core developers. IDEs have the possibility to integrate this tool to display its feedback on the same line

[viscid spire]

You need to configure some things likely

[lethal shard]

How does it work in Python? I mean Python stays Python, so a tool like mypy basically does the surveillance of code during development?

[echo knot]

Look at the PyCharm plugin store. I have seen that there is Mypy plugin that probably makes the job. Maybe you installed mypy using pip and the PyCharm IDE cannot linkt/find it

[viscid spire]

Mypy analyzes your source code to verify types

[lethal shard]

Does it do so in real time during the development?

[viscid spire]

only if you have a language server setup. Otherwise you can do something like run mypy every time you save your file

[lethal shard]

Yeah, I found it there, already enabled it shows

[lethal shard]

ah that might be the case

I was expecting some real time highlighting

[viscid spire]

that may be possible but idk the answer sorry

[echo knot]

I am not sure if mypy is capable for real time. It is kinda slow as many other type checkers

[viscid spire]

This comes from Intellisense btw

For TS

so that is a language server

[lethal shard]

I didn't know that, thanks for telling me

Tbh I dont really know how it works under the hood in terms of TS. It just did the job automatically so I just rolled with it

Apart from having to setup the .tsconfig file

[viscid spire]

There is the TS compiler, then there is Intellisense. Intellisense runs as a language server for real time autocomplete and type feedback and docs etc

[lethal shard]

I see, thanks. You seem to be well versed in terms of types

[lethal shard]

Does Intellisense use the same TS compiler to accomplish this?

[viscid spire]

Not sure tbh

both made by Microsoft tho

[lethal shard]

It sometimes got really slow, especially when I started working with React. Might be my computer though

Anyway, I found the article on how to configure mypy on Pycharm. Thanks for putting me on the right track @viscid spire @echo knot

[viscid spire]

👍

[echo knot]

Good luck with it!

[viscid spire]

thing with like VSCode is that it is pre-setup for TS

[lethal shard]

One thing about Python hasn't changed. It is really fun to write and to read

[viscid spire]

because both made by Microsoft

[viscid spire]

Yeah it's fun

typing add an extra layer to the fun 😄

in future you will want to look into collections.abc and typing modules for typing needs as well. They provide important functions and classes for more complex typing

[echo knot]

And just in case you need to know. Mypy comes with some features/rules disabled by default, as they might be too annoying sometimes. But if you want to get better at it, it might be interesting to enable them

[viscid spire]

The most strict mypy settings are great to learn about typing, then relax them as you see fit

[lethal shard]

Yeah, I always try to configure the settings as strictly as possible

[viscid spire]

It is important to remember that typing is completely optional

So it is entirely up to you what you do and do not do with it

Ok glhf 👋

[lethal shard]

u2!

[rustic gull]

Has anyone proposed a multiline type ignore yet?

Like unsafe in rust

[viscid spire]

perhaps a decorator could achieve this for functions

although idk why you want multiline type ignoring

typically you don't want to ignore, but rather cast

[acoustic thicket]

theres talks about deprecating the typing.no_type_check decorator in the mailing lists

!d typing.no_type_check

[rough sluice]

typing.no\_type\_check

@typing.no_type_check```
Decorator to indicate that annotations are not type hints.

This works as a class or function [decorator](https://docs.python.org/3/glossary.html#term-decorator). With a class, it applies recursively to all methods and classes defined in that class (but not to methods defined in its superclasses or subclasses). Type checkers will ignore all annotations in a function or class with this decorator.

`@no_type_check` mutates the decorated object in place.

[viscid spire]

it just seems like not a good idea to ignore stuff

[viscid spire]

Reading this doc, definitely looks like it should be deprecated

[hallow flint]

to clarify, no_type_check is not currently going anywhere. there is a separate symbol, no_type_check_decorator, that no one uses and no one supports, which is what we’re thinking of deprecating

[acoustic thicket]

oh

my bad

[trim tangle]

haha fake news

[lethal shard]

Ok, Idk what it is but I am finding mypy to be extremely buggy. It reported 5 problems, I have fixed 1 which was unrelated to the other 4 and suddenly all other problems are no longer picked up, even though I know they are still there (such as incorrect return type)

Oh and now the problems reappeared

after 3 empty manual scans

[echo knot]

If you can send a small snippet maybe I can try to see where is the issue

[lethal shard]

The issue is with the mypy, not the code itself

I have a function that returns a completely wrong type

it gets reported every 3-4 manual scans

then this issue. I have installed the required type, the error does not go away (well it does go away for 3-4 scans and then reappears)

The bad function simplified:

    def get_resources(self, docs_type: type[T], url: str, **kwargs) -> PaginatedResources[T]:
        docs = []

        return docs

[echo knot]

Did you do pip install types-requests? This error happens because not all packages provide the type information. To solve this, some developers decided to create separate type information for some packages

[lethal shard]

Yes, I did. the error stays

[echo knot]

This is happening because "docs" is infered to be type "list" with. However, mypy expects that you return a specific type based on docs_type variable

Like, the return type does not depends on "docs_type". Therefore you will not be meeting the signature types

[viscid spire]

Is PaginatedResources a type or a TypeAlias?

[lethal shard]

class PaginatedResources(msgspec.Struct, Generic[T]):
    """A struct describing PaginatedResources"""
    docs: list[T]
    totalDocs: int
    limit: int
    page: int
    totalPages: int
    pagingCounter: int
    hasPrevPage: bool
    hasNextPage: bool
    prevPage: None | int
    nextPage: None | int

[rare scarab]

a dataclass?

[lethal shard]

Ok, but why does it provide self contradicting reporting? One scan it says it is fine, then during the next scan it reports it as a problem?

[viscid spire]

Believe not as there seems to be non-fields here

they just separated types out of the init likely, or something else is going on

[lethal shard]

msgspec struct *

https://jcristharif.com/msgspec/structs.html

Which btw is also reporting errors

[echo knot]

It should not 😅
Mypy is pretty stable and deterministic

[lethal shard]

once every 4 scans..

[echo knot]

What you need to do inside the function def is creating an object of type PaginatedResources where docs is a list containing types indicated by your input argument docs_type

[lethal shard]

Maybe my mistake is running it on windows..

[viscid spire]

Did you verify you have correctly targeted the version of python that you are using in pycharm?

[lethal shard]

Yes, I am returning the wrong type on purpose to test mypy

[viscid spire]

Because if you aren't targeting the same version you installed types to, it's not gonna work

[echo knot]

Oh sorry 😅

[lethal shard]

Come to thing about it, my machine is a mess. I have several Python versions installed differently under different venvs

Uninstalling Python from Windows is a pain in the butt

[viscid spire]

Might want to do some tidying to get stuff in order

[viscid spire]

The installer also has an uninstaller

[lethal shard]

I know, but it leaves artifacts

[viscid spire]

yeah pip caching I think and stuff

You could try a full reinstall

[lethal shard]

what is the go to way to install and manage python versions nowadays?

[viscid spire]

pylauncher is an option at install

[lethal shard]

I remember there being an easy way to switch out Python versions

🤔

[viscid spire]

and not putting python or pip on path

[lethal shard]

at one point I used Anaconda, not sure if it is still popular/ go to

[viscid spire]

then all commands go through py

py accepts a version argument, and for pip you do py -m pip ...

[echo knot]

If you do not have company based restrictions, I would pick 3.9/3.10 or later (stable releases only)

[viscid spire]

You can also enable pip to only install in venvs

that will add peace-of-mind in where your site packages are installed to and accessible from

[lethal shard]

Using 3.10

[echo knot]

To be honest I have never had this issue. It was always stable. And I have worked with 2 different IDEs, from terminal and in 2 different OS, including Windows. So I do not think I can help you

[lethal shard]

I currently don't have time to address this till Monday evening to ship the first version

But i will try to reinstall everything and run it on Linux some time later

I have seen things either work or not work. But I probably have never seen things work selectively on some occasions

[echo knot]

I think your problem is not the OS, but a messy environment probably

[echo knot]

Neither do I. If you are struggling with deadlines, I would suggest to temporary forget about type hinting. Or use the one that is already integrated in PyCharm

[thick quiver]

Trying to make an abstract parent class that defines that there are two read-only classvars. The subclasses should implement them. In theory like this:

@dataclass
class Parent(ABC):
    min: ClassVar[Final[int]]
    max: ClassVar[Final[int]]

    other_field: str

@dataclass
class Child1(Parent):
    min: ClassVar[Final[int]] = 0
    max: ClassVar[Final[int]] = 10

@dataclass
class Child2(Parent):
    min: ClassVar[Final[int]] = 100
    max: ClassVar[Final[int]] = 200

But apparently ClassVar and Final can't be combined.

Complicating things, this can be re-implemented as a property:

@dataclass
class Child3(Parent):
    some_other_field: ClassVar[Final[int]] = 3
    min: ClassVar[Final[int]] = 0
    @property
    def max(self) -> int:
        return 1 << self.some_other_field

I could just remove the Final[]s, but then mypy wants me to implement a setter property for max as well... but max is supposed to be final

A bit lost on how to solve this, sorry for dumping here

[viscid spire]

you can simply omit the ClassVar

ClassVar doesn't mean that you can't access the attribute on an instance. It means that you can't redefine that attribute on an instance

I suppose the issues come in the ABC part

because typehinting something as Final needs to eventually have a value in the same namespace

[thick quiver]

Right, that's what mypy has an issue with now

$ mypy scratch_1.py
scratch_1.py:10: error: Final name must be initialized with a value  [misc]
scratch_1.py:11: error: Final name must be initialized with a value  [misc]
scratch_1.py:16: error: Cannot override final attribute "min" (previously declared in base class "Parent")  [misc]
scratch_1.py:17: error: Cannot override final attribute "max" (previously declared in base class "Parent")  [misc]
Found 4 errors in 1 file (checked 1 source file)

[viscid spire]

basically ClassVar and Final are conflicting types in specificity

becasue Final stops all redefinition, while ClassVar stops only redefinition on an instance

[viscid spire]

might want to use a property perhaps

wait sorry that's deprecated

[thick quiver]

@dataclass
class Parent(ABC):
    other_field: str

    @property
    @abstractmethod
    def min(self) -> int: ...

    @property
    @abstractmethod
    def max(self) -> int: ...

this works

it's deprecated?

[viscid spire]

abstractclassmethod and abstractproperty are

says to do just like you have there

[thick quiver]

gotcha, thanks

I remember reading something else about some kind of decorator chaining becoming deprecated too

forget the details

[viscid spire]

now in your children, you can have these access a Final[int]

[thick quiver]

Perfect, that works!

[oblique urchin]

@classmethod plus @property I believe

[viscid spire]

thought it was @abstractmethod

oh wait misread my bad

class Parent(ABC):
    @property
    @abstractmethod
    def min(self) -> int: ...


class Child(Parent):
    _min: Final[int] = 0

    @property
    def min(self) -> int:
        return self._min

this should be what your code is gonna approximately look like @thick quiver

[rare scarab]

I have to use a metaclass as a type-friendly workaround, right?

[thick quiver]

ah, right - https://github.com/python/cpython/issues/89519#issuecomment-1093931569

[thick quiver]

This actually works fine: ```python
@dataclass
class Parent(ABC):
other_field: str

@property
@abstractmethod
def min(self) -> int: ...

@property
@abstractmethod
def max(self) -> int: ...

@dataclass
class Child1(Parent):
min: Final[int] = 0
max: Final[int] = 10

[viscid spire]

oh damn really?

cool

well, pyright doesn't like it

[thick quiver]

passes mypy and works at runtime

ahh

[viscid spire]

because it actually changes the type of min

[thick quiver]

What if you throw a @classmethod in there, for fun?

[viscid spire]

3 decorators on one function 🥴

doesn't change anything

there is another issue tho...

from typing import Final
from dataclasses import dataclass


@dataclass
class Child:
    min: Final[int] = 0


x = Child(1)

dataclasses don't respect the Final typehint

so min will be added in the init

[thick quiver]

wow

[viscid spire]

you need this

from dataclasses import dataclass, field
from typing import Final

@dataclass
class Child:
    min: Final[int] = field(init=False)

but idk how to put a value

[thick quiver]

@dataclass
class Child1(Parent):
    min: Final[int] = field(default=0, init=False)
    max: Final[int] = field(default=10, init=False)

[viscid spire]

ah right, default

it wasn't showing up in my intellisense lol

I was looking at the args and simply wasn't there

[echo knot]

You mainly have "default" and "default_factory" for those objects that are mutable

[viscid spire]

yeah ik I have used it before

slipped my tiny mind unfortunately

[thick quiver]

This looks like a bug, dataclasses should respect Final

[viscid spire]

well actually...

if you think about it kinda does

[thick quiver]

https://github.com/python/cpython/issues/89547

[viscid spire]

it's just that specifying = some_value isn't really the super correct way for dataclasses

should be = field(...) if you need to do that

[thick quiver]

Yeah, but IMHO it should work the same as a ClassVar

[viscid spire]

🤷‍♂️

[gray bronze]

would NoReturn be a valid typehint to give to a method that will never return, but can raise exceptions and can be stopped by Ctrl+C

[trim tangle]

f()
print("Hello!")

If the print will never run, then f() returns NoReturn

[gray bronze]

alr thanks

would this be valid for a coroutine?

[lost warren]

Hey folks, might be a dumb question, but is there a way to add a generic to a specific method of a class? I have a worker pool whose role is to pass requests to a worder and I struggle to type it

[trim tangle]

Yes

[lost warren]

official doc only shows how to make the whole class generic, not 1 method

[trim tangle]

Can you show the code?

[trim tangle]

🙂
https://decorator-factory.github.io/typing-tips/tips/no-return-confusion/

[lost warren]

class Foobar:
  def send_stuff(request_name: str) -> T:
    pass_request(request_name)

#and then
foobar.send_stuff[(number, number)]("request_that_would_return_some_tuples")

Here is some peudocode

coming from JS this how I think about it

in the official doc I only see the case where you pass T to the whole Foobar class

and instanciate "foobar" this way

[trim tangle]

That's the same as returning Any from the method

[lost warren]

or examples of functions where the generic is derived from an arg (eg return a string if I call with a string)

[trim tangle]

Using a TypeVar only once generally makes no sense, because there's nowhere to "get" it from.

What the method is really doing is returning Any. You can then assert it to be a specific type later

[lost warren]

any idea how to do so ? In TS I'd cast with "as" or "satisfies", didn't yet figure the equivalent in python

doing val : str = foobar.send_stuff() didn't seem to work

because send_stuff returns "Any" so the str type is ignored

[trim tangle]

Wdym is ignored?

What type checker are you using?

If it doesn't work you can use typing.cast

[lost warren]

nvm it does work !

first time I run Python on this machine so I may have messed up something

thanks for the "cast" type

tip*

what a beauty ret: Tuple[List[Union[(bool, int, bool), Exception]],] = self.dsw_pool.pass_request_to_workers(

technically I should probably wrap that into another function to get the proper type

but each type of request is called roughly in one place only

like specific class -> calls works pool -> calls worker and type in "specific class"

[soft matrix]

Fwiw (bool, int, ...) isn't a valid type

[lost warren]

Right changed for "Tuple"

somehow vscode cringed onthe first tuple but not the nested one

[trim tangle]

If you want to be more type-safe you can use this pattern:

import uuid

class TypedKey(Generic[T]):
   def __init__(self, name: str) -> None:
        self._key = f"{name}:{uuid.uuid4()}"

    def key(self) -> str:
        return self._key

my_key = TypedKey[int]("my_key")

Then you can do e.g.

def register_worker(pool: Pool, worker: Callable[[T], None], key: TypedKey[T]) -> None: ...

def submit_task(pool: Pool, key: TypedKey[T], value: T) -> None: ...

This way you can't get mismatched types

[lost warren]

nice one

thanks!

[slender timber]

any idea to better define an enum with tuple values in a type friendly and enum impl friendly way?

@enum.unique
class ChannelID(EventEnum):
    IsEnabled = (0, BoolEvent)
    _VolByte = (2, U8Event)

[soft matrix]

Why is a tuple an enum?

[lethal shard]

Why will docs_type: type[T] result in: Mypy: Variable "docs_type" is not valid as a type [valid-type] when used?

[rare scarab]

Make it a typealias

docs_type: TypeAlias = Foo

[lethal shard]

I am trying to make it into a generic, so I am not sure if I can assign it to anything?

T = TypeVar("T")


class PaginatedResources(msgspec.Struct, Generic[T]):
    """A struct describing PaginatedResources"""
    docs: list[T]

def get_paginated_resources(self, docs_type: type[T], url: str, path_params: Optional[list[str]] = None,
                          **kwargs: str) -> PaginatedResources[T]:
  
   
   paginated_response = self.session.get(request_url).content
   return   msgspec.json.decode(paginated_response,type=PaginatedResources[docs_type])

[trim tangle]

I would just type-ignore in this case

[lethal shard]

I see. Could you tell me why mypy doesn't like this?

[trim tangle]

I think because it only lets you index generics with "type forms" (like type aliases or classes), not arbitrarily variables

[lethal shard]

Hmm. Well I'd like to pass a class, the problem is I don't know which (different api endpoint - different class)

[trim tangle]

I think what you're doing is fine, just ignore mypy 😛

[lethal shard]

ok, thanks!

[echo knot]

I was curious about the issue you were getting but I do not have a laptop now. I copy pasted your code into "mypy playground" on the phone and after fixing some formatting and removing the msgspec thingy, mypy did not report any issue

Maybe I made a mistake on the phone, as it copy-pasted the code in a weird way

[lethal shard]

It could be because I am running --strict 🤔

[echo knot]

I tried with this code (strict and no strict) and it was all fine. I am using (again, probably I broke something on the phone) https://mypy-play.net/?mypy=latest&python=3.10&flags=strict

from typing import TypeVar, Generic, Optional 

T = TypeVar("T")


class PaginatedResources(Generic[T]):
    """A struct describing PaginatedResources"""
    docs: list[T]

    def get_paginated_resources(self, docs_type: type[T], url: str, path_params: Optional[list[str]] = None,
                          **kwargs: str) -> T:
  
       return docs_type()

[trim tangle]

This is very different

Here you're not using docs_type as a type parameter

[echo knot]

Oh my bad. I removed those lines thinking it was not important. I thought that the issue was in the method signature and not on those lines. I totally ignored them

[grizzled smelt]

Hey all 👋🏻 , a niche question I can't find the answer to online.

If you are publishing a library that is 100% statically typed (py.typed marker included etc). But those types depend on 3rd party stub packages. Do those stub packages become primary dependencies of your library? Or do they stay as dev dependencies and downstream mypy etc can handle the missing stubs?

[slender timber]

They stay as dev dependencies

Except if you use a type hint at a place other than an annotation, like as a base class

[grizzled smelt]

cool thanks! I'm guessing downstream mypy will go, hey you need stubs for this library and ask you to install the same stub packages as your own dev dependencies?

[oblique urchin]

I don't think it will. You probably do need anything that gets used in a public API to be a full dependency

Or maybe you can add an extra like [types]?

[grizzled smelt]

yeah this is what I thought as well 🤔 I guess anything you import directly needs to be a main dependency, weird that I can't find discussion about this online

but I guess if the libraries just provide stubs then no point including

and they are dev dependencies

[slender timber]

if a package contains of only a single file of code in the root directory (like bottle.py for e.g), where do I place py.typed?

does py.typed require a flat or src layout?

[oblique urchin]

you cannot, py.typed only works for packages, not single-file distributions

[oblique urchin]

doesn't matter what you do locally, it only matters that the file is there when the package is installed

[slender timber]

alr

btw should tests be in their own directory even if its a single test_lib.py?

i'd rather prefer it in the root directory if its that simple

[young zealot]

I have a base class that is subclassed by a few different subclasses, I want a function that takes the type of one of those subclasses and will return an instance of that subclass, how can I type hint that?

I think this is working:

T = TypeVar('T', bound=BaseClass)
def func(the_type: Type[T]) -> T: ...

[echo knot]

Yes! It is like that

[echo knot]

@haughty stag

from typing import overload

@overload
def process_input(data: str) -> str:
    pass

@overload
def process_input(data: int) -> int:
    pass

def process_input(data):
    if isinstance(data, str):
        # Realizar operaciones para cadenas
        return "¡Procesando cadena: " + data + "!"
    elif isinstance(data, int):
        # Realizar operaciones para enteros
        return data * 2

As you can see here, the method that is not decorated is the "broadest" one

(example generated with chatgpt. It mixes spanish with english hhaah)

[haughty stag]

AAAAAAAAAAAAAA

[echo knot]

And then type checker will start looking at the signatures from top to bottom

[haughty stag]

:P

[echo knot]

Therefore, it will go like: is data string? No? Is data a int? No? Then take the last method signature

Oh

I pinged the wrong one hahahahahahahahahaha

[haughty stag]

Lol

[fluid jay]

def stat_data_onlevel(self, level: int) -> Dict | str:
    level_iterator = count(start = 20, step = 10)
    level_list: List = list(next(level_iterator) for _ in range(7))
    if level in level_list:
        stat_dict: Dict = self.content["levelData"]
        for data in stat_dict:
            if data["maxLevel"] == level:
                return data
    else:
        return QueryError.leveldata_outofrange()

This function, returns a dict when if condition is met and returns a str when it isn't .
mypy is saying i'm missing a return statement?

data_query\character_data.py:26: error: Missing return statement  [return]
Found 1 error in 1 file (checked 1 source file)   

not sure what I'm missing

[trim tangle]

It probably means that your function can return None in some cases

if stat_dict is empty, for example. Or if none if its items satisfy data["maxLevel"] == level

If you're sure that this will never happen, add an assert False after the loop

[fluid jay]

hmm thats true, this may be of concern as the data gets updated
stat_dict reads from multiple json files. Since these json files contains info of a playable character, stat_dict should typically never be empty, as all characters must have levelData to be playable.

I'm gonna add a few lines to check for that as well

[echo knot]

Haver somebody ever used "PEP696: Type defaults for TypeVarLikes"? (https://peps.python.org/pep-0696/)

I am using it in my project and it is not working with mypy.
I am just copy pasting the examples into mypy playground from the phone. Same example provided is giving a different output (see image). Type is supposed to be Bot. Since this feature was added one year ago, I am searching in mypy repo if it is supported, but I did not find anything

The code in case somebody needs to copy and paste:

class Bot: ...

BotT = TypeVar("BotT", bound=Bot, default=Bot)

class Context(Generic[BotT]):
    bot: BotT

class MyBot(Bot): ...

reveal_type(Context().bot)         # type is Bot  # notice this is not Any which is what it would be currently
reveal_type(Context[MyBot]().bot)  # type is MyBot

[oblique urchin]

it's not implemented by mypy yet

(though some experimental support has been added recently)

[echo knot]

Ugh. I am really interested in this feature. I guess I will have to wait more

Do you have a thread or something I can follow? I cannot find it in mypy repo 🤔

[oblique urchin]

https://github.com/python/mypy/issues?q=is%3Aopen+is%3Aissue+label%3Atopic-pep-696

[echo knot]

Great! Thank you so much. Such a coincidence to see you not only in Pyanalyze repo but also here 😁

[young zealot]

I'm having trouble getting this TypeAlias (in the example A.T) to work properly:

from __future__ import annotations

class A:
    T = tuple[A | B, ...] # This TypeAlias

class B:
    def __init__(self):
        a = A()

https://mypy-play.net/ doesn't complain about anything, though Pylance complains about A and B not being defined in the TypeAlias. If I run it python also raises an exception saying A is not defined. If I do tuple['A' | 'B'], Pylance no longer complains, but when I run it I get an exception about no operator | between str types. What am I doing wrong, or how can I make this work?

[trim tangle]

Does T = tuple["A | B", ...] work?

from __future__ import annotation has no effect because it only applies to annotations

[young zealot]

This seems to work 🤔 I'll give it a try in my full case and see, thank you!

[young zealot]

Yeah makes sense thanks

[blazing nest]

Is there an immutable dict I can type something as? I cannot use Mapping because I use dictionary methods, but if I use dict I cannot passed TypedDict's to it.

[brazen jolt]

what dictionary methods do you use? Mapping should have all of the methods you need other than those which actually mutate the dict

!e ```py
from collections.abc import Mapping

print(set(dir(dict)).difference(set(dir(Mapping))))

[rough sluice]

@brazen jolt :white_check_mark: Your 3.11 eval job has completed with return code 0.

{'__ior__', '__delitem__', '__ror__', '__or__', '__setitem__', 'copy', 'pop', 'popitem', 'setdefault', 'update', 'clear', 'fromkeys'}

[trim tangle]

__or__ 😛

[brazen jolt]

wonder why that isn't on mapping actually

[oblique urchin]

it's newer than the Mapping ABCs, and adding methods to an ABC breaks compatibility

[brazen jolt]

oh, that seems annoying, so any additions to dicts/lists won't carry over to the collections ABCs? Why does it break compatibility though? Does it need to be abstract? Couldn't you just write a default implementation there, so anyone inheriting from Mapping will get __or__ for free?

[oblique urchin]

you don't have to inherit from an ABC to implement it

[brazen jolt]

oh right, hm, that's annoying

[trim tangle]

Guess what __or__ does with defaultdict 🥴

I mean, whatever it is, it's not amazing

!e

from collections import defaultdict
a = defaultdict(int)
b = defaultdict(bool)
print(f"{a | b =}")
print(f"{a | dict() =}")
print(f"{dict() | b =}")

[rough sluice]

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

001 | a | b =defaultdict(<class 'int'>, {})
002 | a | dict() =defaultdict(<class 'int'>, {})
003 | dict() | b =defaultdict(<class 'bool'>, {})

[trim tangle]

yeah, it picks the function from the first dict

[slender timber]

ugh

[brazen jolt]

oh no

[trim tangle]

?

[brazen jolt]

glad I don't really use default dicts

this seems weird lol

[trim tangle]

well, you don't have many options with defaultdict here

[brazen jolt]

dict.setdefault usually does the trick

[brazen jolt]

yeah, I'm not saying I don't understand why it's like this

just that it's kinda weird

and perhaps unexpected

[trim tangle]

yep

I only thought about it literally right now. never occurred to me

chaotic evil

def defaultdict.__or__(self, other):
    if random() > 0.5:
        return defaultdict(self.fn, {**self, **other})
    else:
        return defaultdict(other.fn, {**self, **other})

(not swapping the order of unpacking to be consistent with dict.__or__)

yall should be thankful I'm not participating in the design of the stdlib

And here we come to another pitfall of inheritance. When you subclass, even if right now you don't violate any invariants of your parent class, you don't know if the parent class will in the future add something you won't be able to reconcile with

[brazen jolt]

let's just move to impls, inheritence sucks anyway

[trim tangle]

Hmm.... Does this mean that adding a public method to a class could be a breaking change?

Actually if I had my own defaultdict, adding __or__ to dict would be a breaking change

[slender timber]

isn't Iterable[str] and str basically the same thing?

do type checkers have some special casing for functions accepting just str to also accept an Iterable[str]?

[rare scarab]

Str is iterable, yes

[oblique urchin]

no, because not every Iterable[str] is a str

most functions that take strs don't just iterate over them

[rare scarab]

Special casing for str and bytes

[slender timber]

bytes.__iter__ yields int

so no need for special casing

[trim tangle]

Yeah, no special casing

str satisfies Iterable[str] already

in fact it's really annoying

(because most of the time passing a str where an Iterable[str] is expected is a bug)

but I don't see a good way out

[chrome hinge]

guess we need a char type for python 4 👍

[trim tangle]

https://doc.rust-lang.org/std/primitive.char.html

(blazing fast)

[slender timber]

C char must be faster than Rust char

(blazingly faster, touches the register (sometimes))

[celest iron]

char would be odd in python bc of how you can use unicode in strings

[trim tangle]

well... in C, char means something completely different 😛

[slender timber]

ik

[trim tangle]

In Rust, char means a unicode character

[slender timber]

C char is just disguised u8

[celest iron]

ah

[trim tangle]

a unicode character
well...
The char type represents a single character. More specifically, since ‘character’ isn’t a well-defined concept in Unicode, char is a ‘Unicode scalar value’.

[mint inlet]

@trim tangle fyi pyright playground has an expired ssl cert (you might already know?)

[trim tangle]

yeah I know I am just really lazy 🥴

[mint inlet]

xd

easy enough to workaround nonetheless, figured id ping you just in case xd

[mint inlet]

(not always... portability moment)

[echo knot]

Does somebody here have used numbers.Real commonly? How are those numbers supposed to work from the point of view of hierarchy/type hinting? I have never used them

Like, doing:

number: numbers.Real = 2

Is wrong according to Mypy, although it seems Python built-in integer implements the same methods stated by the abstract class numbers.Real. I thought the difference between numbers.Real and int would be the same as with Mapping and Dict for example. But there seems to be something else

[chrome hinge]

My understanding is that numbers is an old and not very useful module. E.g. Fluent Python ed.2 says about it:

As I review this in July 2021, the numbers package is not supported by PEP 484 or the Mypy type checker. Since 2017 there is an open issue in the Mypy project titled “int is not a Number?”. This is not a Mypy bug; it reflects a shortcoming of the numbers package, which I explain below.

[...]
Sadly, the numeric tower was not designed for static type checking. The root ABC—numbers.Number—has no methods, so if you declare x:
Number then type checkers will not let you do arithmetic or call any methods on x.
[...]
In conclusion, although numeric types should not be hard to type check, the current situation is this: the type hints PEP-484 eschews the numeric tower and implicitly recommends that type checkers hard code the subtype
relationships between built-in complex, float and int. Mypy does that, and also pragmatically accepts that int and float are consistent-
with SupportsComplex, even though they don’t implement __complex__.

[echo knot]

This is great! It solves all my doubts. Thanks! I was already reading fluent python but did not reach that chapter yet 🙂

[echo knot]

Do you know a tool or package that works with typing.Annotated to provide runtime checks? Or any other interesting feature that makes use of them (either runtime or not). I think the potential of this type hint is great. I searched this months ago and I could only find some web development related tools

[tranquil turtle]

Beartype uses that to do custom runtime type checks

[chrome hinge]

Pydantic allows using typing.Annotated to add constraints, validators, etc (instead of putting them in parameters of Field):
https://docs.pydantic.dev/latest/usage/validators/#annotated-validators

Msgspec too: https://jcristharif.com/msgspec/constraints.html

[echo knot]

I will take a look at these! Thanks both 🙂

[slender timber]

what would be the correct type hint for accepting an iterable of chars but not an iterable of strings?

[trim tangle]

If by "char" you mean a 1-character string, that's not possible

[viscid spire]

you can accept an iterable of the union of all possible one-char strings 😂

[rare scarab]

>>> print(f"Character = Literal{list(string.printable)!r}")
Character = Literal['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', '!', '"', '#', '$', '%', '&', "'", '(', ')', '*', '+', ',', '-', '.', '/', ':', ';', '<', '=', '>', '?', '@', '[', '\\', ']', '^', '_', '`', '{', '|', '}', '~', ' ', '\t', '\n', '\r', '\x0b', '\x0c']

I almost typed all that out manually

[trim tangle]

This barely scratches the surface

😔

[rare scarab]

want me to include the entire short range?

here you go. ```sh
python -c 'print(f"from typing import Literal\n\nCharacter = Literal{[chr(x) for x in range(0xffff)]!r}")' > char.py

[tranquil turtle]

Pyright will die on this file

[rare scarab]

Black seems to have parsed it ||eventually||

[tranquil turtle]

Now make typealias for nonegative integers

[rare scarab]

here you go. ```sh
python -c 'print(f"from typing import Literal\n\nWholeNumber = Literal{[*range(1, 0x0fffffff)]!r}")' > whole.py

[tranquil turtle]

Can black parse that?

[rare scarab]

Idk, I didn't run it

[tranquil turtle]

Is it even a valid syntax? Isnt there some parser stack overflow?

[rare scarab]

There's no functions. Why would it stack overflow?

If anything, it would MemoryError

[tranquil turtle]

!e ```py
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
while True:
print('hi')

[rough sluice]

@tranquil turtle :x: Your 3.11 eval job has completed with return code 1.

001 |   File "/home/main.py", line 21
002 |     while True:
003 |     ^^^^^^^^^^^
004 | SyntaxError: too many statically nested blocks

[tranquil turtle]

20 indents - ok, 21 - syntax error

!e this works ```py
x=[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[0 for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()] for _ in ()]
print(x)

[rough sluice]

@tranquil turtle :white_check_mark: Your 3.11 eval job has completed with return code 0.

[]

[tranquil turtle]

f"from typing import Literal\n\nWholeNumber = Literal{[*range(1, 0x0fffffff)]!r}" - this string takes 2.8GB
parser allocated 30GB of memory, froze my pc, and i killed the process

[rare scarab]

Lol

The typing system doesn't support predicates

[hasty jungle]

why not smth different, since it's not possible (see above)

_Hidden = NewType('_Hidden', str)
Char = NewType('Char', _Hidden)

def as_char(x: str) -> Char:
  assert len(x) == 1
  return Char(_Hidden(x))

def as_chars(x: Iterable[str]) -> Iterable[Char]:
  return (as_char(_) for _ in x)


# example
def f(x: list[Char]): ...

f(as_chars(["a", "5"]))

with assertions turned on while testing, I guess you'd see "incorrect" usages enough?

on another note, it feels wrong to accept any kind of chars, disregarding control ones and normalisation. maybe you actually dont want any kind of chars, but a very small subset

[tranquil turtle]

I think you can remove _Hidden there

[hasty jungle]

it's to "disallow" direct subtyping from outside the module, without going through as_char (or just to make it quite explicit it shouldnt be called directly)

[tranquil turtle]

That makes sense

[hasty jungle]

if one day they optimise an idemtity call over regular method ones, one could also do

def or_identity(f):
  def id(x): return x

  b = False
  assert (b := True)
  return f if b else id


@or_identity
def as_char(x: str) -> Char:
  ...```

[tranquil turtle]

  b = False
  assert (b := True)
  return f if b else id
``` this is extremely weird
`return f if __debug__ else id` - this is a lot cleaner and a bit faster (compiler (or peepholer) can optimize `if __debug__` checks)

[hasty jungle]

yeah I didnt remember the dunder 😄

[proud brook]

This is really fun to scroll through on mobile

[slender timber]

I think Literal is badly named

It should have been called OneOf or something

[trim tangle]

why?

OneOf would be Union

[slender timber]

because an argument accepting a literal, doesn't really need a hardcoded literal, a variable can work as well

So its Literal only in terms of its declaration, not real usage

[trim tangle]

OneOf would be more confusing IMO

[slender timber]

Yea I think Union could have been reused too

Or the | syntax

[trim tangle]

Would Union['X'] mean a forward reference to class X or the literal string 'X'?

Or, well, just 'X'

For | syntax you'd have to add __or__ to strings, integers, None, Enum values and so on.

[slender timber]

Yea I completely forgot forward refs lul

[trim tangle]

TypeScript has much lighter-weight syntax, as you proposed

but that's TypeScript 😛

[slender timber]

TypeScript was invented because the base language is shit af, probably unusable

Python's lucky to have type hinting syntax inbuilt

But besides that

Literal wasn't a very good name choice

[trim tangle]

What would you name it?

[slender timber]

Choice?

[trim tangle]

That's still a synonym for Union

[slender timber]

You can cry about union all day

[trim tangle]

😂

[slender timber]

But its not supposed to be used anyways now

I mean a better syntax is usable in most cases 3.7+

[trim tangle]

I think we would be better off without that tbh

Type hints having runtime semantics is a bit of a nightmare

[slender timber]

I don't trust complex code without type hints

For one depending on the variable name entirely is hard

Not everyone is good in naming

[trim tangle]

What I mean is, instead we should've gotten something like TypeScript.

[slender timber]

And code comments telling what code does is so C

[trim tangle]

mypy existed before annotation syntax

[slender timber]

huh

and what did it do?

[trim tangle]

Yeah

[trim tangle]

It used comments!

[slender timber]

# mypy pre type hinting
def main(): pass
```?

[trim tangle]

Annotations are really just comments in a different syntax

[slender timber]

that was based af

[trim tangle]

pre-annotations

Initially annotations were added to account for many different use cases. But then it kinda died and it's more or less only used for static types

!pep 3107

[rough sluice]

**PEP 3107 - Function Annotations**

Link

Status

Final

Python-Version

3.0

Created

02-Dec-2006

Type

Standards Track

[trim tangle]

So now we're in this awkward situation with runtime semantics, forward referencing issues etc.

[trim tangle]

so in hindsight^

Python stays a simpler language. And the type system can evolve independently of it

[slender timber]

Lambdas are neutered anyway.

Still are

Will always be perhaps

And why doesn't vscode use type hints from stub files placed along side code?

Like isn't that the most basic thing

[trim tangle]

good question
i have no idea

[slender timber]

Ffs look at C# tooling (🥵)

[trim tangle]

TypeScript tooling as well 😛

[slender timber]

Both MS

Even pyright ms

MS has a legit type hint fetish

or it wants everything to look more like C#

[trim tangle]

I think Anders Hejlsberg was behind both TS and C#

wait what,

he was the original author of Turbo Pascal and the chief architect of Delphi.

is there a language this person hasn't created

[tranquil turtle]

In most cases you can slap from __future__ import annotations and dont worry about runtime behaviour of annotations

I have this import in literally every my file

[trim tangle]

I wonder what will happen to it since PEP 649

!pep 649

[rough sluice]

**PEP 649 - Deferred Evaluation Of Annotations Using Descriptors**

Link

Status

Accepted

Python-Version

3.13

Created

11-Jan-2021

Type

Standards Track

[tranquil turtle]

1. several years after pep649 stringized annotations will work as before
2. if you remove from future import, pep649 behaviour will work
3. after several years from future import annotations will be deprecated and will have no effect. New behaviour is almost equivalent to old behaviour, so nothing will be broken

I think nothing will be broken (except for some code that relyes on the fact that annotations are strings)

[echo knot]

I dont like how that import looks like. I think because of that I am using the string baded type hints. Which might be even worse 🥹

[tranquil turtle]

I have a blind spot on this import line, so i dont care about how it looks 😄

[echo knot]

This and the if TYPE_CHECKING line to avoid circular import issues

[tranquil turtle]

I never had circular import issues when i use from future import annotations
just do import x; a: x.X instead of from x import X; a: X

[echo knot]

But if you have a.py and b.py, and you import a from b.py, and b from a.py (only for thpe hinting purposes), the from __future__ import annotations still does not fix the circular import issue, no?

Or yes? 🤔

[tranquil turtle]

There is no problem at all

You always can do that

I think

Let me check

[echo knot]

My common sense tells me that unless future annotations are able to somehow detect that this import is only done by type hinting purposes, the circular import issue will still be happening

But I never tried. I directly go with if TYPE_CHECKING + string-based type hint with that specific type

[tranquil turtle]

# a.py:
print(f'a, before import')
import b
print(f'a, after import', b)

# b.py:
print(f'b, before import')
import a
print(f'b, after import', a)
``` and then i do `import a` in this directory

there is one problem: you can import a from b, but it will have no fields (it is not fully initialized)
so you can do import a but you cant do from a import X (because a has no X at this moment)

[echo knot]

I am so confused now

[echo knot]

Oh, I see. I did not note this difference

[tranquil turtle]

it is even more confusing if i do python a.py:

it import a twice 😄

because first a is actually a __main__ module

[echo knot]

I tend to do the "from X import X", which typically triggers circular import issues if you have one. So I thought they would always happen regardless of how you based your imports

[upper flame]

Is there some typing trickery to allow not getting a type error on the assignment line here?

class MyList(list):
    def my(self) -> None:
        pass

a: MyList = [1, 2, 3]

[oblique urchin]

well that code is clearly incorrectly typed, so why do you want it?

the glib answer is # type: ignore but I don't understand why you would want to write that line

maybe you want a NewType instead, or use a = MyList([1, 2, 3])

[upper flame]

Why is a good question 🙂
We basically used PEP 3107 hints for other purpose (not PEP 484) and now I'm trying to reconcile things in a large codebase lol
So the a: MyList part comes from the requirement to be that in runtime
While the = [1, 2, 3] part comes from a desired to have better ergononomic. We do support a = MyList([1, 2, 3]) but it's very verobse to have around.

The longer I'm contemtplating what I can do with all this the more I'm leaning towards just

if typing.TYPE_CHECKING:
    MyList = typing.Any

[dull lance]

So the a: MyList part comes from the requirement to be that in runtime
What do you mean by that? Since according to

a: MyList = [1, 2, 3]

you already assigned something that is clearly not MyList to the a variable at runtime

[hallow flint]

seems like there's a good chance

if TYPE_CHECKING:
    MyList = list

will work better for you than MyList = Any

[brisk hedge]

Have you tried making some kind of type alias like MyList = Annotated[list[...], extra_data]? That doesn't interfere with type safety

[vivid ore]

Is there a way to specify that an abstract static method returns an instance of the non-abstract subclass?

-> Self does not work

and -> AbstractBaseClass does not work since that would be too general a type

[oblique urchin]

what's wrong with -> Self? though maybe you need an abstract classmethod instead

[viscid spire]

I was thinking so too yeah

[oblique urchin]

there isn't really a difference between an abstract classmethod and an abstract staticmethod anyway

[oblique urchin]

yeah try a classmethod

[vivid ore]

ok that works, thanks!

[wicked scarab]

I don't know if this has enough data to debug but:

    def compare(self, target: Mapping[Variable, VAR_VALUE | set[VAR_VALUE]]):
        for var in target:
            reveal_type(target[var])
            if isinstance(target[var], set):
                raise TypeError()
            reveal_type(target[var])
``` I have this code and both of the `reveal_type()`s prints the same:
```py
filepath:107:25 - information: Type of "target[var]" is "int | Fraction | list[Group | Operator | ParenthesizedGroup | Fraction] | set[int | Fraction | list[Group | Operator | ParenthesizedGroup | Fraction]]"
filepath:111:25 - information: Type of "target[var]" is "int | Fraction | list[Group | Operator | ParenthesizedGroup | Fraction] | set[int | Fraction | list[Group | Operator | ParenthesizedGroup | Fraction]]"
``` Why doesn't pyright narrow it (remove the set)?

[oblique urchin]

I don't think it narrows x[y] where y is a variable in general

[trim tangle]

rather, where x is a variable

[wicked scarab]

oh

[oblique urchin]

no?

[trim tangle]

if that's what you meant

hm?

[oblique urchin]

x[<literal>] would get narrowed

[trim tangle]

ah

[oblique urchin]

e.g. if isinstance(target["x"], set): reveal_type(target["x"]) would output set

[trim tangle]

I would extract target[var] into a variable

like foo = target[var]

or just accept Mapping[Variable, VAR_VALUE] 😛

[oblique urchin]

actually, in this case, do for var, foo in target.items()

[trim tangle]

(make it the caller's headache)

[wicked scarab]

hm, why does it work tho?

[wicked scarab]

it broke like 99% of my other code ._.

[trim tangle]

that's terrifying 👍

[upper flame]

Almost, I also want it to know about .my()

[rare scarab]

Should I bother with using GenericAlias manually

 SelfListType = cast(type[list[Self]], types.GenericAlias(list, cls))

Or should I just do it normally via type: ignore? ```py
SelfListType = list[cls] # type: ignore[mypy]

[trim tangle]

technically it is not sound to narrow here because your Mapping can do arbitrary things when you look up the key

[upper flame]

Yes, I was thinking about it, but it would require some changes to the framework which I'm not sure are streighforward.

[rare scarab]

I'm looking at it, and I don't like what I did.

[oblique urchin]

that's not the reason though, since type checkers are happy to narrow for literal keys where the same thing applies

it's just the cross-variable dependency

[trim tangle]

oh

[vivid ore]

If you have one class that’s generic over something bound by another class, can you express that the bounding class is itself a subclass of the bounded class with whatever subclass of the bounding class it is as the generic parameter?
I ask because I want to represent linear algebra, where every linear space is related to some field, but every field is a linear space with itself as the field.

I would like to write

Scalar = TypeVar("Scalar", bound="Field")

class Space(ABC, Generic[Scalar]):
    ...

class Field(Space[Self])
    ...

but mypy complains

[trim tangle]

Maybe you want something like a TrivialSpace(Space[Scalar])?

I'm not sure such trickery with Self works

[echo knot]

I think from the point of view of a real case it should not be possible. A superclass that depends on some of its subclass it is a problem. Because what would happen if you create another subclass from Space? This one would be linked to Field even if they should be completely independent. Maybe I am just wrong. But I have never seen something like that

Because in the end subclass should be aware about what its superclass implements. But not the other way around: a superclass that depends on its subclass

[trim tangle]

a subclass MySuperField would be a Space[MySuperField]

which, yes, would break LSP because Scalar is invariant

(🤓)

[echo knot]

Ah, I understand the case better now

But why would you care about which types are assumed for Space? If it is abstract and it is never going to be instantiated, you only need to care about which types are inferred for the subclasses like Field, no? This is confusing me even more 😂

[trim tangle]

Suppose you have something like this ```py
class Eq(ABC, Generic[T]):
@abstractmethod
def equals(self, other: T) -> bool:
...

If you have a type of `Eq[int]`, you can compare it to an int.
But if you have:
```py
class Foo(Eq[Self]):
    ...

if you have an instance of Foo, you can compare it to itself.

[echo knot]

So, if the TypeVar is always going to take the value T = CurrentClass, why you just dont do:

class Eq(ABC):
    @abstractmethod
    def equals(self, other: Self) -> bool:
        ...

In the end Self is an alias for a TypeVar that always takes the value of the current instance type

This way any class you create from Eq, would have other being same type as its superclass

[trim tangle]

Because this is not the only way to use Eq

You can also be Eq[SomethingDifferent]

In OP's case you could have class IntVec3(Space[int])

[echo knot]

I see 🤔

[buoyant ridge]

I've cloned this pypi library but it doesn't have any static typing and not using it basically gives me whiplash at this point. I'm not sure whether to add a directory for type stubs or just rewrite them inline however - the library seems to be 2 years dead, otherwise I'd go for type stubs

But if I'm likely going to be moving it into a fork I suppose it makes more sense to be writing it inline

[vivid ore]

Actually that's a module and not a space because integers are a ring and not a field

[trim tangle]

wdym

oh right

yeah I'm bad with maths, sorry

int doesn't have an inverse

[vivid ore]

I think I will do this (but with a class factory function)

[brisk hedge]

I also think algebraic structures fit really well with Protocol types

[vivid ore]

it would be cool if you could somehow tell the type system that two classes will never overlap

that way, an isinstance check for one could rule out the other

[trim tangle]

That's possible if they have incompatible metaclasses

IIRC

[vivid ore]

oh cool

gah

I know why you can’t but it bothers me

I wish you could do issubclass with a TypeVar

[autumn fern]

Is there a pattern for dataclass typing to declare an abstract class attribute for a base class, and for the linter to not complain when children implement it? At runtime this works, but pylance complains:

class Base(ABC):

    @property
    @abstractmethod
    def arg(self) -> Foo:
        ...

@dataclass
class Child(Base):
    # pylance/pyright reports this as incompatibleMethodOverride
    arg: ClassVar[Foo] = Foo.ENUMERATION_1

[slender timber]

You are overriding a method as an attribute? Thats incorrect

[autumn fern]

Yes I know it's weird, but literally every SO thread I can find has "this is the pattern for declaring an abstract attribute"

[tranquil turtle]

It is not weird. Typecheckers should understand that attr satisfy requirements of @property, but they are not smart enough

[oblique urchin]

pyright intentionally complains about this, the reasoning is that a property has more specific semantics than an attribute

[tranquil turtle]

If protocol has property (read only) and class that implements this protocol has attr (read and write), how that violates lsp?

[oblique urchin]

if you do Base.arg you get a property object, if you do Child.arg you don't

(note that I disagree with this argument, I think it's too puristic, but it does make some sense)

discussed in https://github.com/microsoft/pyright/issues/2943 (though Eric makes an exception for Protocols)

[tranquil turtle]

If Child has slots then you get slot descriptor object, if it has no slot for this arg you get AttrError

In any case (slot, no slot or property) mypy will think that Child.attr is an int object (if you annotated it as int), not property or slot descriptor

https://github.com/python/mypy/issues/13608

https://github.com/python/mypy/issues/13103

Oh, it is fixed actually

[plain dock]

hey @trim tangle: is there an ETA on when we should expect significant additions to Typing Tips? ParamSpec, maybe? 🥺

[trim tangle]

Is the PR button not working? 🙂 You should contact GitHub

||/lh||

I guess there's the typing docs, the typing tips are kinda redundant

[plain dock]

fix, i'm not asking so i can add additions, i'm asking so i can finally use paramspec without a zillion errors and reverting to not using it

[trim tangle]

Yeah I understand 🙂

[trim tangle]

https://typing.readthedocs.io/en/latest/

There's something in here for example:
https://typing.readthedocs.io/en/latest/source/generics.html?highlight=ParamSpec#declaring-decorators

[trim tangle]

I personally never used ParamSpec and I have no idea about all the caveats. Every time I tried using it I kinda failed as well.

I am currently in the middle of a scorching burnout so don't expect any kind of magnum opus any time soon

If you're in immediate danger of type unsafety, my advice would be: replace ParamSpec with TypeVarTuple

The biggest problem with ParamSpec for me is the awkwardness of adding/removing parameters, especially on methods. It kinda looks like it was designed just for the cases of:

1. Passing a function, args, and kwargs (likeThread(...))
2. Making a decorator with no/minimal transformation of arguments

I think 1. is silly anyways, just pass a lambda/partial

Though again, I haven't used it. So maybe I'm missing something

[mint inlet]

ive confronted paramspec not working by sending prs into mypy 😭

my use case being a function that you use to design your own function signature

... obviously this use case has been supplanted by typevartuple

[echo knot]

Just curious . Do any of you type hint numpy-array shape? If do, how do you do it? The only possible way I found is by using external wrappers (nptyping) that provide a way of indicating shape for each array

[rare scarab]

The generic in this function appears to break in pylance...

def find_annotated(field: FieldInfo, annotation_typ: type[T]) -> T | None:
  ...

# Argument of type "type[InlineValue]" cannot be assigned to parameter "annotation_typ" of type "type[T@find_annotated]" in function "find_annotated"
#  "type[InlineValue]" is incompatible with "type[bool]"
#  Type "type[InlineValue]" cannot be assigned to type "type[bool]"
if not (foo() or find_annotated(f, InlineValue)): ...

works fine when I inverse the if. ```py
if not foo() and not find_annotated(f, InlineValue):

[trim tangle]

bug report time

[rare scarab]

it's possibly because foo() returns a bool

[rare scarab]

Ok, so the workaround is to be specific and check is not None

[rare scarab]

In pydantic v2, is there any way to add a field validator in __pydantic_init_subclass__?

[native marsh]

Is there a way of annotating a generic function to being not None? I have a FastAPI application where I found myself writing

result = db.query(...) # some DB query
if result is None:
  raise HTTPException(404)

quite frequently. I wanted to write an expect() function to try and shorten this construct. I came up with the following

def expect(
    value: T | None,
    error_msg: str | None = None,
    status_code: int = 404,
) -> T:
    """
    Unwrap a value or raise an HTTPException if the value is None.
    """
    if value is None:
        raise HTTPException(
            status_code=status_code,
            detail=error_msg,
        )
    return value

but this isn't really perfect, is it possible to have a TypeVar that is not None, or is there something I've missed that makes this possible?

[rare scarab]

Try using a TypeGuard

it's a function that returns a bool. If it's true the input is the typeguard's arg

e.g. ```py
def is_not_none(value: T | None) -> TypeGuard[T]:
return value is not None

if is_not_none(xxx):
xxx.yyy

though typeguard doesn't work with exceptions

[native marsh]

The issue with this is that MyPy doesn't recognize the type since the Generic could be None itself right?

[oblique urchin]

I'm working on this in https://github.com/hauntsaninja/useful_types/pull/7 but every variation I tried so far fails on some cases for either pyright or mypy

[native marsh]

it was my understanding that you can't really have T | None since T could be None itself

[oblique urchin]

I don't think TypeGuard really helps here, it doesn't add anything over just checking is not None

[native marsh]

This looks really interesting, thanks. I haven't actually tried my example with Pyright, but I know MyPy complains about it

[rare scarab]

Custom Annotated like this probably won't be possible, right? ```py
Ts = TypeVarTuple("Ts")
Inline = Annotated[T, InlineValue(), Unpack[Ts]]

Just an annotated that acts as its own Annotated so you could do Inline[str, SomethingElse()]

[native marsh]

Out of curiosity, do you know if this is a bug in one of the type checkers, or is this behaviour not really specified? I was just looking through the typing documentation/ PEPs and I couldn't really see any mention of overloads with generics

[oblique urchin]

the latter, overload behavior isn't well specified

[slender timber]

Is TypedDict special cased?

It looks like a normal class but instead of __getattr__ all the typing enhancements get applied to __getitem__. If I subclass TypedDict and also use the dataclass decorator, would that mean I get easydict like behavior for free?

[spice locust]

is it possible to create a class that acts like a Union[str, bool, int, float] type hint, but allows me to add custom methods to the class? or would i need to subclass Union to do this?

alternatively, can i create a class inheriting from Generic[T] where T is bound to str, bool, int, float, and then have literal values be compatible with the class type hint?

e.g.

T = TypeVar("T", str, bool, int, float)

class Foo(Generic[T]):
  pass


def bar() -> Foo[bool]:
  return True  # type checker failure

[acoustic thicket]

wouldnt that be unsafe? if i had an x: MyType and called x.my_method(), it would fail if x was a bool literal

[tranquil turtle]

import typing as t
import dataclasses
V = t.TypeVar('V')

class easydict(dict[str, V]):
    def __new__(cls, *_, **__) -> t.Self:
        self = super().__new__(cls, *_, **__)
        self.__dict__ = self
        return self

@dataclasses.dataclass
class X(easydict[int | str]):
    a: int
    b: str

x = X(42, 'foo')
reveal_type(x.a) # int
reveal_type(x.b) # str
reveal_type(x['a']) # int | str
reveal_type(x['b']) # int | str

``` is this good enough for you?
i dont think you can subclass TypeDict, because there are no TypeDict instances actually at runtime

[spice locust]

i see the flaw in my first idea, but what about the second one?

[acoustic thicket]

doesnt it have the same problem

[tranquil turtle]

second is also unsafe
because True is not an instance of foo

[acoustic thicket]

if literals were to be compatible with this typehint then it couldnt have custom methods

[tranquil turtle]

you cannot do Foo.bar on bool values

[slender timber]

This reminds me of C++ template fuckery where you can "enable" a method if T satisfies some condition

[spice locust]

makes sense. in practice, the value passed from the users function goes straight into json serialization. i need the type hint for metadata

[slender timber]

Just about ok

[spice locust]

basically what im trying to achieve is this:
the user writes a function, for example

@my_deco
def foo() -> RawResponse:
  return True

then, my_deco knows that the user is intending to return a RawResponse and can record some metadata like some_object.response_type = "it returns a raw response"

alternatively, if the user does not intend to return a raw response, it would go something like this

@my_deco
def foo() -> AnotherResponse:
  return AnotherResponse(
    some_field="xyz",
    some_other_field=42,
  )

then my decorator would record as metadata that this function returns an AnotherResponse, and can then pass that metadata to an api

[acoustic thicket]

hm and whats the type hint needed for

[spice locust]

telling the logic behind the decorator what the function intends to return, so it can map that return type to an id that the api understands

and also for type checking in general, so the user doesnt accidently return an object that isnt recognized by the api, like a list

[trim tangle]

!d typing.Annotated

[rough sluice]

typing.Annotated

typing.Annotated```
Special typing form to add context-specific metadata to an annotation.

Add metadata `x` to a given type `T` by using the annotation `Annotated[T, x]`. Metadata added using `Annotated` can be used by static analysis tools or at runtime. At runtime, the metadata is stored in a `__metadata__` attribute.

If a library or tool encounters an annotation `Annotated[T, x]` and has no special logic for the metadata, it should ignore the metadata and simply treat the annotation as `T`. As such, `Annotated` can be useful for code that wants to use annotations for purposes outside Python’s static typing system.

[spice locust]

it was added in 3.9, im working with 3.8 🥲

[tranquil turtle]

pass AnotherResponse to decorator (or make another decorator which will set metadata on function) and annotate function as -> bool

@my_deco(rettype=AnotherResponse)
def foo() -> bool:
  return True

[spice locust]

i think for now i'll go with making RawResponse an alias for Union[str, bool, int, float]

is there a way to make a parameterized union, so that the user could choose which of the Union's parameters they intend to return?

like

MyType = Union[str, bool]

def foo() -> MyType[bool]:
  ...

i think this is something typevars could be used for, but not sure how exactly

T = TypeVar("T", str, bool)

def foo() -> T[bool]:  # error
  ...

[spice locust]

whats going on here?

all the id properties return str

[tranquil turtle]

i guess you are overriding this property with incompatible type

[lost spoke]

can you help me with this?

i = 1 

while i < x: 
    print('candy')
    i += 1 

it gives me this :

[tranquil turtle]

wrong channel: #❓｜how-to-get-help

[lost spoke]

no one answered

[trim tangle]

!pypi typing-extensions

[rough sluice]

typing-extensions v4.7.1

Backported and Experimental Type Hints for Python 3.7+

[fluid jay]

def db_connect(choice: str) -> None:
    if choice == "lc":
        return sql.connect(key["LIGHTCONE"])
    elif choice == "fsearch":
        return sql.connect(key["FSEARCH_DB_LOCATION"])
    else:
        return None

how do I typehint the return of the sql.connect()?

[acoustic thicket]

what does it return

[fluid jay]

it returns a <sqlite3.Connection object at 0x000001A24D18DD50>

[rare scarab]

sql.connect is a class?

type it as sql.connection

[fluid jay]

yes
its from sqlite3, im using that

[acoustic thicket]

-> sql.connection | None

[rare scarab]

I assume you did something like import sqlite3 as sql?

[fluid jay]

ah yes

[fluid jay]

also, the way im checking my type hints is through mypy, I did this and it complains that sql.connection is not defined

i mean i get it, but im not sure what to do

[acoustic thicket]

sqlite3.Connection

or if you have that from import then sql.Connection

[fluid jay]

oooh

ok so for the sake of future type hint adventures, the way to know what typehint a return is just go print(type(func)) or var or whatever and just use that

[rare scarab]

sqlite3 is also in typeshed, so you can probably also use reveal_type(sql.connect) to get the return type of a function

also also also

[fluid jay]

oh there's a package for external typehints?

[rare scarab]

it's included with pyright and mypy, but you shouldn't import them. It takes care of it for you.

[fluid jay]

aah ok thats good to know
i already have mypy on

[rare scarab]

stdlib stubs are here

https://github.com/python/typeshed/tree/main/stdlib

https://github.com/python/typeshed/blob/main/stdlib/sqlite3/dbapi2.pyi#L205-L214

[rough sluice]

stdlib/sqlite3/dbapi2.pyi lines 205 to 214

def connect(
    database: StrOrBytesPath,
    timeout: float = ...,
    detect_types: int = ...,
    isolation_level: str | None = ...,
    check_same_thread: bool = ...,
    factory: type[Connection] | None = ...,
    cached_statements: int = ...,
    uri: bool = ...,
) -> Connection: ...```

[fluid jay]

def fsearch(entry, choices) -> str | None:
    cutoff: float = 0.55
    candidates = {}

    for i in choices:
        score: float = ratio(entry, i)
        if score > cutoff:
            candidates[i] = score

    try:
        return max(candidates, key=candidates.get)
    except ValueError:
        return None

mypy is complaining:

error: Argument "key" to "max" has incompatible type overloaded function; expected "Callable[[Any], SupportsDunderLT[Any] | SupportsDunderGT[Any]]" 

so do i typehint the kwargs as well?

[oblique urchin]

I think the problem is candidates.get may return None

(it can't in this case, but mypy doesn't know that)

[fluid jay]

now that you've mentioned it

return max(candidates, key=candidates.get(candidates.keys()))

its weird but now it shows the None complaint

 error: Argument "key" to "max" has incompatible type "float | None"; expected "None" 

[oblique urchin]

that code doesn't really make sense, candidates.keys() can't be a good argument to candidates.get and you probably need a lambda

[fluid jay]

yeah, i was just wondering
i'll try the lambda

[rare scarab]

Wouldn't candidates.__getitem__ also be an option?

or even max(candidates.items(), key=itemgetter(1))[0]

[fluid jay]

yep lambda works and mypy stops complaining

return max(candidates, key=lambda x: x)

[fluid jay]

ill try these

[rare scarab]

that's an identity function, which is the default

[fluid jay]

oh wow yeah it works

TIL

[rare scarab]

You probably wanted lambda x: candidates[x]

[fluid jay]

that works and the x: x also works
doesn't both return the values all the same?

[rare scarab]

are you wanting to get the max key lexographically?

[fluid jay]

sorry what does lexographically mean

the max here is comparing the value of the keys, which is a levenshtein ratio for fuzzy logic (float)

and what i want to be returned is the keys

so the x: x make sense to me since it'll get the key

but x: candidate[x] also works
doesn't that mean it'll return the value (float) instead? since its indexing candidate["something"]

[oblique urchin]

only one of these should work as you want. if you observe that they both work, you may need more test cases

[rare scarab]

It means by characters

[oblique urchin]

also with the code you showed above, lambda x: candidates[x] should fail for some inputs (where the ratio is below the cutoff)

[echo knot]

What's your opinion from other type checkers? Considering speed, how quick they implement new features, bugs... I typically use Mypy. I used pyright in the past but it had some bugs. What about pytype for example?

[fluid jay]

that portion of the test fails as intended, but for the case of x: x and x: candidates[x] both return the keys, is that the intended result of both lambdas, or should one of them return the value instead of the key?

[fluid jay]

aah got it

[fluid jay]

found out the problem, well it's not a problem with the code. i jsut forgot how lambdas work

so basically since the cutoff is quite high relatively to the choices, it discards most results and in the end sometimes there's only one key:value pair in the dictionary, when this is the case, max() will of course return only one thing regardless of the key=

and the correct argument is indeed either __getitem__ or x: candidates[x].

evaded a potential disaster there

[carmine phoenix]

remind me from what Python version does from __future__ import annotations work?

[soft matrix]

3.7

[carmine phoenix]

thanks bro

[slender timber]

mypy used to annoy me like hell once upon a time i.e. pre 1.0, but its pyright that does so now. mypy has really improved

however pyright lsp is kinda instant as opposed to mypyd (or whatever vscode uses)

[trim tangle]

VSCode uses Pylance which is based on Pyright

mypyd doesn't provide general LSP features like go to reference

[rare scarab]

Because mypy is a type checker. Pylance uses pyright for type support

[slender timber]

why no fix error btw?

[trim tangle]

am smol dukci

[trim tangle]

pyright also provides an LSP with some stuff

and e.g. Rename

[celest iron]

microsoft built an separate mypy ext recently

[rare scarab]

Same with black and flake8

[blazing cobalt]

I've found pyright to be a bit ahead of mypy. E.g.: https://dogweather.dev/2022/10/03/i-discovered-that-python-now-can-do-true-match-exhaustiveness-checking/

[eager vessel]

I think mypy also supports that 🤔

[blazing cobalt]

I could only find a lot of articles showing faked-up exhaustion checking with mypy. Maybe it's been added since I wrote that blog

[eager vessel]

def func(a: int | str | datetime) -> None:
    match a:
        case int():
            return
        case str():
            return
    reveal_type(a)

$ mypy t.py 
t.py:11: note: Revealed type is "datetime.datetime"
Success: no issues found in 1 source file

[blazing cobalt]

If you remove the reveal_type, do you get an error? That's the feature, IMO

[eager vessel]

Ah, probably not? As it shouldn't be an error?