#type-hinting

1 messages Β· Page 20 of 1

rare scarab
#

Though you should probably explicitly type your callable

#

!zen explicit

rough sluiceBOT
#
The Zen of Python (line 1):

Explicit is better than implicit.

echo knot
#

Any just tells the type checker to not perform any checks on that variable, so it makes sense to me

twin tendon
rare scarab
#

Why would you need ai to do this?

rough sluiceBOT
#

mypy_gpt/mypygpt.py line 124

def try_to_solve_issues(self,errors):```
tranquil turtle
twin tendon
#

Yes, I am not a strong believer in enforcing it. It is nice to have in a small project. And I gradually add more coverage while testing it.

twin tendon
rare scarab
twin tendon
#

Really? I thought you said it is not needed. Or just that you don't need AI for it?

#

I know that mypy doesn't offer fixes generally

fair forge
#

But if your ide use inference to get the type of a variable you have to type as well

fair forge
echo knot
#

You encounter this typically many times. I cannot imagine an autofix that would run on my code base fixing these issues under the interpretation of chatgpt

twin tendon
#

I had a case where it repeadily changed it back and fourth between int and float. Did some unbased casting.

rare scarab
#

Ai is dumb

high current
#

The below code works, but I'm not too confident about the type hints. Could someone look over this please?

_T = TypeVar('_T')
_Transformer = TypeVar('_Transformer', bound=discord.app_commands.Transformer)

def simple_transformer(to: type[_T]) -> Callable[[type[_Transformer]], _Transformer]:
    def decorator(cls: type[_Transformer]) -> _Transformer:
        return discord.app_commands.Transform.__class_getitem__((to, cls))

    return decorator
#

And here's the docstring which I redacted for readability:

A decorator for discord.py transformers which makes them easier to use in type annotations.

Before::

class BooleanTransformer(app_commands.Transformer):
    def transform(self, interaction: discord.Interaction, value: str) -> bool:
        return True if value.strip().lower() in ('true', 'yes', 'y') else False

@app_commands.command()
async def say_hello(
    interaction: discord.Interaction,
    all_caps: app_commands.Transform[bool, BooleanTransforer]
):
    await interaction.response.send_message('HELLO' if all_caps else 'hello')

transformer = BooleanTransformer()
manual_transform = transformer.transform(some_interaction, 'yes')

After::

@breadcord.helpers.simple_transformer(bool)
class BooleanTransformer(app_commands.Transformer):
    def transform(self, interaction: discord.Interaction, value: str) -> bool:
        return True if value.strip().lower() in ('true', 'yes', 'y') else False

@app_commands.command()
async def say_hello(interaction: discord.Interaction, all_caps: BooleanTransformer):
    await interaction.response.send_message('HELLO' if all_caps else 'hello')

manual_transform = BooleanTransformer.transform(some_interaction, 'yes')

:param to: The type which the transformer should output.

echo knot
high current
echo knot
high current
#

Humans have context to the code and what it's supposed to do, which is why I'm asking here

#

because PyCharm is whining about it, and I don't know how to do this best

fathom river
high current
#

I know that typing.Annotated is involved somehow as well

fathom river
#

What are you trying to do exactly?

high current
#

I need the return value of the decorator to be usable in place of the Annotated[_T, _Transformer] type

#

So let's say _T is int

#

And let's call the return value x

#

Then x should be a Transformer, and if I use foo: x, the type of foo should be int

fathom river
#

But you don't want to assign the explicit type to the argument, correct? You want it to be kind of inferred by simply assigning the transformer?

round bolt
#

Would type hinting length of a sequence like this: Annotated[List[int], 300] be recommended, or would List[int] be better.

*MyPy seems to have no problem with the first one.

eager vessel
round bolt
eager vessel
brisk hedge
#

Nothing supports "list of specific length"

rare scarab
#

types-sqlalchemy-utils seems incomplete. ```py

def getattr(name: str) -> Any: ... # incomplete

#

that's all there is in every file

#

with misc imports

#

I don't think pyright likes imports with this

regal summit
# 
@overload
async def get_images_from_urls(image_urls: Iterable[str], base_filename: str, file_ext: str, discord_wrap: Literal[True]) -> list[discord.File]:
    ...
@overload
async def get_images_from_urls(image_urls: Iterable[str], base_filename: str, file_ext: str, discord_wrap: Literal[False]) -> list[bytes]:
    ...
async def get_images_from_urls(
    image_urls: Iterable[str],
    base_filename: str = 'image',
    file_ext: str = 'png',
    discord_wrap: bool = True,
) -> list[discord.File] | list[bytes]:
    return await asyncio.gather(*(
        get_image_from_url(url, base_filename=base_filename + str(i), file_ext=file_ext, discord_wrap=discord_wrap)
        for i, url in enumerate(image_urls)
    ))


@overload
async def get_image_from_url(url: str, base_filename: str, file_ext: str, discord_wrap: Literal[True]) -> discord.File:
    ...
@overload
async def get_image_from_url(url: str, base_filename: str, file_ext: str, discord_wrap: Literal[False]) -> bytes:
    ...
async def get_image_from_url(
    url: str,
    base_filename: str = 'image',
    file_ext: str = 'png',
    discord_wrap: bool = True,
) -> discord.File | bytes:```
image.png image.png
#

for some reason it thinks discord_wrap for get_image_from_url can only be False?

#

I thought I pretty much understood overloads but I have no clue why its doing that here

brazen jolt
#

this happens because discord_wrap in the image there is a bool, but the overloads only allow literal true/false values

trim tangle
#

Yeah

brazen jolt
#

that means you'd need to do discord_wrap=True or discord_wrap=False, i.e. specify it literally

trim tangle
#

I think the argument must have the type to be exactly Literal[True] or Literal[False]

brazen jolt
#

bool is not the same as Literal[True] | Literal[False]

regal summit
#

oh I hate that

# 
@overload
async def get_image_from_url(url: str, base_filename: str, file_ext: str, discord_wrap: Literal[True]) -> discord.File:
    ...
@overload
async def get_image_from_url(url: str, base_filename: str, file_ext: str, discord_wrap: Literal[False]) -> bytes:
    ...
@overload
async def get_image_from_url(url: str, base_filename: str, file_ext: str, discord_wrap: bool) -> discord.File | bytes:
    ...
async def get_image_from_url(``` seems to do the trick, is this the correct way to fix this
trim tangle
#

Can you show the code for get_image_from_url?

#

I think it's generally a "smell" when you have a boolean flag like this. I would return discord.File and let the user sort it out

brazen jolt
#

btw, I'd suggest just converting to discord file separately, if it's a simple thing, just handle the logic directly after calling, if not, make a function taking the bytes, returning the discord.File

#

yeah, exactly what fix error said too

regal summit
#

I mean I shouldnt really be touching this, its for a client, just wanted to type hint this so I dont see ugly red lines :D

brazen jolt
#

having an argument like this is just not a great practice in general, what if you'd also eventually need a twitter_file conversion, and a reddit_file or whatever else

#

you'd need to make overloads for everything, and it'd just be a huge mess

regal summit
#

yeah well not really my concern, this is for a discord bot and never gonna change

#

anyway, thanks for help all!

tranquil turtle
rare scarab
#

Why did I write yet another simple str type function? ```py
import types
from typing import Any, Union, get_args, get_origin

def simple_str_type(t: Any) -> str:
match t:
# catches x | y
case _ if get_origin(t) in {Union, types.UnionType}:
return " | ".join(simple_str_type(u) for u in get_args(t))
# catches x[y, z]
case _ if o := get_origin(t):
return (
simple_str_type(o)
+ "["
+ ", ".join(simple_str_type(a) for a in get_args(t))
+ "]"
)
case types.NoneType | types.NoneType():
return "None"
# normal type
case type():
return t.name
case _:
raise TypeError(f"'{type(t).name}' is not a type")

# 
>>> simple_str_type(list[dict[str, int|str]] |None)
'list[dict[str, int | str]] | None'
#

in unrelated news, I learned you can do conditional cases

soft matrix
#

whats wrong with just normal str?

oblique urchin
#

<type 'str'>

soft matrix
#

oh yeah

trim tangle
#

why the hell are Any and Union highlighted as keywords

soft matrix
#

you know how the docs say

Called by the repr() built-in function to compute the β€œofficial” string representation of an object. If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment)
why doesnt type follow that?

tranquil turtle
# 
>>> object()
<object object at 0x000002E5137750A0>
# 
>>> lambda: ...
<function <lambda> at 0x000002E515A36840>
#

there are a lot of exceptions to this rule in stdlib

soft matrix
#

yeah but those at least i can explain away

#

object.repr cant realistically be reconstructed and lambdas are anonymous and whilst you could reconstruct their inner code that sounds awful for perf

#

i mean both are awful for perf

trim tangle
#

I mean, object's potential repr is object()

brazen jolt
#

that'd be pretty annoying when trying to distinguish them

#

you'd need to call id manually

#

(when logging, I know you could just compare the 2)

trim tangle
#

!e
ooh this is fancy:

print(object.__repr__("foo"))
rough sluiceBOT
#

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

<str object at 0x7f3bb9917770>
trim tangle
#

!e
and this is cursed

print(int.__repr__(True))
print(int.__repr__(False))
rough sluiceBOT
#

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

001 | 1
002 | 0
brazen jolt
rare scarab
lunar dune
rare scarab
#

This is how npm's is-array is implemented

brazen jolt
#

oh god

#

lua is sort of similar too

rare scarab
rough sluiceBOT
#

index.js lines 31 to 33

module.exports = isArray || function (val) {
  return !! val && '[object Array]' == str.call(val);
};```
soft matrix
#

lmao what

undone saffron
#

I think the "rule" that repr should result in a way to reproduce the object is a silly rule that ignores everything from stateful objects, things not intended to be constructed manually, things which have multiple methods of construction (classmethods vs __init__) and is generally just pretty safe to ingore.

Make repr how you want a developer using it at a cli testing random things see it, and don't worry too much about that "rule"

trim tangle
#

Yep

#

It should pretty much communicate what kind of object that is, and possibly something about its state

brisk hedge
#

reproducible reprs are very nice

trim tangle
#

One thing I'd love to have is for functions with closures to show the captured objects' values in the repr. Not sure what can go wrong here

#

maybe if the closure is too big

heady flicker
trim tangle
mental loom
#

what if you said repr should have all the info a human needs to see to build the object?

trim tangle
#

Or an expression using ctypes to fill in the exact bits. With optional networking calls if the object is holding some sockets

tranquil turtle
#

!e ```py
import fishhook
import types

@fishhook.hook(types.FunctionType)
def repr(self: types.FunctionType) -> str:
return '<'+' '.join([self.class.name,self.qualname,*(f'{n}={v.cell_contents}'for n,v in zip(self.code.co_freevars,self.closure or()))])+'>'

adder = lambda x: lambda y: x + y
add2 = adder(2)
add3 = adder(3)
assert add2(3) == 5
print(f'{adder = }')
print(f'{add2 = }')
print(f'{add3 = }')

f = lambda a, b: lambda: a + b
print(f'{f = }')
print(f'{f(3, 5) = }')

rough sluiceBOT
#

@tranquil turtle :white_check_mark: Your 3.11 eval job has completed with return code 0.

001 | adder = <function <lambda>>
002 | add2 = <function <lambda>.<locals>.<lambda> x=2>
003 | add3 = <function <lambda>.<locals>.<lambda> x=3>
004 | f = <function <lambda>>
005 | f(3, 5) = <function <lambda>.<locals>.<lambda> a=3 b=5>
tranquil turtle
trim tangle
#

Nice

trim tangle
tranquil turtle
trim tangle
#

Yeah I've seen this one πŸ˜”

#

I should install a spell checker or something

wicked scarab
# 
from typing import overload, Literal

@overload
def foo(a: int, b: Literal[False]) -> list[int]: ...
@overload
def foo(a: int, b: Literal[True]) -> int: ...
@overload
def foo(a: int, b: bool = ...) -> int | list[int]: ...
def foo(a: int, b: bool = True) -> int | list[int]: ...

reveal_type(foo(1))
``` Defaulted value is `True`, but pyright returns `"int | list[int]"`. Any idea?
oblique urchin
wicked scarab
oblique urchin
#

the solution is to add a default to the correct overload

wicked scarab
#

That works as expected! Thanks

shell furnace
echo knot
# wicked scarab I see. Is there a reason for type checkers not to do that?

It goes from top to bottom checking the overloads. If you trigger foo with the default value for b, the two first overloads are skipped, as these do not contemplate default values. These overloads are still valid as they are a subset of what the original signature proposes, so nothing is being broken

#

That's why what Jelle said works. If you do Literal[True] = True, the type checker will also assign this overload when you are using foo with the default value for b

viscid spire
echo knot
#

What is your opinion about inheritance vs typing.Protocol? I use both of them but I am not sure if there could be a major difference between them

I tend to use more inheritance as the problems can be caught statically and at runtime, while with Protocol is purely static. Therefore, I use it as an adapter when I am using a library I cannot modify.

But, can Protocol be useful in some other scenarios? Like, to avoid problems with the MRO or something?

rare scarab
#

jinja has a pretty neat protocol. ```py
class HTMLRenderable(Protocol):
def html(self) -> str: ...

echo knot
brazen jolt
#

inheritence is slower on runtime

#

not by a huge amount, but it's there

#

if a function expects HTMLRenderable, it's up to your type-checker to ensure that you've passed in the corect thing

#

but it is assumed that all of the protocol methods will be implemented

#

regardless of whether that class did or didn't inherit from the base

#

this can be especially useful when you're using instances from some third party library

#

those don't necessarily inherit from your base class, and you'd need empty classes just to make them inherit from both

#

it is true that protocol is less "safe" in that it allows you to initialize a class even if some methods weren't implemented, but that's why we have type checkers, protocols are a typing feature, they're not meant to be a replacement of ABCs

#

as an example, take this protocol: ```py
class SupportsEqWithBools(Protocol):
def eq(self, other: bool) -> bool:
...

#

it would be really annoying if a function that was doing an equality check with bools could only accept instances of classes that have directly inherited from an ABC like this, but with a protocol, inheritence isn't required, a type checker will just verify whether this is something you class supports, and if it does, you can use it.

# 
@dataclass
class Foo:
    x: bool

    def __eq__(self, other: bool) -> bool:
        return self.x == other

# you can still inherit from a protocol, but it's optional.
# if you do, a type checker will make sure that you've implemented
# all of the needed methods
class Bar(SupportsEqWithBools):
    x: bool

    def __eq__(self, other: bool) -> bool:
        return self.x == other


def do_stuff(x: SupportsEqWithBools):
    if x == True:
        print("yay")
    else:
        print("nay")


do_stuff(Foo(False))  # valid, a type checker won't complain
do_stuff(Bar(False))  # also valid
#

@echo knot

echo knot
#

Thanks for the long explanation, it always gives me another perspective

brazen jolt
#

well yeah, without a type checker, protocols would be pretty useless

echo knot
brazen jolt
#

type checker tells you before you run the code!

#

you just have to have it configured in your editor

brazen jolt
#

just like say flake8

echo knot
#

They see it as a way of documenting code, but not the IDE autocompletion features and static analysis you can get. The boost is huge

brazen jolt
# echo knot They see it as a way of documenting code, but not the IDE autocompletion feature...

yup, and not just completion wise, your code is also much less likely to contain bugs. The type checker will scream at you very loudly if you try to access a variable that doesn't exists (say you made a typo, or you thought you were working with some other variable). It will make sure you're always passing exactly what functions expect from you, so doing add_numbers("hi", "hello") would show up as an issue too, because the func expected ints, etc.

regal summit
#

what does this even mean

image.png
# 
    class _WordleData(TypedDict):
        _1: _Node
        _2: _Node
        _3: _Node
        _4: _Node
        _5: _Node
        _6: _Node
        _X: _Node``` it will work that aint the problem
tranquil turtle
#

You dont need typeddict, use dict[str, _Node]

chrome hinge
#

or e.g.

Sym = Literal["1", "2", "3", "4", "5", "6", "X"]

and have the dict be dict[Sym, Node]

trim tangle
#

Ah yes good ol "LiteralString" is not a string literal

#

Such clear much error

native marsh
#

Is it possible to use an @overload decorator as a sort of type guard inside a function? I have the following code and I'm not sure how to properly type it.

from typing import Literal
from typing import TypedDict
from typing import overload

toggleT = Literal["one"] | Literal["two"]

class Foo(TypedDict):
    foo: int

class Bar(TypedDict):
    bar: int

@overload
def func(x: Literal["one"], y: Foo) -> None: ...
@overload
def func(x: Literal["two"], y: Bar) -> None: ...
def func(x: toggleT, y: Foo | Bar) -> None:
    if x == "one":
        print(y["foo"])
    if x == "two":
        print(y["bar"])
#

MyPy complains about the above example with the following

test.py:19: error: TypedDict "Bar" has no key "foo"  [typeddict-item]
test.py:21: error: TypedDict "Foo" has no key "bar"  [typeddict-item]
Found 2 errors in 1 file (checked 1 source file)
viscid spire
viscid spire
#

also note that you wrote y == "two" by accident

native marsh
viscid spire
#

I think the issue is that passing something of type Foo | Bar is also valid

native marsh
#

Was it clear from the example what I'm trying to achieve though?

viscid spire
#

yes

native marsh
#

I don't really mind about the arguments at this point I don't think, I'm mostly trying to achieve type narrowing of one type based on the value of another type

#

I think if I can do that, I should be able to work back up to the arguments

tranquil turtle
#

It is not possible
Typechecker dont care about overloads when typechecking implementation, it look only at implementation signature to typecheck implementation

native marsh
#

Thanks, I sort of assumed that would be the case

echo knot
#

Is there a way to speed up Mypy? Either by any plugin or trick.

pylizer static type checker is in Rust but still in a very early stage

trim tangle
echo knot
whole grotto
#

for me pyright is much slower than mypy

viscid spire
#

Ruff

trim tangle
#

at least for me

echo knot
# viscid spire ~~Ruff~~

Already using it. And that's what motivated me to research more about speeding up the static type checking, as I love having such quick feedback while coding 😁

rare scarab
#

to me, pyright is slower when doing a cold start. After it warmed up, it's much faster.

leaden oak
#

dmypy is definitely worth trying, but pyright is usually faster regardless

rare scarab
#

I wish mypy --watch

#

It's why we need a type checker written in rust

#

the whole ci toolchain should use rust

trim tangle
rare scarab
#

and the shell script interpretter

#

rsh

#

rash

#

rsh is posix. rash has bashisms

#

not to be confused with rsh (remote shell)

echo knot
# rare scarab I wish `mypy --watch`

Watch would be only for tracking changes right? I am not sure about it

There is one guy that is doing a whole progress implementing first useful type checker in rust. He is literally doing it alone

rare scarab
echo knot
rare scarab
#

interesting

#

Incremental mode:
Adjust how mypy incrementally type checks and caches modules. Mypy caches type information about modules into a
cache to let you speed up future invocations of mypy. Also see mypy's daemon mode:
mypy.readthedocs.io/en/stable/mypy_daemon.html#mypy-daemon

--no-incremental Disable module cache (inverse: --incremental)
--cache-dir DIR Store module cache info in the given folder in incremental mode (defaults to
'.mypy_cache')
--sqlite-cache Use a sqlite database to store the cache (inverse: --no-sqlite-cache)
--cache-fine-grained Include fine-grained dependency information in the cache for the mypy daemon
--skip-version-check Allow using cache written by older mypy version
--skip-cache-mtime-checks
Skip cache internal consistency checks based on mtime

echo knot
rustic gull
#

I find it odd that, in terms of gradual typing in Python, tuple and list are considered two very divergent concepts despite the fact one has been designed as immutable (tuple), the other one being mutable (list)
Both are generic. For tuples, this takes a tuple of types (like tuple[int, str]), but for lists, it takes a single type (like list[int | str]).
Meanwhile, in Rust, both tuples and arrays can be mutable (as long as you declare them mut)

leaden oak
#

If you really want a list-like tuple, you can do tuple[str, ...]

echo knot
mortal cipher
#

is there a way to model this in python types? i want to be able to tell the type checker that i've added an annotation to a model by using a method on the QuerySet (this is django):

from django.db import models

_M = TypeVar("_M", bound=MyModel)

class AnnotatedWithFoo(Protocol):
    foo: bool

class MyQuerySet(models.QuerySet[_M]):
    def with_foo(self) -> QuerySet[???]:  # In Typescript I'd do "M & AnnotatedWithFoo", what to do here?
        return self.annotate(foo=...)


class MyModel(models.Model):
    pass
#

i'd rather not add something like class MyModelWithFoo(AnnotatedWithFoo, MyModel): because that's gonna explode in combinations with 3-4 annotations

#

python typing makes me sad sometimes.

narrow remnant
#

oh sorry, I'm new here

trim tangle
echo knot
mortal cipher
#

it feels like such a simple complement to union types, can't believe it's been open for 7 years :^(

undone saffron
# mortal cipher it feels like such a simple complement to union types, can't believe it's been o...

It's a little less simple than it otherwise would be because of the existence of Any. (And that the definition of Any presents some contradictory conclusions).

There's a lot of ongoing discussion right now that has covered how some of this is more complex than it appears on the surface, even though it feels like it should be a simple complement.

The crux of the issue is here https://github.com/python/typing/issues/213#issuecomment-1646681413

If you'd like a summation of what that has led to in terms of potential options for reconciling Any, we have: https://github.com/CarliJoy/intersection_examples/issues/1#issuecomment-1655245452

There's also another approach which would say that Intersections don't intersect types directly but create a virtual protocol. This would make intersections structurally typed only.

#

that said, I think there's real progress being made right now towards it being a thing, it's not just endless bikeshedding on why it can't work, we're looking for pragmatic options.

red atlas
#

is there a proper way to annotate "iterable of strings except str itself" now? last I remember pytype was the only type checker that had a special case for this

trim tangle
#

I don't think there is

#

This one is pretty tricky because, fundamentally, a string is an iterable of strings... And there's not a way to express "X-but-not-Y"

tranquil turtle
#

X & ~Y

rare scarab
#

Is there a typesafe way to check an attribute for a certain type (i.e. not none)?

#

The way I'm doing it, the type checker still thinks its none. ```py
case Schema(defs=defs) if defs is not None:

tranquil turtle
#

Looks like a bug

#

What typecheker are you using?

rare scarab
#

pyright

#

oh, it's not a bug

#

I was wasn't actually using the arg

#

I was doing this. ```py
match schema:
case Schema(defs=defs) if defs is not None:
do_something(schema.defs)

rare scarab
#

Oh, I realized I can also do case Schema(defs=dict() as defs)

granite coyote
# 
    epaisseur = difprops.epaisseur
    profondeur = difprops.profondeur
    tenon_cadre = difprops.tenon_cadre
    bord_cadre = difprops.bord_cadre
    largeur_diffuseur = difprops.largeur_diffuseur
    offset_mortaise_interne = difprops.offset_mortaise_interne
    tenon_peigne = difprops.tenon_peigne
#

Is there a quicker way to write this ?

#

i think that difprops is a class

#

so it is not iterable

rare scarab
#

Not in a type-safe way.

#

locals().update(difprops.__dict__)

tranquil turtle
rare scarab
#

I did say it wasn't typesafe.

tranquil turtle
#

You have to perform some ctypes trickery after that

tranquil turtle
#

!e ```py
def f():
x = 1
locals()['x'] = 2
print(x)
f()

rough sluiceBOT
#

@tranquil turtle :white_check_mark: Your 3.11 eval job has completed with return code 0.

1
tranquil turtle
#

So that will not work after ctypes trickery too

rare scarab
#

if only we had proper dict destructuring

tranquil turtle
#

We have pattern matching

rare scarab
#

so... ```py
match difprops:
case Foo(a=a, b=b, c=d):
pass

print(a, b, c)

regal summit
# 
def create(depth: int) -> ...:
    if chances[depth] > random.random():
        return [create(depth + 1) for _ in range(4)]
    return depth``` is there a way to typehint this function return type?
viscid spire
#

Think you can do it with the trick where you put the type in a string maybe

# 
from typing import TypeVar
T = TypeVar("T")
NestedList = list[T | "NestedList[T]"]
...
#

believe that is what you gotta do pre 3.12

regal summit
#

dont think its gonna recognise NestedList pre 3.12

#

not working for me anyway

viscid spire
regal summit
viscid spire
# 
from typing import TypeVar
T = TypeVar("T")
NestedList = list[T | "NestedList[T]"]

x: NestedList[int] = [1, [1, 2], [[4, 3, "a"]]]
#                                         ^
#                                    incompatible type
viscid spire
regal summit
#

doenst recognise it in the defenition I guess

image.png
#

might have to update to .12 been wanting to do that anyway

viscid spire
#

Pylance can handle it for me πŸ€·β€β™‚οΈ

#

in 3.11.3

regal summit
#

oh my pylance was outdated I guess

viscid spire
#

πŸ˜„

regal summit
#

oh not really actually

#

doesnt seem to be complaining abt x: NestedList[int] = [1, [1, 2], [[4, 3, "a"]]]

viscid spire
#

did you reload the window?

regal summit
#

after a reload pylance seemed to be outdated again

#

im not sure, ill try a few things and figure it out for myself, thanks!

cosmic cave
#

Is there a correct way to declare a mixin class that requires subclasses of the mixin to implement some interface?

For a game, I want to write a mixin that can be added to any class with a position: Tuple[float, float] property. This mixin will add a variety of methods for manipulating the position: move left, right, up, down, towards a point, apply velocity, compute distances, etc.

This mixin can only be applied to classes that implement a position: Tuple[float, float] attribute. But the mixin doesn't care if position is a property or a plain attribute. Subclassing from ABC effectively forces position to be a property

Is there a better way to declare this requirement?

#

I can declare a position attribute directly on the mixin. The mixin will never initialize that attribute -- that responsibility still rests with the subclass. But maybe that's the best option

trim tangle
#

generally I would avoid these mixins and use a simple function

cosmic cave
#

ok thanks

trim tangle
#

if what you really have is a function (Position, Foo) -> Bar, no need to couple it with some other stuff

#

tbh I just never understood this style of mixin

cosmic cave
#

Yeah, it's for a pre-existing library, trying to reduce duplication of a pattern that exists across multiple classes, without changing their interface

finite moon
#

Quick question: is there any way to "require" that an object inherit from two classes?

def f(obj: A | B): ...

This requires obj to either inherit from A or from B, but not necessarily both. I'd like to take in objects that inherit from both.

Something like the following (which obviously isn't valid):

def f(obj: A + B): ...  # or A & B

I know that if both A and B are protocols, I can just do:

class AB(A, B, Protocol):
    ...

Then use AB; so that's solved. But my problem is that one of the two classes has to be Exception, which I don't think I could make a proto for... e.g.:

def f(serializable_exc: Exception + Serializable): ...
oblique urchin
finite moon
#

@oblique urchin btw, do you know if there's any ongoing discussion around making typecheckers respect the return type of __class_getitem__?

oblique urchin
finite moon
#

ok thanks!

stoic warren
#

why doesn't the following work? it seems like + should concatenate the tuple types but it doesn't ```py
X = TypeVarTuple("X")
Y = TypeVarTuple("Y")

def concat(x: tuple[*X], y: tuple[*Y]) -> tuple[*X, *Y]:
return x + y

#

x + y has the type tuple[Union[*X@concat] | Union[*Y@concat], ...] which doesnt really make sense

chrome hinge
viscid spire
#

they should allow it πŸ₯Ί

stoic warren
#

the issue with + does though

#

more like this ```py
X = TypeVarTuple("X")
Y = TypeVarTuple("Y")

@dataclass
class Foo(Generic[*X]):
x: tuple[*X]

def concat(x: tuple[*X], y: tuple[*Y]) -> Foo[*X, *Y]:
return Foo(x + y)

chrome hinge
#

I think this isn't different - you can't unpack more than one TypeVarTuple into the same list of parameters.
The PEP doesn't say this explicitly though, hmm. I think it's unspecified: https://peps.python.org/pep-0646/#acceptance

soft matrix
echo knot
#

Does it make sense to have some kind of AbstractClassVar that would enforce the developer to override it if they create a subclass? So far the only option I think it is is just combining decorators classmethod with abstractmethod, but I find it quite verbose. Maybe there are better ways

#

If I do something like this, MYPY does not throw any errors in the whole file. However, it will throw an error in runtime

from typing import ClassVar
from abc import ABC

class A(ABC):
    var: ClassVar[int]
    
    def method(self) -> None:
        print(self.var)
        
class B(A):
    var: ClassVar[int] = 2
    
    
class C(A):
    ...
    
C().method()   # No MYPY error
fossil sorrel
#

i will ask a very dumbo thing

# 
 filter: Optional[Dict[str, Any]] = None,root: Optional[rootmodel] = None
```` how can i type hint to say "filter is required if root is none" ?
soft matrix
#

you need overloads

fossil sorrel
soft matrix
# 
@overload
def foo(filter: Dict[str, Any], root: rootmodel): ...
@overload
def foo(): ...
#

i think that works

fossil sorrel
carmine phoenix
#

should I just use the regular SQLAlchemy instead of Flask-SQLAlchemy for a flask app because typing experience is very bad?
it uses alot of magic with __getattr__, so its impossible to create stubs for the most part.

rustic gull
soft matrix
#

typing

rustic gull
soft matrix
#

It's in every version of typing, the only thing that typing extensions might be useful for in that regard is get_overloads

echo knot
eager vessel
granite coyote
#

Hey guys !
What is the syntax to add list in list only if condition is true ?

image.png
trim tangle
#

This is a channel for talk about type annotations

granite coyote
#

ok!

fierce ridge
# 
GraphNodeSpec: TypeAlias = SimpleTask | tuple[SimpleTask, Collection[SimpleTask]]

@dataclass()
class Graph:
    nodes: MutableSet[GraphNode]

    def _to_graphlib(self) -> Mapping[SimpleTask, Set[SimpleTask]]:
        return {node.task: set(node.dependencies) for node in self.nodes}

    def topological_sort(self) -> Sequence[SimpleTask]:
        ts = TopologicalSorter(self._to_graphlib())
        return list(ts.static_order())

    @classmethod
    def build(cls, node_specs: Collection[GraphNodeSpec]) -> Self:
        nodes: set[GraphNode] = set()
        for node_spec in node_specs:
            match node_spec:
                case SimpleTask() as task:
                    nodes.add(GraphNode(task, frozenset()))
                case [SimpleTask() as task, [*deps]]:            # <-- Mypy error
                    nodes.add(GraphNode(task, frozenset(deps)))
        return cls(nodes=nodes)

mypy says this:

Argument 1 to "frozenset" has incompatible type "List[object]"; expected "Iterable[SimpleTask]"  [arg-type]

is there a better way to write this case clause so that mypy doesn't get confused? i'd like to avoid a typing.cast or assertion if possible here

#

(Collection here is collections.abc.Collection)

#

i rewrote it as case [SimpleTask() as task, deps]: which i guess is fine

#

heck i realized i can just do case [task, deps] because the type hints take care of it all

#

types are good

rustic gull
#

i was wondering if i could set default value of cast to str so i would get rid of this if check if self.cast is not MISSING:

fierce ridge
#

are you writing your own type checker or something?

rustic gull
fierce ridge
#

i'm wondering what this Variable class is for

brazen jolt
#

looks like something for env vars

rustic gull
#

to retrieve value from env

fierce ridge
#

ah

#

you can generalize cast to Callable[[str], T]

#

(and i'd suggest calling it convert or load instead of cast)

rustic gull
fierce ridge
#

depending on how fancy you want to get, you can maybe even allow the callable to return MISSING and fall back to the default

#

i'm also not sure i follow the intended usage of this with __get__ but you're just asking about the types

#

oh, you intend to use this as a descriptor

#

i personally would write a plain get() method and have __get__ just call that

rustic gull
#

Expression of type "type[str]" cannot be assigned to declared type "(str) -> T@Variable"
Β Β No overloaded function matches type "(str) -> T@Variable"

brazen jolt
#

also, you might not always want to have the default type go through your cast function, I did something similar somewhere in my projects, though it was just an overloaded function to obtain the env vars, not a full class, and I basically used 2 type vars, one for the result of a that convert/cast function, and the other for the default value, returning a union

#

this imo makes a bit more sense usually, as if you wanted the default value to go through the casting process, you can always do that yourself when initializing

#

it's just that often you want say None in there, and it'd be annoying to have that go through your func

fierce ridge
#

e.g. pydantic has a toggle for it (but annoyingly it's only for the whole class, not per-field)

#

in general i think i agree that the default value shouldn't need to be converted most of the time

fierce ridge
brazen jolt
#

yeah true, I just found in my experience that I often wanted None especially to be the default there, so not converting made more sense, it does depend on what you're doing

rustic gull
# 
T = t.TypeVar("T")


@dataclass(kw_only=True)
class Variable(t.Generic[T]):
    name: str
    default: t.Any = MISSING
    cast: t.Callable[[str], T] = str

    def __post_init__(self) -> None:
        self.value = os.getenv(self.name, self.default)
        if self.value is MISSING:
            raise MissingEnvironmentVariable(self.name)

        try:
            self.value = self.cast(self.value)
        except Exception as e:
            raise ConversionError(self.name, self.cast, e) from e

    def __get__(self, instance: t.Any, owner: t.Any) -> T:
        return t.cast(T, self.value)
fierce ridge
rustic gull
#

no when defining cast field

#

cast: t.Callable[[str], T] = str

fierce ridge
#

oh, str isn't being recognized as a str -> str

rustic gull
#

it doeesnt like that i set default value as str

fierce ridge
#

just set it to MISSING by default

# 
cast: t.Callable[[str], T] | type[_MissingSentinel] = MISSING
rustic gull
#

yeah so i need that if either way

brazen jolt
#

no, what you'd actually need is making the callable take in a union of str and any

#

since your default value has t.Any

#

ideally, convert that into another generic typevar

fierce ridge
#

that default should be T not Any

brazen jolt
#

well not necessarily

fierce ridge
#

if the T itself is a union then fine, but i don't think you want two type parameters there

brazen jolt
#

since the default goes through the func itself

#

at least currently

rustic gull
# fierce ridge that default should be `T` not `Any`

now when i do that its this ```py
Argument of type "str | T@Variable" cannot be assigned to parameter of type "str"
Β Β Type "str | object*" cannot be assigned to type "str"
Β Β Β Β "object*" is incompatible with "str"

#

when trying to call cast

fierce ridge
#

true @brazen jolt , but i'd agree with you that it shouldn't be

rustic gull
brazen jolt
#

well yeah, that was the point I was making before

rustic gull
brazen jolt
#

either that, or have it be of type T as well, and skip casting

rustic gull
brazen jolt
#

yes

fierce ridge
#

i'd also strongly suggest not doing i/o in __post_init__. completely unrelated to types

brazen jolt
rustic gull
fierce ridge
#

in fact i see another issue. self.value is un-annotated and appears out of nowhere. i don't like that, it's a very old school python thing and i think it's best left in the history books.

fierce ridge
brazen jolt
#

you could go with a property/cached property here

rustic gull
# rustic gull now when i do that its this ```py Argument of type "str | T@Variable" cannot be ...

modified to this: ```py
@dataclass(kw_only=True)
class Variable(t.Generic[T]):
name: str
default: T = MISSING
cast: t.Callable[[str], T] = MISSING

def __post_init__(self) -> None:
    self.value: str | T = os.getenv(self.name, self.default)
    if self.value is MISSING:
        raise MissingEnvironmentVariable(self.name)

    if self.cast is not MISSING and not self.default:
        try:
            self.value = self.cast(self.value)
        except Exception as e:
            raise ConversionError(self.name, self.cast, e) from e

def __get__(self, instance: t.Any, owner: t.Any) -> T:
    return t.cast(T, self.value)


this still appears
#

maybe i should start from the beggining and type hint it properly from start

fierce ridge
#

this is actually a little bit of a thorny one. the types don't unify the way you'd expect them to

#

for example if you do something like self.value = value if isinstance(self.cast, Missing) else self.cast(value) then you get problems unifying T with str

#

that is, there's no way to tell it that T must be str when cast = MISSING

#

and yeah you have the other problem where str isn't recognized as a Callable[[str], T], nor is def identity(x: T) -> T: return x

#

you can do this with overloads but it gets a little "combinatorial"

rustic gull
#

yeah and i belive t.Type[T] is enough here

fierce ridge
#

up to you. definitely makes your life easier if you don't mind being limited to int() bool() etc

brazen jolt
# 
import os
from functools import cached_property
from typing import Any, Callable, Generic, NewType, TypeVar, cast, overload

_T = TypeVar("_T")
_Sentinel = NewType("Sentinel", object)
_MISSING = cast(_Sentinel, object())


class Variable(Generic[_T]):
    @overload
    def __init__(self: "Variable[str]", *, name: str, cast: None = None, default: _Sentinel = _MISSING) -> None:
        ...

    @overload
    def __init__(self: "Variable[_T]", *, name: str, default: _T = ...) -> None:
        ...

    @overload
    def __init__(
        self: "Variable[_T]", *, name: str, cast: Callable[[str], _T], default: _T | _Sentinel = _MISSING
    ) -> None:
        ...

    def __init__(self, name: str, cast: Callable[[str], _T] | None = None, default: _T | _Sentinel = _MISSING) -> None:
        self.name = name
        self.default = default
        self.convert = cast

    @cached_property
    def value(self) -> _T:
        if self.name not in os.environ:
            if self.default is _MISSING:
                raise

            # only _MISSING is of type _Sentinel, we know this is _T
            return cast(_T, self.default)

        value = os.environ[self.name]

        if self.convert is not None:
            try:
                return self.convert(value)
            except Exception:
                raise

        # When cast is None, and no default, _T has to always be str, which matches value
        return cast(_T, value)

    def __get__(self, instance: Any, owner: Any) -> _T:
        return self.value

I don't even know what I made at this point lol, it pases ig but it's very hacky

trim tangle
brazen jolt
#

why?

#

first overload handles that

trim tangle
#

hm... I suppose yeah, they're considered in order

brazen jolt
#

I mean I said it's hacky

#

it probably works though

#

whoops, the 2nd overload should have "Variable[str]" there, and default: str actually: ```py
@overload
def init(self: "Variable[str]", *, name: str, default: str = ...) -> None:
...

#

Β―_(ツ)_/Β―

#

not sure if you want to use something this complex though

rustic gull
#

yeah.. i have no idea what it does

brazen jolt
#

well, it uses overloads to bind the typevar to something specific

rustic gull
#

isnt that opposite of what typevars should do

brazen jolt
#

not necessarily

#

typevars live through the whole class

#

this tells the instance which type should the type var get bound to

#

during initialization

#

a type checker isn't smart enough to figure out what the type var should be if we leave both cast and default empty

#

(i.e. have them fall back to the defaults, which is just None/_MISSING)

#

the first overload tells the type checker that it should actually become str in this case

#

since we'll just return the value directly, and env values are strings

#

in the second case, we do have a default value, but still no cast func, so we know this again has to be a string, since after value is obtained, it's a string, and we won't do any casting, so it will stay a string

#

We might fall back to default, if it wasn't there, so we also specify default should be a string

#

and lastly, when we do have a cast function (3rd overload), the return type from that cast callable should be the type our typevar should become, and that's also the type for the default values

rustic gull
#

okay i see now

brazen jolt
#

but yeah, it's still pretty complex code, and it might not be worth putting in as it could be annoying to maintain

rustic gull
brazen jolt
rustic gull
brazen jolt
#

how so?

#

you're creating an instance of a class only to use it's __get__ and get back a single value

rustic gull
brazen jolt
#

that's a sign you might want a function instead

rustic gull
#

lets say i come from OOP language

brazen jolt
#

I mean it will work, but there's a lot of needless initializations, and the function would do exactly the same thing, but faster and I'd say it's much cleaner to understand

#

not many people are familiar with descriptors, and it's usually a bad idea to introduce needless complexity

#

especially if this is for an open source project where others might contribute, really complex code can easily discourage people from doing so

rustic gull
#

its not having any methods so

fierce ridge
#

this is basically reinventing Mapped from sqlalchemy

#

unfortunate that it doesn't work with dataclass

lethal delta
#

Is there a type annotation to annotate an argument which is a module?

tranquil turtle
#

x: types.ModuleType

lethal delta
#

Ah, that explains it

#

I looked in typing

lethal delta
fierce ridge
tranquil turtle
#

Also it is not good to use types for type annotations. A lot of things in types are "bad" from typing perspective (modules are indistinguishable, functions and methods has no signatures attached to them, ...)

fossil sorrel
# 
pydantic_core._pydantic_core.PydanticSerializationError: Error calling function `_serialize`: AttributeError: 
``` am i the only one finding pydantic very fucking annoying
#

raises error, without a message

#

is there like, pydantic but more user friendly for custom types :/

undone saffron
#

msgspec or attrs (with cattrs for (de)serialization) provide good alternative options

fierce ridge
#

if it makes you feel better I haven't yet encountered a problem where the error messages are opaque and bad. usually that's a pandas issue

fierce ridge
#

i've really grown to like the pydantic "validator" system and recently drafted up a mini framework that i think is a cleaner implementation of the same idea. mostly an exploration of design space but maybe also something that someone might want to use

undone saffron
#

pydantic by default will (implicit behavior, not explictly requested) coerce types to validate them, which I find to be distasteful for my own use, and it's also just measurably slower than other options, by orders of magnitude.

fierce ridge
soft matrix
#

i liked the look of pydantic from a talk from its creator about v2 if you put it into strict mode

fierce ridge
#

also conflating attrs/dataclass-like "record" objects with deserialization causes problems with type annotations

#

very often you want a much more flexible type in the signature of __init__ than on the attribute itself

fierce ridge
rare scarab
undone saffron
rare scarab
#

How can I ignore method-assign in mypy?

# 
        if handle_request is not None:
            self.handle_request = handle_request  # type: ignore [mypy=method-assign]

error: Unused "type: ignore" comment [unused-ignore]
error: Cannot assign to a method [method-assign]
note: Error code "method-assign" not covered by "type: ignore" comment

#

seems only # type: ignore [method-assign] works

oblique urchin
rare scarab
#

I could've sworn there was a way to mark it was only mypy or only pyright, etc

lunar dune
#

No, just a lot of people asking for that

young zealot
#

Why is method assign an issue? Is there something dangerous about it or ?

rare scarab
#

I also ended up doing this instead. ```py
self.handler = handle_request or self.handle_request

rustic gull
#

Hello! I'm here to ask about intersection types i'm afraid :')

Although we can't hint them directly, pyright can infer intersection types:

def foo(bar: int):
    if not isinstance(bar, str):
        raise
    return bar

x = foo(1)
reveal_type(x) # Type of "x" is "<subclass of int and str>"

Does anyone know of a way that I could take of advantage of this in a TypeGuard?

My specific case is that I would like to write a type guard for the nullability of one of my class's attributes:

class Item:
    name: Optional[str]

@runtime_checkable
class HasName(Protocol):
    name: str

    def __instancecheck__(self, __instance):
        return isinstance(__instance, Item) and __instance.manufacturer is not None

def hasName(item: Item) -> TypeGuard: # Expected a single type argument after "TypeGuard"
    return isinstance(item, HasName)
#

I could have sworn I had once seen a similar outcome achieved with NewType, but after looking up NewType it seems that's not what it's for

rare scarab
#

you could combine protocols

#

TypeGuard[HasName]

fierce ridge
#

the deserializers would run in the order that they're listed

#

internally it just applies each deserializer sequentially and then does class_or_callable(**data)

soft matrix
#

does anyone know of a tool to create a typed dict definition from an input dict

rare scarab
#

You mean like TypedDict("DictName", {"foo": str})?

#

or maybe something more like json schema

#

!pip jsonschema-gentypes

rough sluiceBOT
soft matrix
#

idk how to get it into a schema

#

howd i get it into json schema from just a value?

rare scarab
#

oh, it's rust

soft matrix
#

you love to see it

image.png
#

time to go and cry to the devs

rare scarab
#

Can you share your json?

soft matrix
#

its a biggen

rare scarab
#

Do it in chunks.

#

note that spaces in keys aren't handled properly for classes

#

e.g. ```py
class Install(TypedDict):
copy files: CopyFiles
run process: RunProcess
registry: Registry
arma 2 cd key: Any
ea oreg: HkeyLocalMachineSoftwareIronLoreTitanQuestOrHkeyLocalMachineSoftwareBohemiaInteractiveStudioArma2OaExpansionsArma2OrEaOreg
registry if not present: RegistryIfNotPresent
microsoft .net framework 4: MicrosoftNetFramework4
chmod: Chmod
firewall: Firewall
run process on uninstall: RunProcessOnUninstall

soft matrix
#

sweet that works

rare scarab
#

Also it might generate a identifyer starting with a number. ```py
class Tracks(TypedDict):
0: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
1: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
10: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
11: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
12: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
13: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
14: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
15: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
16: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
17: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
18: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
2: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
3: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
4: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
5: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
6: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
7: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
8: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1
9: 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1

class 4Or18Or9Or2Or6Or12Or8Or3Or14Or10Or0Or16Or5Or7Or11Or15Or13Or17Or1(TypedDict):
discnumber: str
m: str
originalname: str
s: str
tracknumber: str

#

That really should be a list instead of an object

soft matrix
#

yeah its really from vdf which doesnt have a notion of lists so what can you do

#

i do have a fake list type for it though

trim tangle
willow crystal
#

How do you type hint the current class in an inheritance structure?
For example, let's say we have

class MyException(Exception, ABC):
    @staticmethod
    def flask_error_handler(e: ?) -> ResponseReturnValue:
        # Do something with e
        # Subclass can overwrite this method, or use the default

class MoreSpecificException(MyException):
    pass

What should the value of ? be? to make it always refer to the current subclass of MyException

oblique urchin
willow crystal
soft matrix
#

nope

#

using Self with staticmethods doesnt make sense

oblique urchin
#

yes, PEP 673 says so explicitly

soft matrix
#

am i losing it?

oblique urchin
#

in your code, maybe it shouldn't be a staticmethod

soft matrix
#

!pep 673

rough sluiceBOT
#
**PEP 673 - Self Type**

Link

Status

Accepted

Python-Version

3.11

Created

10-Nov-2021

Type

Standards Track

oblique urchin
#

no I'm agreeing with you. "Note that we reject Self in staticmethods."

soft matrix
#

oh ok cool

willow crystal
soft matrix
#

can it not pass cls?

#

aka make it a classmethod?

willow crystal
#

So flask will invoke the function like so:

### handling a MyException
e = MyException()
f = error_handlers[e.__class__]  # handlers registered for each Exception class
f(e)

Notice it is NOT calling e.flask_error_handler(), but just a flask_error_handler(e) on the corresponding class that I registered

oblique urchin
willow crystal
oblique urchin
willow crystal
soft matrix
#

you dont have to pass cls in classmethods (generally)

willow crystal
#

Oh! Ok, I'll try this approach

willow crystal
oblique urchin
#

sorry I was distracted by a deer on my balcony, lol

willow crystal
willow crystal
oblique urchin
willow crystal
#

Thank you everyone

trim tangle
#

classmethod binds the class

willow crystal
# trim tangle classmethod binds the class

I understand now. I guess I wasn't quite familiar with classmethod. I thought the cls would have to be specifically provided. Now I think of how self works in regular instance method without having to be supplied explicitly, it makes more sense now.

willow crystal
#

Sorry one more question, how do I denote in typing a subclass of a class?
In my case, I am trying to define a utility function to get all subclasses of a class:

def all_subclasses(cls: Type) -> set[Type]:
    subclasses = set(cls.__subclasses__())
    for subclass in subclasses:  # may be buggy, but idea is same
        subclasses.update(all_subclasses(subclass))
    return subclasses

How do I indicate that the returned set of classes are all subclasses of cls?

oblique urchin
#

def all_subclasses(cls: type[T]) -> set[type[T]]: where T = TypeVar('T')

willow crystal
#

I have seen generics in Java, didn't know it is a thing in Python too! Thanks!

willow crystal
#

@oblique urchin Do you know if there is any alternative to Self before 3.11?

rare scarab
#

typing_extensions.Self

#

You could also add a typevar on self

hidden apex
viscid spire
#

Self is not equivalent to the class exactly

# 
class C[T]: ... # Self is equivalent to C[T], not C
rare scarab
#

It also covers subclasses

viscid spire
#

🀯

lethal delta
willow crystal
fossil sorrel
#

erm

#

how can i help the type checker to infer ```py

@mongo_exception_handler
async def find_one_and_update(
    self,
    collection: str,
    filter: Mapping[str, Any],
    update: Union[Mapping[str, Any], _Pipeline],
    projection: Optional[Union[Mapping[str, Any], Iterable[str]]] = None,
    sort: Optional[_IndexList] = None,
    upsert: bool = False,
    return_document: bool = ReturnDocument.AFTER,
    array_filters: Optional[Sequence[Mapping[str, Any]]] = None,
    session: Optional[AgnosticClientSession] = None,
) -> Mapping[str, Any]:``` the parameters when used with a decorator?
# 
(method) def find_one_and_update(...) -> Coroutine[Any, Any, Mapping[str, Any]]
``` intellisense shows the function as this atm
# 
MongoDBHandlerType = TypeVar("MongoDBHandlerType")
def mongo_exception_handler(
    f: Callable[..., MongoDBHandlerType]
) -> Callable[..., MongoDBHandlerType]:
    @functools.wraps(f)
    def wrapper(*args: Any, **kwargs: Any) -> MongoDBHandlerType:
        ...my code here.
    return cast(Callable[..., MongoDBHandlerType], wrapper)``` decorator is just, classic decorator.
#

i cant share any more details :/

soft matrix
#

!d typing.ParamSpec

rough sluiceBOT
#
typing.ParamSpec 

class typing.ParamSpec(name, *, bound=None, covariant=False, contravariant=False)```
Parameter specification variable. A specialized version of [`type variables`](https://docs.python.org/3/library/typing.html#typing.TypeVar "typing.TypeVar").

Usage:

```py
P = ParamSpec('P')
```  Parameter specification variables exist primarily for the benefit of static type checkers. They are used to forward the parameter types of one callable to another callable – a pattern commonly found in higher order functions and decorators. They are only valid when used in `Concatenate`, or as the first argument to `Callable`, or as parameters for user-defined Generics. See [`Generic`](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") for more information on generic types.
soft matrix
#

you need to either use this or a bound TypeVar

fossil sorrel
#

but wouldnt this generate boilerplate?

#

like i write parameter specifications twice?

#

or wait do i use the parameter specifications in function too

#

i totally understood this from my ass sorry

soft matrix
#

ok i kinda cant be bothered explaining ParamSpec so for now just use a bound TypeVar

fossil sorrel
#

lmao, thanks

soft matrix
#

annotate f as HandlerFactoryT where HandlerFactoryT = TypeVar("HandlerFactoryT", bound=Callable[..., Any])

#

and make the return type HandlerFactoryT as well

fossil sorrel
#

TypeVar bound type cannot be generic Pylance
(class) Callable

soft matrix
#

oh sorry

#

use Any

fossil sorrel
#

yeeeeeeey it works

#

thanks a lot

echo knot
#

I need some help or opinions about one small issue I have

I am creating an API that many people will use in my company. In order to document/type hint properly, this API relies on many @overload. Source files are starting to be quite annoying when it comes to develop or scroll through code, as a high percentage of it is just useless overloaded function signatures.

I thought about using stubs. But as far as I know, Python only uses either stubs or inline type hints. As a consequence, all type hints will be moved to a .pyi. But now another problem arises, that leads to two scenarios:

  • Type hints are removed from py source file, which makes it harder for developers to read the code.
  • Inline type hints are provided in thr py file. However, there will be a duplication as they also need to be in the stub based file. Which causes some divergente and annoying problems related to maintenance

My question is, are there better alternatives? Like, would it be possible to move all @overload methods to a separare file and making the type checker scan both? Note that most of them are methods, so they are inside a class, which might make it harder

undone saffron
#

depending on why the overloads are the way they are (are you using type vars where you can?) you might be able to eliminate some for other reasons.

carmine phoenix
echo knot
rustic gull
# 
Argument of type "(self: Self@General, ctx: Unknown) -> Coroutine[Any, Any, None]" cannot be assigned to parameter of type "(Context[Unknown], **P@group) -> Coro[Any]"
Β Β Type "(self: Self@General, ctx: Unknown) -> Coroutine[Any, Any, None]" cannot be assigned to type "(Context[Unknown], **P@group) -> Coro[Any]"
Β Β Β Β Parameter 1: type "Context[Unknown]" cannot be assigned to type "Self@General"
Β Β Β Β Β Β "Context[Unknown]" is incompatible with "General"

class General(BaseCog):
    def __init__(self, bot: "Skurczybyk") -> None:
        super().__init__(bot)
        
    @commands.group()
    async def something(self, ctx):
        ...

    @something.group()
    async def subgroup(self, ctx):
        ...

library used is nextcord
and BaseCog is just class BaseCog(commands.Cog):

undone saffron
# echo knot Thanks for the feedback! Yes, I am using typevars whenever I can. But in these c...

would require intersections and the also proposed Not, but you'd be able to condense the signature into 1 type signature, typing that function as an intersection (via type comment, not directly)

This has how that would work, but there's still a lot of details being worked out, so it's still a "may" help there, not will: https://github.com/CarliJoy/intersection_examples/issues/17#issuecomment-1662856538

undone saffron
rustic gull
# 
class General(BaseCog):
    def __init__(self, bot: "Skurczybyk") -> None:
        super().__init__(bot)
        
    @commands.group()
    async def something(self, ctx: commands.Context[Skurczybyk]) -> None:
        ...

    @something.group()
    async def subgroup(self, ctx: commands.Context[Skurczybyk]) -> None:
        ...

Argument of type "(self: Self@General, ctx: Context[Skurczybyk]) -> Coroutine[Any, Any, None]" cannot be assigned to parameter of type "(Context[Unknown], **P@group) -> Coro[Any]"
Β Β Type "(self: Self@General, ctx: Context[Skurczybyk]) -> Coroutine[Any, Any, None]" cannot be assigned to type "(Context[Unknown], **P@group) -> Coro[Any]"
Β Β Β Β Parameter 1: type "Context[Unknown]" cannot be assigned to type "Self@General"
Β Β Β Β Β Β "Context[Unknown]" is incompatible with "General"
echo knot
errant tulip
#

Why doesn't python currently allow to have extra keys in TypedDict? We have Any type. I just want to set any extra key of type Any except for specific ones

undone saffron
# errant tulip Why doesn't python currently allow to have extra keys in `TypedDict`? We have `A...

Because the use case they were added for didn't need it and it was left for later: https://peps.python.org/pep-0589/#rejected-alternatives

TypedDicts were added for known fixed key structures. Having Any here doesn't exactly fit that, and gets complicated for reasons involving the theoretical differences between gradual and static types.

Should the intersection pep get finished and accepted, you'll be able to handle this with typing your dict as

SomeTypedDict & dict[str, Any]

soft matrix
#

shouldnt it be Mapping instead of dict?

undone saffron
#

possibly MutableMapping[str, Any], but I think either should probably be fine there.

I also checked if mypy or pyright had special casing for ChainMap already which could be leveraged, but unfortunately not.

soft matrix
#

oh its somewhere between Mapping and MutableMapping

#

cause pop doesnt really work

#

but anyway its beside the point, itd be cool either way

slender timber
undone saffron
# slender timber > `SomeTypedDict & dict[str, Any]` is that a sum type?

Intersection. Conceptually, merging of the two types, requires the types be compatible, etc.

Where Unions are "this might be either of these, and you can use things that all members have", at a simple level, Intersections as the dual to them are "The thing here works as each of these, and you can use the things available from each part"

There's a lot of details being fleshed out on these still, and we don't know if that will result in acceptance or not yet.

slender timber
undone saffron
wet vapor
#

How do we call the capacity of a type to describe an object ?

This isn't related to Python but more to the type theory in general.
Sadly I didn't found an answers while googling.
(In fact I might have come across the answer without even understanding that it was what I was looking for.)

rare scarab
#

Type completeness?

#

Is this homework?

wet vapor
tired stirrup
#

What's the standard way to annotate a Sequence with a fixed-size number of elements?

I always annotate a Sequence with 3 int's like this: Sequence[int, int, int]. However, I discovered that it's not valid Python.

I came across a solution on Stack Overflow https://stackoverflow.com/a/67146944 which used typing.Annotated[Sequence[int], 3]. I was skeptical of the answer and wonder if it were the standard way to do it.

Stack Overflow
Specify length of Sequence or List with Python typing module

I'm giving the Python typing module a shot.

I know that it's valid to specify the length of a List like the following*:

List[float, float, float] # List of 3 floats <-- NOTE: this is not valid

oblique urchin
tired stirrup
#

For real?

oblique urchin
rigid sapphire
#

hello

rustic gull
arctic ledge
#

To handle a signal (I mean SIGINT/SIGTERM), you pass a callable that takes (signal, frame) as arguments.

How do I annotate arguments in a function that takes those callables?

Example:

def handle_signals(on_sigint: callable(signal, frame) = None):
    signal.signal(signal.SIGINT, on_sigint)

How do I even figure out what types are those arguments supposed to be?

trim tangle
rough sluiceBOT
#

stdlib/signal.pyi line 73

def signal(signalnum: _SIGNUM, handler: _HANDLER) -> _HANDLER: ...```
arctic ledge
#

Thank you! I've figured out a lot of stuff while investigating this!

viscid spire
#

(unless I am unaware of the case of a single arg?)

arctic ledge
#

Based on what I see (PyCharm succeeding or failing to parse properly), this should be the correct way:

def myfunction(a: int = 0):

In case of a callable, its arguments are indeed in []'s.

#

Actually, now I see that it did pick up on the callable's arguments without annotations - because I've assigned the default to be a suitable callable!

viscid spire
# 
def register(on_sigint: Callable[[int], FrameType] | None = None):
wet vapor
#

Based on this issue (https://github.com/python/typing/issues/159), it seems like Python's core developers have already taken notice of the impossibility to annotate a slice.
However it isn't clear it will be implemented or if it has been judged as useless.

Does anyone know if slice annotation has been rejected or not ?

For people interested it's possible to annotate slices in function like that slice[int, Callable, str] by writing from __future__ import annotations at the start of your program.
The reason is that it will be interpreted as a string and so the interpreter won't detect that slice isn't subscriptable.

GitHub
Issues Β· python/typing

Python static typing home. Hosts the documentation and a user help forum. - Issues Β· python/typing

soft matrix
#

slice will be made generic if pep 696 happens

#

speaking of which i should probably make a list of the new generics for that so i remember to add them

wet vapor
#

ok thanks πŸ™‚

rustic gull
trim tangle
#

oh wait I can't read

rustic gull
trim tangle
soft matrix
#

Cog doesn't need to be generic

rustic gull
trim tangle
#

I think you'll have to type it as ctx: commands.Context and then to ctx: commands.Context[Skurczybyk] = ctx # type: ignore

trim tangle
trim tangle
soft matrix
soft matrix
rustic gull
# trim tangle wdym 
class MyBot(commands.Bot):
    ...

    def my_super_method():
        # do something

``` then when i say its generic of type MyBot i will get typehints about custom methods
soft matrix
trim tangle
trim tangle
rustic gull
trim tangle
soft matrix
#

But it's been a long time since I've really used the library

rustic gull
#

i can try setting it up with discord.py if you give me few minutes

trim tangle
rough sluiceBOT
#

discord/ext/commands/__init__.py lines 11 to 14

from .bot import *
from .cog import *
from .context import *
from .converter import *```
trim tangle
#

I don't want to clone the whole library and boot up my editor just to search for a single definition

rustic gull
#

you can search on github too

trim tangle
#

Its search is... not great

rustic gull
#

if thats what you mean. .

trim tangle
#

oh I think I found it

#

what version of nextcord are you using?

rustic gull
#

newest

#

2.5.1 i belive

#

!pypi nextcord

rough sluiceBOT
rustic gull
#

2.5.0

rustic gull
rough sluiceBOT
#

nextcord/ext/commands/core.py line 1769

def group(```
rustic gull
#

i belive thats it

trim tangle
#

what a pile of overloads

rustic gull
#

just a few

trim tangle
#

my brain refuses to understand this right now, sorry

#

it is about 33C here and I am melting

rustic gull
#

understand appreciate the effort though

frigid jolt
#

isn't it coz ContextT is bound to Context which is Context[Unknown]?

#

oh no that's not it

#

if i'm not mistaken that type hint doesn't take into account the possibility to be into a Cog, so to have self as first argument

#

ah no there's CogT

#

mb

undone saffron
rustic gull
undone saffron
#

literally just a poorly behaved fork causing you issues with typing here.

rustic gull
#

But actually

#

This is not the case i was showing

#

I was creating a subgroup

undone saffron
#

same deal if you change the last decorator to this.group

rustic gull
#

Okay then

undone saffron
#

(not gonna keep posting pics of code here)

trim tangle
undone saffron
#

sure, 1 sec...

undone saffron
trim tangle
rustic gull
#

I need to get through source code then

#

To see how it differs from nextcord implementation

undone saffron
#

Β―_(ツ)_/Β―

#

It's properly using concatenate and paramspec for decorators

#

and things which users can replace are generic to support it

#

not much to it beyond that really.

trim tangle
rough sluiceBOT
#

discord/ext/commands/core.py line 1717

def __call__(self, func: Callable[Concatenate[CogT, ContextT, P], Coro[T]], /) -> Group[CogT, P, T]:```
trim tangle
#

so it is kinda like a # type: ignore and lets you accept the context for any bot

undone saffron
#

that's not ignoring it, it's still checked as a valid context and used in the result type

trim tangle
#

You can still do def this(self, ctx: Context[FooBot]) and pass in a BarBot into the cog, no?

rustic gull
#

FooBot sounds really cool

undone saffron
#

That's not ignoring it though, you're missing the point slightly.

trim tangle
#

sorry if I phrased it badly
what I meant is that they compromised the "type safety" slightly

undone saffron
#

nope, not at all

trim tangle
#

then I don't understand you

undone saffron
#

None of the type safety is compromised here, this is correct expression. it's wrong to think of this as a type ignore, it isn't one.

trim tangle
#

which will cause an error in runtime

undone saffron
#

That's not compromising type safety, that's the person attaching it to something other than what they said providing a lie to the type checker at that point

#

You can't actually do anything about this without python supporting higher kinded types either

trim tangle
#

Well... a lie to the type checker is compromising type safety

#

what's the difference?

undone saffron
#

discord.py isn't lying here and not compromising type safety, the person who specified one bot but attached to another is the one who compromised type safety by doing that

#

and there is no way for discord.py to do more here without higherkinded types.

trim tangle
undone saffron
#

That's not how that works at all

#

The potential to lie here is entirely on the user end, not on discord.py's end.

If you lie to the type checker, you should expect that things stop making sense.

This exists in a space where in the future discord.py could add a little more type information to prevent users from doing this, but they can continue to lie with a #type ignore, this isn't a type safety issue for discord.py

trim tangle
#

For example, what discord.py could have done is make Cog generic in the type of bot ```py
class MyCog(Cog[MyCustomBot]):
...

undone saffron
#

Doesnt work

trim tangle
#

this would not allow providing a custom subclass of a Context, yes

#

if that's what you meant by requiring HKT

undone saffron
#

If you'd like to see this kind of issue checked for, go argue for HKT addition. I promise the way you are proposing doesn't work without it, and it's been discussed in the discord.py server before

#

you can't have a typevar bound that is itself generic right now

#

so this falls apart in many places without it

trim tangle
#

I think I found the actual solution

image.png
undone saffron
#

Well, when people want type safety and a language isn't providing it...

#

But this isn't discord.py discarding type safety, it's the language not providing the tools to be more safe in a case where users misuse/lie

trim tangle
#

I suppose that is true

#

It's more of a "type assertion" than type unsafety

cosmic cave
#

Is there a way to make sphinx documentation describe type aliases?
Given the following code, I'd like the docs to include two entries, one for the alias, the other for the function

from __future__ import annotations
from typing import TypeAlias
Point: TypeAlias = tuple[float, float]
"""A point in 2d space"""
def scale_point_magnitude(p: Point, mag_factor: float) -> Point: ...
"""scale both x and y components of a point"""

I would like sphinx automodule to create a docs page roughly like this:

Point = Tuple[float, float]
A point in 2d space

scale_point_magnitude(p: Point, mag_factor: float) -> Point
scale both x and y components of a point

rustic gull
#

sphinx-doc/sphinx#10455

mighty lindenBOT
rustic gull
#

is that what you are looking for?

#

or is that the other way arround and i missed it

cosmic cave
#

Almost. But that github issue does not say that the type alias will itself be documented, just that it will be used in the signatures of other docs entries

rustic gull
#

mhm i see i found this too sphinx-doc/sphinx#6518

mighty lindenBOT
cosmic cave
#

That issue seems to omit the same

#

If the docs say foo(p: Point) then Point should be clickable and direct to a description of Point
same as we'd get for a class, protocol, struct, interface, etc in python or any other programming language ecosystem: rustdoc, godoc, typedoc, javadoc, etc

undone saffron
# 
#: This is a Type Alias and equivlaent to ...
Point = ...
#

just nornal sphinx doc stuff for module level definitions, might need 1 of the issues above resolved though

cosmic cave
#

I see, even an empty comment #: triggers it to be included in the output. Using 'undoc-members': True does not
I can use a #: comment above and a proper docstring below, and the docstring will render into html

However, this still doesn't make references to Point clickable links. So I still need to solve that problem

I have tried with and without autodoc_type_aliases set. With it set, references to Point render as non-clickable Point. Without it set, references to Point render as expanded Tuple[float, float]
from __future__ import annotations present/absent does not have an effect

undone saffron
#

hmm, I'm not sure and not in a position to check something about this right now, but there should be a way to do this that works, That said, I don't remember what was done exactly in a project that documents type aliases this way, and I don't have access to either the code or the documentation from this device.

Doesn't look like it's directly supported yet: https://github.com/sphinx-doc/sphinx/issues/7896 and I remember that internal project had more than a few sphinx extensions which were written internally for our needs.

cosmic cave
#

Dang, this is frustrating. sphinx kinda already understands that it's a type alias, since it renders alias of Tuple...
That image is rendered from this source:

#:
FourIntTuple: TypeAlias = Tuple[int, int, int, int]
#:
FourFloatTuple: TypeAlias = Tuple[float, float, float, float]

It just doesn't do hyperlinks from usages to the declaration

I feel like we need to wait for a clean-slate doc generator that pulls its info from pyright or mypy or something like that. But that's a ton of work, hence the piles of hacks that make up sphinx and its ecosystem

image.png
oblique urchin
#

PEP 695 should make this a bit easier

cosmic cave
#

!pep695

rough sluiceBOT
#
PEP 8

PEP 8 is the official style guide for Python. It includes comprehensive guidelines for code formatting, variable naming, and making your code easy to read. Professional Python developers are usually required to follow the guidelines, and will often use code-linters like flake8 to verify that the code they're writing complies with the style guide.

More information:
β€’ PEP 8 document
β€’ Our PEP 8 song! :notes:

cosmic cave
#

!pep 695

rough sluiceBOT
#
**PEP 695 - Type Parameter Syntax**

Link

Status

Accepted

Python-Version

3.12

Created

15-Jun-2022

Type

Standards Track

trim tangle
cosmic cave
#

I've been reading about the current best practices for that. I've found stuff about Point: typing.TypeAlias = Tuple[float, float] and other stuff talking about creating an instance of typing.TypeAliasType? Or typing_extensions.TypeAliasType equivalent?

#

Do you know which is the current recommendation?

oblique urchin
cosmic cave
#

Thanks, am I wrong to think that Point = TypeAliasType('Point', tuple[float, float]) will work?
Here is what I see comparing the two styles of alias, neither does the right thing unfortunately

OldSchoolAlias: TypeAlias = Tuple[Literal['old-school'], int]
"""docstring of old-school type alias"""
NewSchoolAlias = typing_extensions.TypeAliasType('NewSchoolAlias', Tuple[Literal['new-school'], int])
"""docstring of new-school type alias"""
def function_uses_both_aliases(old: OldSchoolAlias, new: NewSchoolAlias):
    """docstring of function using both in its signature"""
image.png
oblique urchin
cosmic cave
brazen jolt
#

I need to decorate a func with lru_cache, but preserve it's call args, the _lru_cache_wrapepr only seems to preserve the return type, but no call args.

image.png
#

I was thinking about doing something like foobar = cast(X, lru_cache(foobar)), I assume I'd have to define the X here myself as a protocol though, since the fucntion has overloads, and that seems really annoying, is there something better?

#

heck, I'd probably be willing to take in a dependency for some kind of typed cache lib if there isn't

#

actually, ig I could just do this: ```py
if TYPE_CHECKING:
def lru_cache(func: T, /) -> T:
...

tranquil summit
#

Hi I need to stash some data that I'm pulling from an API. I do not want to lose the typing information by storing it in JSON, nor do I want to use Pickle since it isn't a stable format. Is there something portable that won't lose typing info for storing nested data structures?

brazen jolt
#

how do you know what typing information is there after pulling it from the API, I'd assume most APIs will just give you JSON back

#

or do you store it in some custom way already, and you just need a way to serialize it?

tranquil summit
#

I'm just working with AWS EC2 inventory data. I just started working on this today. (It's refactoring some old code I wrote where I just flattened the data and stuffed it into googlesheets via an intermediate JSON file.)

Note that he SDK I use is Boto3 and it knows what the types are, so the data structures I get back maintains stuff like dates properly. Google tells me maybe to check out MessagePack as dataformat. (The API calls take 10 mins to run, so I want to dump the data locally for iterating my code.)

brazen jolt
#

oh so the data is currently in received a JSON, and already doesn't contain any typing info, you just want to add some?

#

check out TypedDict

# 
from typing import TypedDict
import json


class MyData(TypedDict):
    username: str
    age: int

data: MyData = json.loads(MY_JSON_STRING)

x = data["age"]  # will be int
y = data["username"]  # will be string
z = data["name"]  # error "name" key doesn't exist in MyData
#

you can then create more complex types, like a type dict with a variable that holds a list of some other typed dict values, etc.

tranquil summit
#

So specifically when I get the returned value from the API, any timestamps in the nested dictionary are automatically parsed to be datetime.datetime types, which is good. When I write these out to JSON I have to explicitly convert to string, which feels like I am losing some information.

e.g. the 'LaunchTime' key has a datetime.datetime value in the native Python dictionary and I'm happy with that. I've closely reviewed and it seems that datetime are the only types that aren't being kept as native JSON types (strings, bools, ints, arrays, objects/dicts.)

I'd prefer a data format to stash the output into that doesn't lose that concept of date, but isn't as ephemeral as Pickle.

oblique urchin
#

you're not going to get any other format that is as stable as JSON and as good at dealing with custom Python types as pickle

brazen jolt
#

you'd probably need to write your own format to do that, but you can always just convert from a string into datetime if you need to

tranquil summit
#

Currently my exploration is leading to write it out to https://en.wikipedia.org/wiki/MessagePack

Specifically I'm worried that if I upgrade Python I won't be able to read my data.

MessagePack

MessagePack is a computer data interchange format. It is a binary form for representing simple data structures like arrays and associative arrays. MessagePack aims to be as compact and simple as possible. The official implementation is available in a variety of languages such as C, C++, C#, D, Erlang, Go, Haskell, Java, JavaScript (NodeJS), Lua,...

oblique urchin
#

(with some limitations because of the str/bytes change)

brazen jolt
#

the only thing to be aware with pickle is that it can potentially be unsafe to load arbitrary pickles you don't trust

tranquil summit
#

I think I internalized some wrong facts about Pickle. I'm only coding in Python, so I can use Pickle for my own storage. Is blosc2 the favored library for compressing pickle files?

oblique urchin
regal summit
#

why does it think this would be Never?

image.png
#

its thinking .name is Never too, but default is an Enum

#

works in production as expected

soft matrix
#

What's the type of default in the parameters?

#

Or locals?

trim tangle
#

And what is Enum?

low escarp
#

Question, how do I correctly type hint a function which returns a type which is an argument?
So I have a function like this:

def map_struct(offset: int, type_: Type[ctypes.Structure]) -> ctypes.Structure:
    """ Return an instance of the `type_` struct provided which shares memory
    with the provided offset.
    Note that the amount of memory to read is automatically determined by the
    size of the struct provided.

    Parameters
    ----------
    offset:
        The memory address to start reading the struct from.
    type_:
        The type of the ctypes.Structure to be loaded at this location.
    """
    return type_.from_buffer(get_memview(offset, type_))

Not sure if this is even right to begin with for the types, but essentially I am passing in the type as the second argument, and the result will be an instance of that type.
vscode is just showing me the type as this

image.png
#

If possible I'd like the type to be nms_structs.cGcPlanet so that I can then get autocomplete on the properties and such I have defined on it, thanks!

tranquil turtle
#

Use typevars

low escarp
#

ok, thanks

#

I swear I tried that

#

smh

rare scarab
#

yup. ```py
T_Structure = TypeVar("T_Structure", bound=ctypes.Structure)

def map_structure(offset: int, type_: type[T_Structure]) -> T_Structure:
...

or in 3.12 generic syntax

def map_structure[T: ctypes.Structure](offset: int, type_: type[T]) -> T:
...

regal summit
#

Not at my pc rn, but default can be anything

hardy linden
#

(Use-case: I set up a type hint in some code I was working on, and got a type-checking error. I've done some research, and I now "soft understand" why my type hint was wrong, but Jelle's answer here is spurring me to try to learn type theory properly (at least up through variance) so that I'm not just copy-pasting StackOverflow suggestions that say to "use a TypeVar and set covariance = True" or something.)

I'm having some trouble understanding this example, from https://peps.python.org/pep-0483/#type-variables:

Y = TypeVar('Y', t1, t2, ...). Ditto, constrained to t1, etc. Behaves similar to Union[t1, t2, ...]. A constrained type variable ranges only over constrains t1, etc. exactly; subclasses of the constrains are replaced by the most-derived base class among t1, etc. Examples:

  • Function type annotation with a constrained type variable:
AnyStr = TypeVar('AnyStr', str, bytes)

def longest(first: AnyStr, second: AnyStr) -> AnyStr:
    return first if len(first) >= len(second) else second

result = longest('a', 'abc')  # The inferred type for result is str

result = longest('a', b'abc')  # Fails static type check

In this example, both arguments to longest() must have the same type (str or bytes), and moreover, even if the arguments are instances of a common str subclass, the return type is still str, not that subclass (see next example).

What I'm trying to understand is: what about TypeVar('AnyStr', str, bytes') is making it so that first and second have to have the same type? Is that just something that happens when you have multiple parameters using the same TypeVar, or is something being derived about str and bytes?

I've even written the stub code myself, where I write it once with a TypeVar and once with a Union, and I can see that the Union is tolerated while the TypeVar isn't, but I don't know why.

brisk hedge
#

It's what happens with all uses of typevar

#

in each "definition" (function, class, ...) that one is used, the same typevar represents the same type

oblique urchin
hardy linden
oblique urchin
#

Note that your example is specifically a constrained TypeVar, which is a pretty weird beast that doesn't exist in a lot of other languages. Constrained TypeVars can only be solved to exactly one of their constraints

#

Normally bound TypeVars behave more intuitively

brisk hedge
#

yep, in addition to both types having to be the same, they have to both be the same and one of str/bytes

hardy linden
#

So it's just a mechanic about how TypeVars work, where, when the type checker encounters a TypeVar, it'll look at the types of the values being fed to it, and the type of the first parameter will "bind" the type of that TypeVar for the remainder of that particular function call?

oblique urchin
#

it's more that for each call, the type checker has to "solve" the TypeVar by setting it to some particular type

hardy linden
#

Okay, I see, yeah - to steal from the next example (which defines class MyStr(str): ...), I defined that in my stub code and I can see that result = longest_with_typevars("a", MyStr("abc")) is evaluated out to str because it's not allowed to be more specific than that; it has to "flatten" the subclass back down to one of its constraints?

oblique urchin
hardy linden
#

Thank you, that makes sense, and I really appreciate the patience for elementary questions.

noble condor
#

i suppose, there is just isnt a better way to type hint same thing, right?

image.png
soft matrix
#

not currently no

noble condor
#

mhm ._.

#

well

#

then just screw type hinting :b

trim tangle
cosmic cave
#

Are there good sphinx conventions for documenting python's special methods, given that users don't actually write any dunders when using them?
For example, if a class Zoo implements __len__ this means users will want to len(zoo), not Zoo.__len__()
Yet sphinx will, by default, show Zoo.__len__() in the docs

not strictly a type-hinting question, but I think this is the best place to ask

soft matrix
cosmic cave
#

so it suggests you might have to sprite_list.len(self) which is a bit weird
Maybe I'm overthinking

soft matrix
#

at least that way its possible to differentiate between having a .len method and the special len method?

cosmic cave
#

Are you saying that is a reason to not use prettyspecialmethods? I'm inclined to agree.
Because you're correct, prettyspecialmethods' current behavior does not adequately differentiate between .len method and special __len__ method

soft matrix
#

no it does, a custom method called len wouldnt show the self parameter whereas len does

cosmic cave
#

That's true, it just looks weird. A user will never type len(self) , it'll be len(my_sprite_list) or similar.
And they'll get an error if they do my_sprite_list.len(self) or my_sprite_list.len(my_sprite_list)

I'm overthinking it.

I guess if it's a really beginner-focused library, you can always explain in the docstring.
Say something like "len(sprite_list) will return the number of sprites" or something.

fossil sorrel
#

without of course ruining the python conventions

#

things like reportUnusedCallResult suggests me to use call results even if they are with _

noble condor
#

uhhh

#

is recursive type hint a thing?

tranquil turtle
#

in recent versions of some typecheckers - yes

noble condor
#

i meant

#

how

#

like

#

there is thisdict[dict | str]

#

but

tranquil turtle
# 
type JSON = list[JSON] | dict[str, JSON] | int | float | bool | None
noble condor
#

waqit

#

i just realised

tranquil turtle
noble condor
#

yeah ok

#

here is this: dict[str, str | dict]

#

how do i make dict refer to itself

#

like

tranquil turtle
#

also not recursive

noble condor
#

ah

#

ok gimmi a moment

tranquil turtle
# 
type X = dict[str, str | X]
#

if you want to make it recursive

noble condor
tranquil turtle
#

also in older versions you can use TypeAlias: ```py
X: TypeAlias = dict[str, 'str | X'] # i believe it is correct

noble condor
noble condor
#

python 3.12 is not a thing, yet

noble condor
noble condor
tranquil turtle
#

yes

noble condor
#

so then

#

just tell current version?

#

anyways

noble condor
#

i think

noble condor
#

why did you put a string there..?

#

hu

tranquil turtle
#

because at this moment X is not defined

noble condor
#

but then wont it interpret that as Literal?

tranquil turtle
#

no

noble condor
#

huh

tranquil turtle
#

Literal[...] is string literal

noble condor
#

iirc in recent versions you dont need Literal

#

anyways

#

it seems it did da job?

tranquil turtle
#

!pep 586

rough sluiceBOT
#
**PEP 586 - Literal Types**

Link

Status

Accepted

Python-Version

3.8

Created

14-Mar-2019

Type

Standards Track

noble condor
#

hm

#

how well does pycharm handle that tho

#

cuz it doesnt seem it likes it

#

function signature expects _TT

#

but that dict (i am passing it) has an int

#

which is not even mentioned in _TT

tranquil turtle
#

str | int | _TT

noble condor
tranquil turtle
# noble condor what

Use that instead of str | _TT
int cannot magically appear in your type if you don't mention it at all

noble condor
#

you dont get it

#

pycharm thinks

#

that dict, which has int in it

#

is a valid _TT (which is dict[str, str | _TT])

tranquil turtle
#

Pycharm is stupid, dont use it for typechecking

#

Use mypy or pyright

noble condor
#

tbh

noble condor
#

wait

#

gimmi a sec

tranquil turtle
#

Or every other editor

noble condor
#

cuz vscode's suggestions are crap

soft matrix
#

how so?

#

ive found them to be better than pycharms because they actually respect type annotations

noble condor
soft matrix
#

when did you last try it?

noble condor
#

uh

#

few months ago

soft matrix
#

thats weird

#

what lsp do you use?

noble condor
#

the default one?

soft matrix
#

well beats me then

noble condor
#

wdym

frigid jolt
#

hi, how can i overload this?

image.png image.png
#

if convert_to_obj is false then it'll return a list of ids, otherwise a list of Story objects

soft matrix
#

you probably shouldnt have this as one function tbh

#

but the return should be list[Snowflake] | list[Story]

#

i think you should have a raw version which is called by the other

frigid jolt
#

alright

#

thank

#

s

rare scarab
#

Your overloads have the same signature (default values are ignored)

#

you should do ```py
@overload
async def fetch_new_stories(
self,
*,
convert_to_obj: Literal[True],
should_cache: bool = ...
) -> list[Story]: ...

@overload
async def fetch_new_stories(
self,
*,
convert_to_obj: bool = ...,
should_cache: bool = ...
) -> list[Snowflake]: ...

async def fetch_new_stories(
self,
*,
convert_to_obj: bool = False,
should_cache: bool = False
) -> list[Snowflake] | list[Story]:
pass

frigid jolt
#

thanks, tho i reworked it to avoid this

carmine phoenix
#

is there a function that explores the type of a list/dict and will return the type with all the types that it contains, i.e. list[list[int | str] | float]?

tranquil turtle
rough sluiceBOT
#
typing.get\_args 

typing.get_args(tp)```
Get type arguments with all substitutions performed: for a typing object of the form `X[Y, Z, ...]` return `(Y, Z, ...)`.

If `X` is a union or [`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal) contained in another generic type, the order of `(Y, Z, ...)` may be different from the order of the original arguments `[Y, Z, ...]` due to type caching. Return `()` for unsupported objects.

Examples:

```py
assert get_args(int) == ()
assert get_args(Dict[int, str]) == (int, str)
assert get_args(Union[int, str]) == (int, str)
```   New in version 3.8.
tranquil turtle
#

also read about typing.get_origin

rare scarab
#

I think it's more like {"x": [0, 2, "3", [0]]} => dict[str, list[int | str | list[int]]]

tranquil turtle
#

that is not possible
if you have [Dog(), Dog()], what is this? list of dogs? list of animals? list of objects? list of any? maybe it is not a list at all, but just an object or Any

rare scarab
#

Don't support non literal types

fierce ridge
rare scarab
#

If only reveal_type() was useful at runtime

tranquil turtle
#

jedi can extract type information at runtime from some namespace and create competions according to that

tranquil turtle
rare scarab
#

Now give it a list

tranquil turtle
# 
>>> t.reveal_type([])
Runtime type is 'list'
[]
>>> t.reveal_type(())
Runtime type is 'tuple'
()
``` yeah, this of course doesnt work
carmine phoenix
#

I'm sure I can write a recusive function that does something similar, but does it exist already?

rare scarab
#

Nothing in stdlib, but you could write a recursive function that handles it

carmine phoenix
rare scarab
#

It gets worse when you consider that Dog could be generic

flat slate
#

True

tranquil turtle
#

you cant handle literals, callables, protocols, typeddicts, ...

rare scarab
#

You may be able to handle callable via inspect

supple hinge
eager vessel
#

You should be able to find compatiable version on GH or PyPi

#

PyPi says 3.7+

#

This may be an issue with your model that you're trying to validate? πŸ€”

supple hinge
#

I'm using pydantic v2, and while the installation works, a specific test panics only on 3.8 and 3.9

eager vessel
#

Also if you want to validate just a model you could use Model.model_validate

#

I think you're using | in your model

#

It wasn't a thing before 3.10

#

T | None -> Optional[None]

#

If you want to support 3.8 and 3.9

supple hinge
#

I thought from __future__ import annotations would have fixed that, like for type checking, but I'll try thanks

eager vessel
supple hinge
#

reverting to Union and Optional fixed the test thanks, I guess I'll stick with the | syntax and disable the test for 3.8 and 3.9 when needed

#

Thought pydantic would have somehow pollyfilled the new syntax on old supported versions

eager vessel
#

If you disable 3.8 and 3.9 expect your library to not be compatible with them pithink

supple hinge
#

No, I'm just skipping this test

#

I don't use pydantic at all for the lib in itslef

eager vessel
#

Ah, I see

supple hinge
#

Thanks for your help πŸ‘

strange tree
#

Question about typing functions.
If I want to make a type for a function regarding its args without names, I can use Callable.
In order to type a function with named args and possible kwargs, I need to use Protocol to specify more complex typing of __call__.
Is there a plan to introduce something like

Callable[{"foo": int, "bar": bool, "*args": list, "**kwargs": SomeTypedDict), Any]

?

#

Also I know about Concatenate that one case use with ParamSpec, but would it be possible to "remove" some args/kwargs from such a type? This would be very helpful to write type hints for partially applied functions

heady flicker
#

Nothing else exists or planned as far as I know though

hallow flint
strange tree
#

Thanks πŸ‘

frigid jolt
#

why does this resolve to type[...]

image.png
rare scarab
#

Because that's what a typealias is?

#

Literal accepts *args btw

rustic gull
#

Cant you put it all in one Literal

rare scarab
# 
LoggingLevels: TypeAlias = Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
oblique urchin
#

usually shouldn't affect actual type checking behavior though

frigid jolt
#

yeah I asked because seen like that it could appear that it is a type so a class

i even changed it to literal values because before with
int | str it was displaying type[int] | type[str]

#

I thought for a moment that it was a bug but later I've seen that it wasn't affecting type checking behaviour

empty torrent
#

I have a function signature like this

def wait_for_message(connection_pipe: mp.Pipe, messages: list[Type[T]] | Type[T]) -> T:
#

How do I express that T should also be a subclass of a class Message?

#

if I do

def wait_for_message(connection_pipe: mp.Pipe, messages: list[Type[Message]] | Type[Message]) -> Message:
#

It thinks the type it returns is a Message and not a subclass of it

#

even when I pass a subclass

void panther
#

make the T typevar bound to Message

soft matrix
#

!d typing.TypeVar

rough sluiceBOT
#
typing.TypeVar 

class typing.TypeVar(name, *constraints, bound=None, covariant=False, contravariant=False)```
Type variable.

Usage:

```py
T = TypeVar('T')  # Can be anything
S = TypeVar('S', bound=str)  # Can be any subtype of str
A = TypeVar('A', str, bytes)  # Must be exactly str or bytes
```  Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function and type alias definitions. See [`Generic`](https://docs.python.org/3/library/typing.html#typing.Generic) for more information on generic types. Generic functions work as follows:
soft matrix
#

you need to use the bound

#

oops i was too slow

empty torrent
#

perfect thanks!

viscid spire
#

type[...] is for classes themselves, vs. instances of classes

#

Seems weird that a messages parameter would take a list of classes

buoyant ridge
#

I want to create a function that can subclass a simple class with annotated fields. Is it possible to get the annotation keys to use for static type checking?

class MyObject:
    foo: int
    bar: str

T = TypeVar("T", bound=object)
def subclasser(cls: type[T], *fields: keyof T.__annotations__):
    return type(
        f"{cls.__name__}Subclass",
        (cls,),
        {
            "__annotations__": {
                k: v for k, v in cls.__annotations__.items() if k in fields
            }
        },
    )

MySubclassedObject = subclasser(MyObject, 'foo')
#

My guess is it isn't - something to do with runtime checking of a type var doesn't make sense. But why is this possible (and easy) in TS..?

#

Would it ever be possible in python? I'm confusing myself with this

soft matrix
#

no this isnt possible

#

just use normal inheritance

buoyant ridge
#

how would i remove some class variables using inheritance?

#

I think doing this is actually like type blasphemy, because surely then it's not a subclass if some class vars are removed?

brisk hedge
#

it's not doable without breaking the subtyping property of subclasses

buoyant ridge
#

Can you expand on what that means a bit more? I'm thinking like... something that inherits from a class is a subclass, but doesn't it actually have to be more generic to work where its parent class is expected..? So it's not a subset of its parent class, which is actually what I want?

#

I think I have a lot of fundamentals confused... I don't actually study CS, lol, apologies

simple field
#

if i have a function like

def patched_type(cls: Type[SomeClass]):
  ...

how can I annotate the return type to say it's a subtype of cls?

buoyant ridge
#

returning the same type -> Type[SomeClass] would work since the subtype will substitute in. i dont know of any way to say that it has to inherit, though

simple field
#

is this ok?

T = TypeVar('T', bound='Type[K]')

def patched_type(cls: T) -> Type[T]:
  ...
soft matrix
#

nope bound should be SomeClass

simple field
#

oops i mean, bound=

#

'Type[K]'

soft matrix
soft matrix
#

unless youre actually returning the metaclass of K for some reason

#

(get rid of the Type[T] in the return then its probably correct)

simple field
#

Type[Arg] means the class object Arg or any of its subclasses. not its metaclass.

soft matrix
#

yes but your bound is already saying it takes the class object

#

so taking the type of the class object is the metclass

simple field
#

if Type[Arg] means class object Arg or any of its subtypes then wouldnt Type[Type[Arg]] still mean the same thing?

soft matrix
#

no