#type-hinting

1 messages · Page 36 of 1

trim tangle
#

ConfigT doesn't have any meaning in that context (because the class isn't generic), which is why mypy is complaining

#

As for this:

    @classmethod
    def config_class(cls) -> type[ConfigT]:
        return BaseConfig
``` mypy complains because this method promises to return some specific subclass of `BaseConfig` depending on the typevar, but it actually returns a plain `BaseConfig`. So if you forgot to override the method in `Foo`, the type signature would be a lie.
#

This method should probably not have an implementation in Interface, i.e. it should be an abstract classmethod

cosmic plinth
#

@trim tangle thanks for explaining! is there a way for the config property to be the type defined by whatever a class defines as the return type of config_class? I also want to allow subclasses the ability to not override that and default to the base class config class

trim tangle
#

I'm not sure what you're making, but it looks like a complicated wrapper around global variables 🙂

#

(I'm assuming self.config_class()() is going to read some sort of environment variables or files)

cosmic plinth
#

I also tried a class level _config_class: type[ConfigT] = ... instead of the method but it gave me an error about being unbound

hasty turret
#

Anyone has some stats / surveys about typing usage in python ? Not sure if this is the right place to ask..

restive rapids
# hasty turret Anyone has some stats / surveys about typing usage in python ? Not sure if this ...

uh, the last thing was https://discuss.python.org/t/2024-python-typing-survey-analysis/61456 iirc

Discussions on Python.org
2024 Python Typing Survey Analysis

2024 Python Typing Survey Analysis Overview With help from the Pylance team at Microsoft and PyCharm at JetBrains, we’ve run a survey on typed Python since late July. I wanted to give an update with two weeks of data. Overall, I am thrilled at the number of responses so far. Here are some key stats: 730 responses as of August 17th, the date o...

hasty turret
terse sky
#

91% of respondents saying they use types “Always” or “Often”

wow

#

actually I guess the survey is likely to be quite biased

#

so less wow

oblique urchin
#

There is also a Python community survey that JetBrains and the PSF run every year, right? That might also have info

oblique urchin
terse sky
#

maybe I'm blind but I see zero questions about type annotations here, or type checkers

#

pretty surprising

#

you'd think especially an IDE survey would care a lot about that

iron vapor
#

Not sure what I've done wrong...

    def process_bind_param(self, value: type[BaseModel] | None, dialect):
        if value is not None:
            value = value.model_dump_json()

pylance says value.model_dump_json() has an error: Argument missing for parameter "self"

#

type[BaseModel] means any child of BaseModel, right? So then in the last line there, value should have model_dump_json as a method? I can't figure out the self note.

keen flicker
iron vapor
terse sky
#

when I have overloads, and optional arguments, it feels like I basically need to repeat the default, or at least repeat some kind of value over and over?

oblique urchin
terse sky
#

annoying

#

I guess the value itself isn't used?

oblique urchin
#

correct. you could just write ... as the default in the overloads too

terse sky
#

and type checkers will respect ... specifically, but not e.g. None?

#

I guess that's reasonable

oblique urchin
terse sky
keen pawn
#

is there a more concise way of defining a contravariant Mapping than this? or is there something else that works the same way? ```py
from typing import TypeVar

DictA = dict[str | int, bool]
Var = TypeVar("Var", bound=str|int, contravariant=True)
DictB = dict[Var, bool]

def works(data: DictB) -> None:
...

def errors(data: DictA) -> None:
...

data = {"x": True}
works(data)
errors(data)

-# real Issue for context: <https://github.com/pola-rs/polars/issues/14468>, <https://github.com/pola-rs/polars/pull/19499/files>
oblique urchin
#

To make a contravariant mapping you'd likely have to create your own protocol with the methods you care about.

keen pawn
summer sable
#

I have a library (sqlitedict) that implements a MutableMapping[any, any] and doesn't allow generics. How do I provide type annotations to an instance of it?

#

I want to override relevant methods with MutableMapping[KeyType, ValueType] while not dropping the relevant methods like commit()

#

this does not seemed to work

image.png
oblique urchin
summer sable
#

I cannot type annotate it by saying SqliteDict[Key, Value]

oblique urchin
summer sable
#

Ok, I will take a look

#

thanks!

summer sable
#

ah, i think my version of pylint does not support bracket style generics

#

that was the major source of my confusion

summer sable
#

sigh

#

I am not sure if i want the mix-in side effects from subclassing collections.abc.MutableMapping

#

guess I will just do this?

image.png
spice iris
#

Why does this type-hinting not achieve the desired result? (Tuples should be all the same type but that type can be one of several):

from typing import Tuple, TypeVar
from typeguard import typechecked

InfoVal = TypeVar("InfoVal", int, float, str, bool)

@typechecked
def foo(data: Tuple[InfoVal, ...]):
    print("OK:", data)


for data in (
    (1, 2, 3),  # ok
    (1.0, 2.0, 3.0),  # ok
    ("1", "2", "3"),  # ok
    (True, False, True),  # ok
    ("1", 2, False),  # TypeError
):
    try:
        foo(data)
    except TypeError:
        print("TypeError:", data)

OK: (1, 2, 3)
OK: (1.0, 2.0, 3.0)
OK: ('1', '2', '3')
OK: (True, False, True)
OK: ('1', 2, False)
oblique urchin
spice iris
#

Not sure I understand why that means it allows mixed types (the last test case) to not error?
(I copied the multiple types in TypeVar from the docs -> A = TypeVar('A', str, bytes) # Must be exactly str or bytes)

oblique urchin
#

I guess typeguard may not implement TypeVars with constraints correctly.

spice iris
#

FWIW MyPy also doesn't seem perturbed. Is there a better way to type a tuple like this?

Pyright seems to get this right:

"tuple[str, int, bool]" is not assignable to "Tuple[str, str, str]"
        Tuple entry 2 is incorrect type
          "int" is not assignable to "str" (reportArgumentType)
oblique urchin
#

Generally I would recommend against using constrained TypeVars though, they behave oddly and unintuitively. In this case you're probably better off with a union, like tuple[str, ...] | tuple[int, ...]

spice iris
#

The mypy error seems unintuitive to me in the linked case

oblique urchin
spice iris
#

Using a union of the different tuple-types seems to produce better results, but I find it a shame the TypeVar solution does not work given how unwieldy the union solution becomes

#

they behave oddly and unintuitively
Do you have any reading on this?

gloomy valley
#

Hey so I was wondering sometimes in built-in methods a parameter is for example from the type SupportsIndex instead of int or LiteralString instead of just str. What does that mean?

restive rapids
# gloomy valley Hey so I was wondering sometimes in built-in methods a parameter is for example ...

Supports{something} usually means a protocol with a __{something}__ method
__index__ is the dunder invoked when something is used as an index in like, a list
LiteralString is well, a string literal. e.g. input() would not be returning a LiteralString, but "hi" is a LiteralString (and strings you make from other literal strings are literal strings too). you could see that being used for e.g. an sql query function: you dont really want to pass arbitrary strings to it

gloomy valley
#

what does __index__ do?

restive rapids
#

it returns an int representing what the object should be as an index
like, for a list xs, xs[idx] would use type(idx).__index__(idx) as the actual index. for ints, __index__ is just a return self

gloomy valley
#

okay im not sure if i completely get it

#

but thanks

restive rapids
#

its basically implicit type casting to be able to use multiple things as an index

gloomy valley
#

SupportsIndex or int?

restive rapids
#

it would be whatever the type of idx is, could be int, could be Zero

class Zero:
  def __index__(self) -> int:
    return 0

SupportsIndex is a protocol, there are no actual instances of it
i did not write idx.__index__() because operations are implemented on types, not instances. if it makes more sense to you, its idx.__index__(), but not really (if you do idx.__index__ = lambda: 0 for some idx, it wont actually become usable as an index)

gloomy valley
#

so you mean SupportsIndex is only for type hinting

restive rapids
#

well, it is also runtime checkable with isinstance, but its basically hasattr(type(idx), "__index__")
but yes, its used to express "some type that has __index__(self) -> int defined"

hasty phoenix
#

Is there a way to tell type checkers to tell which attributes are valid and not valid when implementing __getattr__() ?

#

I'd like to explicitly tell which attributes exists and their type hints and give typing errors on the other. It works using @overload to specify a function prototype for an attribute which is handled by __getattr__() and likewise for a regular attributes, e.g. key: int. But even when __getattr__()raises AttributeError for its unknown attributes, the type checker seems to assume a type for every attribute it finds and that default behavior I'm looking for.

cinder bone
#

Although I do question __getattr__ if you only have a limited set of attributes you can access

hasty phoenix
#

I'd wish one could define that undeclared attributes should err, not attempt to interpret __getattr__ (because it doesn't do that too well).

stable fjord
grave fjord
#

Or a Literal["attr", "other_attr"]

hasty phoenix
# grave fjord I think you can use @overload

@overload helps on defining the methods so the type checker will find the signature, but it turns out it doesn't work since the method will be registered as an attribute and __getattr__ will never be called. The TYPE_CHECKING approach seems to work thou

hasty phoenix
#

It turns out to require a lot of sugar for typing either way if the intent is to implement two classes that contains the same methods, but with "reduced" set arguments in one of the classes.

hasty phoenix
#

I thought it was a good idea to use Protocol to declare the functions, but then __getattr__() isn't called for the methods declared in the protocol when used withclass A(MyProtocol):...

harsh lantern
#

how to represent variadic argument in callable type hint?

jolly cipher
#

we'd need more details regarding your problem though

harsh lantern
#

i want the type that accepts any number or arguments as long as they are all of a some type (like int for example) and the function returns that type
sof(x: int) -> int f(x: int, y: int) -> int f(x: int, y: int, *args: int) -> intare accepted, but not```
f(x: int, y: str) -> bool

terse sky
#

out of curiosity, why not accept a list?

#

(or Sequence, probably better)

# 
_T = TypeVar("_T")
def f(x: Sequence[_T]) -> _T:
    ...
harsh lantern
#

idk

#

i think use Protocol?

terse sky
#

I'm not sure I understand

tacit ridge
#

Hey, i'm trying to create overloads for a number of functions that have a parameter which defines wether the function should be sync or async. I want the overloads to show that the function can either return T or Coroutine[Any, Any, T] based on the boolean value of the use_async parameter

Some of these functions have very long parameter lists, so i would rather not write all of the overloads manually since the other parameters have no effect on the return type

Here is what it looks like manually

def example_func(... (many kwargs), use_async: Literal[False]) -> str: ...

@overload
def example_func(... (many kwargs), use_async: Literal[True]) -> Coroutine[Any, Any, str]: ...

def example_func(... (many kwargs), use_async: bool = False) -> str | Coroutine[Any, Any, str]:
   ...```


Since I have so many functions like this, it would be very frustrating creating all of the overloads manually (not to mention a maintenance nightmare)

I have been trying to come up with a way to programmatically mark a function and generate the overload data for it.

I've tried something like this but mypy/vscode intellisense does not recognise it

```python
def asyncify_type(func):
    @wraps(func)  # Preserve metadata
    def _sync(): ...

    _sync.__annotations__["use_async"] = Literal[False]
    overload(_sync)


    @wraps(func)
    def _async(): ...

    _async.__annotations__["use_async"] = Literal[True]
    _async.__annotations__["return"] = Coroutine[Any, Any, func.__annotations__["return"]]
    overload(_async)
    return func

@asyncify_type
def example_func(input: int, use_async: bool) -> str:
    pass
#

Anybody know if something like this is possible?

grave fjord
tacit ridge
#

One of my goals is to provide access to both the sync/async versions of the function through a single interface. I already have the implementation done, i'm just trying to add some nice type hints to make it very clear what the return type will be

tacit ridge
terse sky
#

the idea I had was to use protocols with an associated type, but I'm not sure if python supports that

grave fjord
pastel egret
#

Callable[[*tuple[int, ...]], object] would also work.

restive rapids
# pastel egret `Callable[[*tuple[int, ...]], object]` would also work.

i dont think thats what they want, and i dont think there's even a way to express what they want
this only works for a function which is defined as taking varargs, they want it to handle a function that has any amount of int args, e.g.

def f(x: int) -> int:
  ...
def g(x: int, y: int) -> int:
  ...

not just

def h(*xs: int) -> int:
  ...

though im not sure what would one even do with such a type, how would you know how many ints to give to it to call?

grave fjord
grave fjord
harsh lantern
#

I ended up just using Callable[..., int]

restive rapids
grave fjord
grave fjord
harsh lantern
#

I don't have access to source code rn

harsh lantern
#

actually nvm it

buoyant swift
#

though it's not its own question

image.png
#

and you can't see the results yet..

hasty turret
rose heath
#

Hello, how usable is the "returns" library? Do people generally recommend using it or to stay away from it?

rose heath
#

So here's something I'm trying, would appreciate tips on if it's feasible

import dataclasses
import inspect
import sys
from typing import Generic, TypeVar

T = TypeVar('T')

@dataclasses.dataclass
class Options(Generic[T]):
  @property
  def class_type(self) -> type[T]:
    return getattr(sys.modules[__name__], self.__class__.__qualname__.replace('.Options', ''))

  def make_instance(self, *args, **kwargs) -> T:
    return self.class_type(self, *args, **kwargs)  # Expected 0 positional arguments

class A:
  @dataclasses.dataclass
  class Options(Options):
    a: int

  def __init__(self, opts):
    self.opts = opts

class AA(A):
  @dataclasses.dataclass
  class Options(A.Options):
    b: int

  x: int

  def __init__(self, opts, x: int):
    super().__init__(opts)
    self.x = x

a = A.Options(a=42).make_instance()
print(inspect.signature(AA.Options(a=42, b=43).make_instance))
aa = AA.Options(a=42, b=43).make_instance(x=44)

want:

  • autocompletion for make_instance based on the constructor of the class (something with Concatenate and ParamSpec?)
  • the return value for make_instance should be correctly deduced statically so autocompletion continues to work

is something like this possible? in this example class_type is some dynamic attr just for illustration purposes but maybe there's a way to automatically fill in class_type such that it propagates through static type checking idk maybe by having a metaclass do something about it?

harsh lantern
#

what is the difference between CalcNumber* and CalcNumber

rare scarab
#

That looks weird. What's the declared type?

harsh lantern
#

this```py
def mul[T: (CalcNumber, CalcVector)](self, other: T) -> T:
if isinstance(other, CalcVector):
return other * self
# assert_type(other, CalcNumber)
a = self.number
b = other.number
if not (_is_rational(a) and _is_rational(b)):
a = _convert_fraction_to_decimal(a)
b = _convert_fraction_to_decimal(b)
res = a * b
if _is_integer(res):
res = int(res)
return CalcNumber(res)

oblique urchin
# harsh lantern this```py def __mul__[T: (CalcNumber, CalcVector)](self, other: T) -> T: ...

Oh, I think pyright's technique for evaluating TypeVars with constraints leaks out here (known as conditional types, https://github.com/microsoft/pyright/blob/main/docs/type-concepts-advanced.md#conditional-types-and-type-variables)

GitHub
pyright/docs/type-concepts-advanced.md at main · microsoft/pyright

Static Type Checker for Python. Contribute to microsoft/pyright development by creating an account on GitHub.

cinder bone
#

Is there a backport of PEP 702 for python 3.9 or 3.10?

rare scarab
#

works on master

oblique urchin
#

PEP 702 support is still being worked on in mypy

rare scarab
#

python/mypy#16111

mighty lindenBOT
rare scarab
#

not check in python/mypy#17264 likely because it isn't released yet

mighty lindenBOT
rare scarab
cunning plover
#

Feeling mad but very satisfied 😇 Managed to write yet another framework for work.
This one autogenerates openapi documentation by importing and parsing our Django code 😋
Polishing last bits and at last submitting PRs for it

The resulting documentation is far more precise than human efforts can achieve.
Everyone was very lazy to fill all fields for nested serializers and django filters manually
All the nested madness of all fields autogenerated

Autogenerated code shows autoparsed DRF Serializers for input, django filters, other fields, and necessary security permissions to execute specific action

Screenshot_20241101_165433.png
#

Kind of high end typing stuff for web python

#

in code overrides are possible to define for desired serializers, queryfilters, actions to enhance built docs ^_^ Overrides are declarable by Pydantic data structs, so you know in advance what can be overriden

cinder bone
terse sky
#

Question about mypy and bytes and memoryview, cough Jelle

#

I see mypy decided to consider bytearray a virtual subclass (or something like that), of bytes, that seems reasonable to me, at least things like this work

def whut(v: str | bytes | bytearray) -> str:
    result = v.decode("utf-8") if not isinstance(v, str) else v
    te.reveal_type(result)
    return result
oblique urchin
#

!pep 688

rough sluiceBOT
terse sky
#

hah, okay...

#

so you would basically recommend writing v: str | bytes above

#

and not depending on this

oblique urchin
terse sky
#

ah right, actually, that's fine

#

but why is memoryview a subclass of bytes if it doesn't support decode, that's the concerning part I guess

#

like, in mypy this code would type check with a bytearray argument, even if bytearray is not in the union

#

that's relatively okay since at least decode works either way

#

but it will also accept memoryview, even though memoryview doesn't have decode

oblique urchin
#

mypy currently accepts both bytearray and memoryview for bytes. PEP 688 says bytes should mean just bytes

#

I agree that among the two memoryview is more sketchy than bytearray

hallow flint
harsh lantern
oblique urchin
idle kindle
#

hello

#

@jelle: hello

grave fjord
idle kindle
#

yes

#

could you explain about type-hinting briefly?

cinder bone
idle kindle
#

thanks

#

I have been working as a React developer for 10 years until now.

#

my friend gave me a task about python framework

viscid spire
idle kindle
#

I use Both of them

#

I mainly used JavaScript in projects

viscid spire
#

i c

#

well on the surface level, python typehinting is much like typescript

#

tho it is still valid Python code

#

we use third-party static type analyzers to verify the types, instead of a compiler

final eagle
#

why doesn't this work?

class Thing:
    pass

ThingCls = TypeVar("ThingCls", bound=type[Thing])

def thing_class_modifier(base: ThingCls = Thing) -> ThingCls:
    return base
#

error I get is: Incompatible default for argument "base" (default has type "type[Thing]", argument has type "ThingCls")

viscid spire
#

you're using mypy?

#

in pylance it's fine

final eagle
#

yea, mypy

trim tangle
#

the pylance behaviour is somewhat surprising here, why is it okay to use Thing as a value for an unknown type ThingCls?

final eagle
#

that would be a default type, as written it's a default value

viscid spire
trim tangle
#

So it seems like the default value changes the type of the function, and you cannot express that type yourself kind of?..

#

I guess it's a way of encoding this overload: ```py
def thing_class_modifier() -> type[Thing]
def thing_class_modifier[T: type[Thing]](base: T) -> T

trim tangle
# final eagle yea, mypy

I think mypy is not wrong here. It's a new inference rule invented by pylance that's not standardized in any PEP

#

You could ask in python/mypy or python/typing on github

final eagle
#

bound wouldn't tell mypy to inference that Thing is acceptable as a default?

idle kindle
#

thanks gxþbxi

#

@viscid spire thanks

final eagle
keen flicker
#

I can't believe I'm asking this here, (mostly because I either don't typehint stuf, or make the absolutely worst conceivable typehints ever)
but, how do you typehint properties?, for example, if I have a type_property class that makes absolutely sure that the property's value is of a type in allowed_types, ¿how do you typehint that? and more so, when to __get__ and __set__ the objects are passed as an argument, so there's two typevars there, this are my type-hinting right now:


class typed_property(property, _T.Generic[A, T]):
    def __init__(self, *allowed_types: _T.Type[T],
                       container_subtype = None,
                       name: str = '', 
                       fget: _T.Callable[[A],    T] | None = None, 
                       fset: _T.Callable[[A, T], T] | None = None
                    ): ...

    def at_set(self, func: _T.Callable[[A, T], T]): ...

    def at_get(self, func: _T.Callable[[A],T]): ...

    def __set__(self, obj: A, value: _T.Any): ...

    def __get__(self, obj: A, objtype: _T.Type[A] | None = None) -> T: ...

and here, A is the TypeVar for the obj this property belongs to, and T is the typevar for the type of this property

#

but Generic[A, T] doesnt seem right, (for no particular reason)

#

(type is forced both before and after the set, that's why the fset function has to return T, and ignore name and container_subtype, the former is for repr if you're fancy, and the latter is internal about the container class this property uses internally)

restive rapids
keen flicker
restive rapids
#

no i mean to not have allowed_types, just a single union type

keen flicker
#

wdym

#

the point of this property is not just type-hinting, but also runtime-checking, and ensure the type at runtime, so I need the allowed_types

#

I have to type-hint it like this

class Example:
  name: typed_property[str, "Example"] = typed_property(str)

for mypy not to complain, is there any other way to avoid this?

restive rapids
#

by creating it from a method, that will infer the class type

keen flicker
restive rapids
#

not sure what you mean by that
and no, you cant use typing.Self here if thats what you're thinking of

keen flicker
#

this won't work if you try self.name = typed_property(self)

restive rapids
#

if you dont provide any getter/setter that would depend on the type of the instance actually being Example - you dont need to have that typevar in the property at all

keen flicker
restive rapids
#

no, you dont use this type at all
just name: typed_property[str]

keen flicker
#

error: "typed_property" expects 2 type arguments, but 1 given [type-arg]

restive rapids
#

yes, i mean to not have it, remove it, it is not used here so you cant do anything with it

keen flicker
#

and if I remove the A typeVar from the generic then there's a bunch other errors

restive rapids
#

what are you even trying to do? why are you validating types at runtime?

keen flicker
#

should I do a help thread or smth? this seems like a big conversation for type-hinting

restive rapids
#

perhaps

keen flicker
#

done

soft matrix
#

ive just found this scary bit of code py class ID(Generic[TypeT], metaclass=abc.ABCMeta): __slots__ = ("id64", "__weakref__") __class_getitem__ = classmethod( GenericAlias ) # want the different behaviour between typing._GenericAlias to do with attribute forwarding

#

and wanted to check if this was intentional

#

i think past me was on about the behaviour difference between forwarding of dunders

oblique urchin
soft matrix
#

yeah i mean same

#

but that will be breaking at least some stuff

#

i think ive found the cause of this comment

IndividualID = ID[Literal[Type.Individual]]

class WrapsUser(User if TYPE_CHECKING or DOCS_BUILDING else BaseUser, Messageable["UserMessage"]):
    """Internal class used for creating a User subclass optimised for memory. Composes the original user and forwards
    all of its attributes.

    Similar concept to discord.py's Member except more generalised for the larger number situations Steam throws at us.
    Slightly different however in that isinstance(SubclassOfWrapsUser(), User) should pass.

    Note
    ----
    This class does not forward ClientUsers attribute's so things like Clan().me.clear_nicks() will fail.

    If DOCS_BUILDING is True then this class behaves like a normal User because we need to be able to access the
    doc-strings reliably and memory usage isn't a concern.
    """

    __slots__ = ("_user", "_cs_channel")

    def __init__(self, state: ConnectionState, user: User | ClientUser):
        IndividualID.__init__(self, user.id64, type=Type.Individual)
```damn i write cursed code
#

but that wouldnt work with the old typing class

oblique urchin
#

the __init__ call on an apparently unrelated class is interesting

soft matrix
#

maybe youd need to have a chat with whoever implemented 585 and ask them if its intentional and the original was only because it was pure python and you have to forward dunders

soft matrix
oblique urchin
gaunt reef
#

Hi, how should I type the following decorator correctly? Both pyright and mypy complain that self is missing from the call to a_method on the last line whereas the code exectues fine.

# 
import functools
from typing import Protocol


class Method[S, R, **P](Protocol):
    def __call__(_, self: S, *args: P.args, **kwargs: P.kwargs) -> R: ...


def decorator[S, R, **P](f: Method[S, R, P]) -> Method[S, R, P]:
    @functools.wraps(f)
    def wrapper(self: S, *args: P.args, **kwargs: P.kwargs) -> R:
        return f(self, *args, **kwargs)

    return wrapper


class SomeClass:
    @decorator
    def a_method(self) -> int: ...


instance = SomeClass()
instance.a_method()
#

The issue is mainly with defining the correct Protocol for methods where I want to type self

rare scarab
#

Don't worry about self. It's part of *args

rare scarab
#

The protocol is unneeded. ```py
def decorator[R, **P](f: Callable[P, R]) -> Callable[P, R]:
@functools.wraps(f)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
return f(*args, **kwargs)

return wrapper
hearty moon
hearty moon
rare scarab
#

You may need to use Concatenate

#

And Unpack

hearty moon
# rare scarab You may need to use Concatenate

That doesn't seem to work in combination with Protocol and __call__. I also checked both the typing spec and the pyright docs. Guidance on typing methods where you want to control what self can be is absent.

tacit sparrow
#

Would it be a PyRight bug that it adds Literal[True] to the type or it's just trying to be clever by deducing that a.y is probably have similar type to the fallback value?

if x := getattr(a, "y", False):
    reveal_type(x) # Type of "x" is "Any | Literal[True]"
stable fjord
#

that seems a bit weird. Open a discussion in the repo and ask if you can't find anything after a brief search

tacit sparrow
ember arrow
#

.

young zealot
#

How can I type hint a function that takes a Callable and then *args, **kwargs, and the Callable takes those args/kwargs as its parameters? Like this example:

def test(callable: Callable[..., None], *args, **kwargs):
    callable(*args, **kwargs)
cinder bone
young zealot
#

Ah ok, I knew it was with ParamSpec somehow but was getting confused

cinder bone
#

two other things to consider:
On python < 3.12, you'll have to do

from typing import ParamSpec
P = ParamSpec("P")

Also consider if you need the function to return None. If any output will suffice, consider using Callable[P, object] instead

young zealot
#

Thanks!

wary pulsar
#

Is there a way to non-exhaustively use Unpack and TypedDict for **kwargs, such that arbitrary keyword arguments are still allowed?

rare scarab
#

There is a pep for that

#

!pep 728

rough sluiceBOT
wary pulsar
viscid spire
#

(So there's not a way yet)

rare scarab
#

Might be in typing_extensions first

oblique urchin
#

Yeah I need to implement the latest version in typing-extensions

errant tulip
#

I'm using the redis-py library, which annotates certain functions like hset with Union[Awaitable[int], int]. Since I'm using the async API, every time I call await client.hset(...), Pyright throws an error saying "int is not awaitable." This happens frequently in my code (10 or more times), making # type: ignore impractical. Is there another way to silence this specific warning?

trim tangle
#

(specifically because they'll need to use # type: ignore on every line)

rough sluiceBOT
#

redis/commands/core.py lines 5016 to 5023

def hset(
    self,
    name: str,
    key: Optional[str] = None,
    value: Optional[str] = None,
    mapping: Optional[dict] = None,
    items: Optional[list] = None,
) -> Union[Awaitable[int], int]:```
trim tangle
#

There isn't a good way to silence this warning, since it's very general. If you turned it off, it would not catch legitimate errors when you're awaiting something that's not awaitable.

#

Actually, pyright just shoves this under reportGeneralTypeIssues

#

You should at least use # pyright: ignore[reportGeneralTypeIssues] instead of a plain # type: ignore, so that other errors like a typoed variable name are still caught

jolly cipher
#

generics without parameters too...
unless default as in pep 696!

trim tangle
#

e.g. User | None

jolly cipher
#

optionals are fine.

#

but things like

#

-> list[str] | str

#

where it could easily be overloaded e.g. with regard to parameter value (aka argument) stream being either True or False

#

those are especially frustrating to work with when they don't have overloads

trim tangle
#

I like to avoid this kind of behaviour in the first place

#

(i.e. maybe it's better to split it into two functions)

#

another thing I don't like is flags to apply some post-processing to the output. Like add(3, 5) == 8 and add(3, 5, stringify=True) == "8"

jolly cipher
#

where iX is something typically giving us an iterator as a result

#

and X an all-in collection

#

for those types of distinctions

jolly cipher
#

in that case i'd at most make sth like add and add_stringified that wraps add

#

given the existence of the second routine would really be necessary

#

one of the things that typing did to me was beginning to rely less on unpredictable things like getattr() or bizarre unions in returned values.

#

i consider that a vast improvement

hoary pagoda
#

let's say I am implementing something like itertools.product, like so

class Product[T]:
    def __init__(self, a: Iterable[T], b: Iterable[T]) -> None:
        self.a, self.b = a, b
    def __iter__(self) -> Self:
        cache = []
        for a in self.a:
            if cache:
                yield from cache
            for b in self.b:
                cache.append(b)
                yield b

Now, I would like Product('AB', 'CD')[0] to be 'AC', but Product(iter('AB'), iter('CD'))[0] to be a type error of sorts. Is that possible within the type system? My only idea is to try to somehow runtime type check and produce a different subclass based on whether both arguments are a sequence, but I would prefer to not do excessive runtime type checks.

restive rapids
#

so you want it to fail when b is an iterator, because it will be iterated multiple times?

hoary pagoda
#

ah yea, I did forget to cache in the impl, my bad.

#

the key point is that when given two Sequences, Product should support len and __getitem__, but when given a non-sequence, it should support neither

restive rapids
#

ah ic

# 
from collections.abc import Iterable, Sequence
from typing import no_type_check, overload


class Product[C: Iterable]:
    @overload
    def __new__[A](cls, a: Sequence[A], b: Sequence[A]) -> Product[Sequence[A]]:
        ...
    @overload
    def __new__[A](cls, a: Iterable[A], b: Iterable[A]) -> Product[Iterable[A]]:
        ...
    @no_type_check
    def __new__(cls, a, b):
        ...
    def __len__[A](self: Product[Sequence[A]]) -> int:
        ...
    def __getitem__[A](self: Product[Sequence[A]]) -> tuple[A, A]:
        ...

something like that maybe
not sure if typing new even works lmao

#

seems to work actually

hoary pagoda
#

oh neat, you can type self further

#

thanks

restive rapids
#

maybe having Product[A] = IterProduct[A] | SeqProduct[A] would be less voodoo
and a function product with the overloads to return the correct one, by checking if len exists, perhaps

hoary pagoda
#

that's sort of what I want to avoid

#

I don't really want to have a bunch of runtime type checks just to satisfy the static type checks

hoary pagoda
#

Oh god I have to make this heterogenous as well

hoary pagoda
#

Is there a type like Sequence but without .count and .index? Just getitem len?

cinder bone
hoary pagoda
#

They aren't exactly meaningful in my case.

cinder bone
#

I mean you could always do

class IndexableSized[T](Protocol):
    def __len__(self) -> int: ...

    def __getitem__(self, idx: int) -> T: ...
#

I don't think there's anything builtin

hollow copper
#

This table shows the progression of builtin types: https://docs.python.org/3/library/collections.abc.html#collections-abstract-base-classes @cinder bone 's suggestion of a Protocol should be good (once you fix the misspelling of __getitem__)

Python documentation
collections.abc — Abstract Base Classes for Containers

Source code: Lib/_collections_abc.py This module provides abstract base classes that can be used to test whether a class provides a particular interface; for example, whether it is hashable or whet...

rustic gull
cinder bone
rough sluiceBOT
rustic gull
weary carbon
#

IS THERE ANYONE THAT CAN CODE GODOT?

im really trying to get into programming but it never wanna do as i want

stone harness
#

Good morning, How can django orm do this?

class User(BaseEntity):
    username = models.CharField(max_length=100)
    avatar_url = models.URLField()
    balance = models.FloatField(default=100)
    booster = models.BooleanField(default=False)
    banned = models.BooleanField(default=False)
    cards = models.ManyToManyField("user.UserCard", blank=True)

    def __str__(self) -> str:
        return self.username


User.username  # CharField[str]
User().username  # str

type hint changes whether the class is instantiated

cinder bone
#

Either they just abuse setattr or CharField is a descriptor

trim tangle
keen flicker
cinder bone
#

pseudocode

field = getattr(self, fieldname)
if isinstance(field, CharField):
    setattr(self, fieldname, some_computed_str)
leaden ember
#

I'm writing a package, and have a type hint along the following:

ErrorOptions = Literal["raise", "warn", "ignore"]

def my_func(option: ErrorOptions):
     ...

How can I surface to my users that the type is of Literal["raise", "warn", "ignore"]? When hovering over it in the IDE it just shows the function signature which isn't really clear to the user what the available options are

trim tangle
#

!cban @river carbon scam

rough sluiceBOT
#

:incoming_envelope: :ok_hand: applied ban to @river carbon permanently.

cunning plover
#

i noticed my vscode shows description of Google Docstrings at least for extra hints right on hover

cunning plover
#

it will accept regular literal strings too 🙂 and comparing them as ErrorOptions.warn == "warn" should equal to true

#

str enums are somewhat dangerous in typing termss though, not recommending them

#

i would more recommend just using regular enum

# 
from enum import Enum, auto

class ErrorOptEnum(Enum)
  raise = auto()
  warn =  auto()
  ignore =  auto()

def my_func(option: ErrorOptEnum):
     ...
#

that will be self documentedly obvious

#

and having better stable interface

viscid spire
#

!e

from enum import Enum, auto

class ErrorOptEnum(Enum):
  raise = auto()
rough sluiceBOT
viscid spire
#

there is the issue with the keyword raise

atomic idol
#

so you cant use it as variable

viscid spire
#

You don't have to ping me for that, I already knew 😉

atomic idol
#

I am having a problem with mypy type checking with awaitable.

# self.connection.accept method
    async def accept(
        self,
        loop: asyncio.AbstractEventLoop,
    ) -> Awaitable[Tuple[socket.socket, Tuple[str, int]]]:
        return await loop.sock_accept(self.sock)

# method using above method
    async def accept(self, loop: asyncio.AbstractEventLoop) -> ConnHandle:
        sock, address = await self.connection.accept(loop) # MYPY error: "Awaitable[tuple[socket, tuple[str, int]]]" object is not iterable [misc]
        logger.info(f"New client: {address}") # MYPY error: Cannot determine type of "address"  [has-type]
        return ConnHandle(sock, address)

I am not able to understand why its not considering the type inside Awaitable why??

stable fjord
#

that looks very strange

#

can you create a standalone minimal repro? What version of mypy?

trim tangle
#

e.g. if you have async def foo(x: int) -> str, then foo(42) is an Awaitable[str]

#

You should try doing typing.reveal_type(self.accept)

#

Honestly I don't like this feature

#

in particular because an async def function with a yield keyword does not have that

soft matrix
#

For the convenience it's nice

#

I hate doing it in ts and having to put Promise after everything

trim tangle
#

inflexibility as in: you can't write an async def function and have its contract be that it returns an Awaitable, not necessarily a Coroutine

#

and it also prevents you from specifying the send and return types on the coroutine

#

I would guess that the main reason this was done was the lack of defaults. Because writing Coroutine[ReturnType, Any, Any] every time would be pretty bad

#

It's the same deal as with *args: Cat, **kwargs: int instead of *args: tuple[Cat, ...], **kwargs: dict[int, str] (TypeScript uses the latter version). It adds a few extra characters, but buys a lot more flexibility: there's no need to add extra constructs to support stuff like *args: tuple[int, str] | tuple[int], **kwargs: MyTypedDict (which required changed to the language/standard library with Unpack)

soft matrix
#

idk isnt a lot of the point that python does make some of these decisions for you

#

i agree it can be annoying but in terms of actual practicality im still firmly on the side of make it easier to do the common one

iron vapor
#

I'd love some help with static methods and typing.Self. I have this:

    @staticmethod
    def get(user_id: Optional[int] = None) -> Self:

But pylance says "Self" is not valid in this context.". Isn't this or classmethods exactly where I'm supposed to use typing.Self?

soft matrix
#

Self makes no sense in a staticmethod because theres no way to refer to the enclosing class in subclasses

viscid spire
#

It would make more sense to have that a classmethod ig?

soft matrix
#

yes

grave geyser
#

How can I define a function that takes two arithmetic types, regardless if they are fundamental types or classes with add etc ?

To simplify, how can I define

def add(a: T, b:T) -> T:
    return a + b

that works for floats, ints and custom classes with __add__ ?

oblique urchin
grave geyser
#

Thats not enough. Fundamental types will not be considered to implement that protocol

oblique urchin
grave geyser
#

Hmm you're right, it does work for floats, but not for my Float2 class. I have

class Arithmetic(Protocol):
    def __add__(self: Self, other: 'Arithmetic') -> 'Arithmetic': ...
    def __sub__(self: Self, other: 'Arithmetic') -> 'Arithmetic': ...
    def __mul__(self: Self, other: 'Arithmetic') -> 'Arithmetic': ...

and I get

 "__add__" is an incompatible type
      No overloaded function matches type "(other: Arithmetic) -> Arithmetic"

for my Float2

def __add__(self, arg0: Union[Float2, Int2, Tuple[float, float]]) -> Float2:
#

I guess its the union that messes things up?

#

but it doesn't seem like it

pastel egret
#

The problem is that your protocol is saying that it's valid to call arith.__add__(another_arith), but Float2 requires a Float2 specifically.

trim tangle
#

Yeah, the protocol specifies that __add__ should work with any other Arithmetic, not just some of them

#

arithmetic operations are kinda complicated to type IIRC

#

because there's also __radd__ and the rules about which method is called in which order

pastel egret
#

I think you'll want to just annotate self, other and the return value with a typevar.

#

Doesn't work for radd though, I think you might need a union of 8 protocols to handle all the combinations?

grave geyser
#

Regardless of how I defined the protocol I cant get it to match my custom Float2 ...

# 
class Arithmetic(Protocol):
    def __add__(self: Self, other: object) -> Self: ...

class Float2:
    def __add__(self, other: 'Float2') -> 'Float2':
        return self

def dup(a: Arithmetic) -> Arithmetic:
    return a + a

dup(Float2())
restive rapids
#

you say that for a type to be assignable as an "Arithmetic", its add has to work with any object as the rhs, but Float2's add only works with another Float2

class SupportsAddWithSelf(Protocol):
  def __add__(self: Self, other: Self, /) -> Self:
    ...

def dup[A: SupportsAddWithSelf](a: A) -> A:
  return a + a

maybe you want something like this

trim tangle
#

if you just need addition, multiplication etc. to work within one type, then what Lambda suggested should work

grave geyser
#

Well the type needs to also accept floats on the right hand side. But always returns Self

#

But other: Self | float may be good enough

trim tangle
#

yeah

# 
from typing import Protocol, Self

class Addable(Protocol):
    def __add__(self, other: Self | float, /) -> Self:
        ...

def double[A: Addable](x: A) -> A:
    return x + x

n = double(3.45)
grave geyser
#

Except now again we need a type that can take itself and float

trim tangle
#

wdym

grave geyser
#

oops sorry, yes it should work

#

will try on the real type

trim tangle
#

actually, it might be fine in a protocol specifically

#

🤔

grave geyser
#

Hmm my Float2 type uses '@typing.overload' to define operators for both Float2 and float which doesnt seem to work...

trim tangle
#

can you show the definition?

grave geyser
#

Ah, I need to use overload also in the Protocol

trim tangle
#

not necessarily

#

maybe you have some other problem

grave geyser
# 
class Arithmetic(Protocol):
    @overload
    def __add__(self, arg0: Self) -> Self: ...
    @overload
    def __add__(self, arg0: float) -> Self: ...

vs

class Float2:
    ...
    @typing.overload
    def __mul__(self, arg0: Union[Float2, Int2, Tuple[float, float]]) -> Float2:
        ...
    @typing.overload
    def __mul__(self, arg0: Union[Int2, Tuple[int, int]]) -> Float2:
        ...
    @typing.overload
    def __mul__(self, arg0: float) -> Float2:
        ...
#

well except the mul add mismatch here

trim tangle
#

You need to add an / in the protocol definition

grave geyser
#

What does that do?

trim tangle
#

positional-only argument

#

otherwise the name of the argument in the implementing class must match

trim tangle
#

if it's not, it's a bug in the type checker

grave geyser
#

Then it's a bug in the type checker

# 
class Arithmetic(Protocol):
    def __sub__(self, arg0: Self, /) -> Self: ...
    def __add__(self, arg0: Self, /) -> Self: ...
    @overload
    def __mul__(self, arg0: Self, /) -> Self: ...
    @overload
    def __mul__(self, arg0: float, /) -> Self: ...

works

class Arithmetic(Protocol):
    def __sub__(self, arg0: Self, /) -> Self: ...
    def __add__(self, arg0: Self, /) -> Self: ...
    def __mul__(self, arg0: float | Self, /) -> Self: ...

doesn't work

#

But can you assume the type checker can combine overloads and see that they correspond to or:ed type expressions?

#

FYI: Final result:

class Tweenable(Protocol):
    def __sub__(self, arg0: Self, /) -> Self: ...
    def __add__(self, arg0: Self, /) -> Self: ...
    @overload
    def __mul__(self, arg0: Self, /) -> Self: ...
    @overload
    def __mul__(self, arg0: float, /) -> Self: ...

def tween[T: Tweenable](start: T, stop: T, steps: int, ease: Callable[[float], float]) -> Generator[T, None, None] :
    x = start
    for i in range(steps):
        x = (stop - start) * ease(i / steps) + start
        yield x
    while True:
        yield x
#

and thanks for the help!

keen flicker
trim tangle
#

(for example a list object is iterable)

keen flicker
#

so Iterator[T] then?

trim tangle
# keen flicker so `Iterator[T]` then?

Iterator[T] is an Iterable[T] that also has a __next__(self) -> T # throws StopIteration. It's the thing that actually goes over a collection and keeps some state about that going-over

#

Generator has extra methods (send and throw)

keen flicker
#

so Iterator[T] is exactly equal to Generator[T, None, None] ¿right? since None, None makes it so that the object doesnt expect any values sent or errors thrown

trim tangle
keen flicker
trim tangle
#

!e

def foo():
    yield 420
    yield 69
    return "done"

gen = foo()
print("first", next(gen))
print("second", next(gen))
try:
    next(gen)
except StopIteration as exc:
    print("StopIteration with value:", repr(exc.value))
rough sluiceBOT
trim tangle
#

here the correct annotation would be -> Generator[int, None, str] (or -> Generator[int, object, str])

keen flicker
#

but the functions

def func():
    for i in range(10):
        yield i**2

the docs you send say that it can be anotated either by Iterator and Iterable, but Iterable is something to call __iter__ upon and Iterator is something that was returned by __iter__ (as I understand it)

#

or calling the function calls __iter__ automatically and returns the generator?

trim tangle
#

Calling the function just gives you the generator object, no intermediate iterable is created

viscid spire
#

some people argue that you should use Iterable intead of Iterator for generator functions, in case you want to change the implemention to return an iterable which is not an iterator, such as a list

trim tangle
#

Yeah, don't publish a contract unless you really need to

trim tangle
#

If you made some changes and now get a list somehow, you can just return iter(that_list) to keep the interface

viscid spire
#

ig

grave geyser
#

If you want a literal number to be treated as an int, do you normally just do int(0) ?

trim tangle
grave geyser
#

I have an array of lambdas that returns tuples of numbers. If I don't put int() around any literal it can't figure out the type of the array

trim tangle
#

Do you have a code example?

#

This works with mypy:

from typing import reveal_type

first = [lambda: (0, 0), lambda: (0, 0), lambda: (0, 0)]
second = [lambda: (1, 2), lambda: (2, 3), lambda: (3, 4)]

reveal_type(first)  # Revealed type is "builtins.list[def () -> tuple[builtins.int, builtins.int]]"
reveal_type(second)  # Revealed type is "builtins.list[def () -> tuple[builtins.int, builtins.int]]"
harsh lantern
#

what type hint should i give for objects having the send method```py
async def send_chunks(sendable: Any, s: str) -> None:
"""Send messages in 2000-character chunks, to bypass Discord's 2000-character
limitations"""
for i in range(0, len(s), 2000):
await sendable.send(s[i:i+2000])

trim tangle
harsh lantern
#

oh i didn't think of that

trim tangle
grave geyser
#

@trim tangle

        z = 3
        fx = [ lambda: (2, 4), lambda: (z, 1) ]
#

But its PyLance in vscode that tells me
Type of "fx" is partially unknown
Type of "fx" is "list[Unknown]"

#

Actually this is enough: fx = [ lambda: 2, lambda: 3]

#

But yes, mypy on the command line with reveal_type() gets it as lambda -> int

trim tangle
#

This does look like a bug in pyright/pylance, perhaps file an issue

#

Alternatively, you can add an explicit annotation:

from collections.abc import Callable
z = 3
fx: list[Callable[[], tuple[int, int]]] = [ lambda: (2, 4), lambda: (z, 1) ]
rare scarab
#

I think python/typeshed#12181 broke my code.

    mode = f"w:{ext}".strip(":")
    with tarfile.open(dest, mode) as tar: # No overloads for "open" match the provided arguments
mighty lindenBOT
iron vapor
#

Can I somehow get a @staticmethod to indicate it's going to return an instance of the class defining it? I asked about Self previously, and the response was it doesn't make sense (which I disagree with, but that's nether here nor there), and if I do -> MyClass, it says MyClass is not defined, which I guess makes sense? Or do I have to make it a class method? As the logic makes more sense as a static method, I'd prefer to leave it as such, but if I have to make it a class method to move forward, I will.

trim tangle
# 
class Animal:
    @staticmethod
    def default() -> Self:
        ...

The semantics of Self are such that if you make a class Dog(Animal), Dog.default() should return a Dog

iron vapor
#

But I fully acknowledge as I don't know the inner workings and have only briefly read over the PEP that added Self, my thinking could be contrary to the implementation.

#

Which is why I said my opinion on it is neither here nor there.

#

If I can't use Self to indciate the return, I'm just wondering if there's another mechanism, or if I have to do a class method.

#

In other programming languages, it's entirely common to have a factory method as a static method, thus it was my first instinct here.

trim tangle
# iron vapor As it's a matter of a type checker, and we're not referencing `self` as in a poi...

Self is not just a shortcut for the current class name, it it equivalent to a using a typevar like this:

class Point:
    def __init__(self, x: int, y: int) -> None:
        self.x = x
        self.y = y

    def double(self) -> Self:
        return type(self)(self.x * 2, self.y * 2)

    # same as:

    def double[S: "Point"](self: S) -> S:
        return type(self)(self.x * 2, self.y * 2)

The purpose of Self is to make the method of a subclass return an instance of that subclass. So if you have a class FancyPoint(Point), doing fancy_point.double() returns a FancyPoint, not just a Point.

If you just want to name the class that's not defined yet (like the current class), you do not need Self. You can put the class name in quotes:

class Point:
    def __init__(self, x: int, y: int) -> None:
        self.x = x
        self.y = y

    def double(self) -> "Point":
        return Point(self.x * 2, self.y * 2)
iron vapor
trim tangle
iron vapor
#

I earnestly tried googling for how to do this, and the most common response was "It can't be done". I was really confused, because this seems like something that would come up often.

iron vapor
jolly cipher
#

there is one disadvantage to Self - typevars aren't parametrizable, so you can't really specify typevars of Self if "Self" (your type) is generic

iron vapor
#

Hm, interesting

jolly cipher
#

which can kinda make sense though

#

given that a generic subclass of your generic class may have a different "generic signature"

# 
class Foo[X, Y]:
    pass

class Bar[Y](Foo[int, Y]):
    pass
#

but yeah, it is what it is

#

Self is very useful regardless

iron vapor
#

Oh, I realize a flaw in my logic... as a staticmethod doesn't yet have it's wrapper class defined, it can't return an instance of itself.

#

I guess I have to make it a class method so it at least has a reference to it's class.

jolly cipher
#

wdym by "wrapper class defined"

iron vapor
# 
class MyClass:
    @staticmethod
    def my_method():
        # MyClass isn't defined here yet, it couldn't return in instance of MyClass
#

I'm recognizing that was meant earlier when i asked the preceeding quesiton.

trim tangle
#

Just like you can do this:

def f():
  return g()

def g():
  return 42
iron vapor
#

Type hinting saying it doesn't knw what MyClass is

#

I'm gonna restart my IDE, maybe there's something funky going on with vs code

#

Yup, it was vs code

#

Sorry

trim tangle
#

Maybe try basedpyright instead of the recommended Pylance 🙂

iron vapor
#

I'll take a look at that, thanks!

trim tangle
#

though that might not fix anything, pyright and its descendants sometimes seem to just get stuck and not brain at all

#

which is relatable, to be fair

iron vapor
#

Yah, and as the addage goes "turn it off and on again"

#

restarting vs code now shows it without error

#

So that was me thinking I misunderstood the logic, due to VSCode saying it was wrong

#

Though I admit, now I'm confused, the method knows what the wrapper class is, but I can't use it for the return type?

#

(without double quotes I mean)

trim tangle
# iron vapor Though I admit, now I'm confused, the method knows what the wrapper class is, bu...

!e
See this: https://decorator-factory.github.io/typing-tips/main-tutorial/1-working-with-classes/#adding-kittens
Basically, when you add an annotation, it's stored on the function for introspection:

class Foo:
    def bar(self, x: list[str]) -> int:
        print(self)

print(Foo.bar.__annotations__)

But when the method is being defined, the Foo class hasn't been created yet. So if you do -> Foo, it's not clear what should be placed in the __annotations__

rough sluiceBOT
iron vapor
trim tangle
#

There have been some attempts to fix this (see PEP 563)
The second attempt (PEP 649) is supposed to land in Python 3.14

iron vapor
#

I look forward to PIthon 😛 (I'm sure that joke has been made a million times)

trim tangle
jolly cipher
# iron vapor ``` class MyClass: @staticmethod def my_method(): # MyClass isn'...

generally speaking, remember that methods come from plain functions, so you can always expect them to behave like ones, with no magic involved like in other languages that can see things from a class inside methods without referencing self/this

when you make a class, what you do is put regular functions in a class scope which is then passed to your metaclass that creates your class, and things you put in the class scope become that class's __dict__

later on, when you access a name via a . from that class/instance (here: not set it like in foo.bar = ..., but access the value to only read it, like in foo.bar), it triggers attribute lookup that returns your attribute immediately or calls __get__ when it's available -- for plain functions, that creates you method objects that can bind self as the first argument of your function wrapped in the method

try this:

def foo(bar: int) -> None:
    print(f"{bar=}")

method = foo.__get__(10)
print(method)  # what is it? equivalent to partial(foo, 10) ;)
method()  # bar is just the conventional self

every function is just a descriptor, which makes methods with self auto-inserted for you possible
staticmethod and classmethod implement custom __get__ that changes the default function.__get__ behavior, because they are custom objects in your class scope that implement new __get__ and they accept your function as their first arg
learn more here:

Python documentation
Descriptor Guide

Author, Raymond Hettinger,, Contact,,. Contents: Descriptor Guide- Primer- Simple example: A descriptor that returns a constant, Dynamic lookups, Managed attributes, Customiz...

Python documentation
3. Data model

Objects, values and types: Objects are Python’s abstraction for data. All data in a Python program is represented by objects or by relations between objects. (In a sense, and in conformance to Von ...

iron vapor
iron vapor
#

Thanks so much for helping direct me. For now, bed, so I can figure out wtf this sqlalchemy documentation is trying to say in the morning, because holy binary, I really hate this documentation.

jolly cipher
#

it actually boosted my vocabulary so much i started getting a lot of A grades for essay assessments in my english classes

#

it can be a good training :)

iron vapor
#

The function descriptors are really good, if overly dense

#

The tutorials are just horrible

jolly cipher
iron vapor
#

I did, but decided sqlmodel wasn't for me, there are still features I'd like that aren't built yet, and it's documentation is still light, given it's relatively recent project

#

But to use SQLModel, I'd still need to learn SQLA's core

jolly cipher
#

i learnt a lot from reading it

#

it's a good read, i really recommend

#

i even have an artifact of doing that!

jolly cipher
iron vapor
#

I'm just bothered by things like this: I'm reading how to do inserts. So it talks about inserts and returning, and the tutorial is unclear. So I'm reading the method, method sqlalchemy.sql.expression.Insert.returning. And the example for that method, a method on the Insert class, it shows an update query.

#

update != insert

jolly cipher
iron vapor
#

Yah, sorry

#

These docs just get me heated 😄

#

Thanks for the help on typing though!

rare scarab
#

Hm.. pyright bug?

import tarfile
from typing import Literal

def open_tar(file: str, fmt: Literal["gz", "bz2", "xz", ""]) -> tarfile.TarFile:

  if fmt == "":
    return tarfile.open(file, "w")

  return tarfile.open(file, f"w:{fmt}") # error

  return tarfile.open(file, "w:" + fmt) # no error
# 
fmt = "gz"

reveal_type(fmt)         # Type of "fmt" is "Literal['gz']"
reveal_type("w:" + fmt)  # Type of ""w:" + fmt" is "Literal['w:gz']"
reveal_type(f"w:{fmt}")  # Type of "f"w:{fmt}"" is "LiteralString"
trim tangle
#

well... not sure if it's a "bug"

#

how type inference works is not specified in any specification

#

seems like it just hasn't implemented this inference rule for f-strings

rare scarab
#

It only became an issue after python/typeshed#12181

mighty lindenBOT
trim tangle
#

ah

rare scarab
rough sluiceBOT
#

stdlib/tarfile.pyi lines 233 to 237

@overload
def open(
    name: StrOrBytesPath | None = None,
    *,
    mode: str,```
rare scarab
#

seems kind of inconsistent within the other overloads

stiff acorn
trim tangle
# stiff acorn How do I type this? ```py from enum import EnumType from typing import Self cla...

EnumType is a metaclass, so e.g. when you do py class ColorChannel(Enum): # same as: class ColorChannel(metaclass=EnumType): red = 1 green = 2 blue = 3 ColorChannel is an instance of EnumType, not a subclass of it. Self means the type of an instance, so this method signature implies that if you had: py class ColorChannel(metaclass=_CaseInsensitiveGetItem): ... then ColorChannel["red"] would return an instance of _CaseInsensitiveGetItem, not an instance of ColorChannel.

#

I'm not sure if there's a way to type this method currently, but you can use __class_getitem__ on a subclass of Enum (not EnumType) instead

#

So something like this ```py
from enum import Enum
from typing import Self

class _CaseInsensitiveGetItem(Enum):
@classmethod
def class_getitem(cls, name: str) -> Self:
if not isinstance(name, str):
raise KeyError(name)

    for key, value in cls._member_map_.items():
        if key.casefold() == name.casefold():
            return value  # type: ignore[return-value]
    raise KeyError(name)
rare scarab
#

The proper way to implement a case insensitive enum is to use _missing_

# 
class Label(StrEnum):

    RedApple = auto()
    GreenApple = auto()

    @classmethod
    def _missing_(cls, name):
        for member in cls:
            if member.name.lower() == name.lower():
                return member
# 
>>> Label('redapple')
<Label.RedApple: 1>
stiff acorn
#

stdlib only has _missing_ which works on call

#

missing only works on the call syntax

#

i.e, Label("redapple") # works by calling _missing_

#

Label["redapple"] # fails, doesn't call _missing_

rare scarab
#

maybe the call will work

#

I don't know OP's usecase

#

!pip aenum if you can use packages

rough sluiceBOT
#
aenum v3.1.15

Advanced Enumerations (compatible with Python's stdlib Enum), NamedTuples, and NamedConstants

Released on <t:1687825192:D>.

stiff acorn
#

i mean fails at runtime

trim tangle
stiff acorn
# 
from enum import Enum

from typing_extensions import Self

class _CaseInsensitiveGetItem(Enum):
    @classmethod
    def __class_getitem__(cls, name: str) -> Self:
        if not isinstance(name, str):
            raise KeyError(name)

        for key, value in cls._member_map_.items():
            if key.casefold() == name.casefold():
                return value  # type: ignore[return-value]
        raise KeyError(name)
    
class Test(_CaseInsensitiveGetItem):
    APPLE = "Fruit"

print(Test["apple"])

Traceback (most recent call last):
  File "\test.py", line 22, in <module>
    print(Test["apple"])
  File "\cpython-3.9.20-windows-x86_64-none\lib\enum.py", line 432, in __getitem__
    return cls._member_map_[name]
KeyError: 'apple'
#

print(Test["APPLE"]) works as expected

trim tangle
#

Hmm, seems like a __getitem__ on a metaclass overrides __class_getitem__

rare scarab
#

metaclasses and typing don't really get along.

lunar dune
lunar dune
rare scarab
#

What if we used a generic on __call__?

# 
class MyMeta[T](type):
  if TYPE_CHECKING:
    def __call__(self, *args, **kwargs) -> T: ...

  def __getitem__(self, key: str) -> T: ...
```?
stiff acorn
#

so is my solution just adding # type: ignore comments?

trim tangle
# trim tangle `EnumType` is a metaclass, so e.g. when you do ```py class ColorChannel(Enum): ...

@stiff acorn The main problem with the original code is this -- Self is not the right return type. You should use the same method signature that's used in EnumType._getitem__:

from enum import EnumType
from typing import Self

class _CaseInsensitiveGetItem(EnumType):
    def __getitem__[T](self: type[T], name: str) -> T:
        if not isinstance(name, str):
            raise KeyError(name)

        for key, value in self._member_map_.items():  # type: ignore
            if key.casefold() == name.casefold():
                return value
        raise KeyError(name)
brittle plover
#

Does someone know a python library or project, that makes extensive use of current (3.12+) type-hinting features. I'd like to explore a bit and learn what can be done when taking the current type system to its limits.

stiff acorn
#

although I still need 2 type ignore comments

#

but that's better than 3

trim tangle
#

Why is the super().__dict__["_member_map_"] needed?

#

it should be the same as self._member_map_

trim tangle
stiff acorn
stiff acorn
# 
from enum StrEnum, EnumType
from typing_extensions import TypeVar

T = TypeVar("T")

class _CaseInsensitiveGetItem(EnumType):
    def __getitem__(cls: type[T], name: str) -> T:
        if not isinstance(name, str):
            raise KeyError(name)

        for key, value in super()._member_map_.items():
            if key.casefold() == name.casefold():
                return value
        raise KeyError(name)


class Test(StrEnum, metaclass=_CaseInsensitiveGetItem):
    APPLE = "Fruit"


print(Test["apple"])

AttributeError: 'super' object has no attribute '_member_map_'
#

super().__dict__["_member_map_"] on the other hand works

trim tangle
#

You don't need super()

#

just self._member_map_

stiff acorn
#

you're right, that works

#

although now i get a new error from mypy

#

"type[T]" has no attribute "_member_map_"

trim tangle
#

Yeah, you'll need to # type: ignore here probably

#

Or maybe this:

from enum import EnumType
from typing import Self, TypeVar

T = TypeVar("T", bound=EnumType)

class _CaseInsensitiveGetItem(EnumType):
    def __getitem__(self: type[T], name: str) -> T:  # type: ignore[misc]
        if not isinstance(name, str):
            raise KeyError(name)

        for key, value in self._member_map_.items():
            if key.casefold() == name.casefold():
                return value  # type: ignore[return-value]
        raise KeyError(name)
#

(the added bound)

#

Some things are just not possible to do without a bunch of # type: ignores, Python is just much more dynamic than the type system supports 🙂

stiff acorn
#

yea that's true, just want to keep them to a minimum and I learned alot of new things from this so it was a productive chat for me

#

thanks a lot

stiff acorn
steep ether
# 
    # Determine the base directory based on user input.
    if choice == 1:
        # Get the custom directory path from the user.
        base_directory: str = input("Enter custom output directory path: ").strip()

        # Check if the custom directory exists and is writable.
        if not os.path.exists(base_directory):
            print(f"Error: The directory '{base_directory}' does not exist.")
            return
        if not os.access(base_directory, os.W_OK):
            print(f"Error: The directory '{base_directory}' is not writable.")
            return
    else:
        # Default to the current working directory.
        print("Using current working directory for output.")
        base_directory = os.getcwd()

if i specified the type-hint once then do i have do it in all the place where im using the same variable

#

like base_directory: str i specified it once, so do i have to do it for all the other lines where i use this variable ?

pastel egret
#

No, it applies to the whole scope (function, class, global). And you shouldn't need to annotate that anyway since input() returns str, and so does str.strip()

elfin junco
#

Hello,
I'm trying to have a generic class with several methods that return different elements of a tuple which type is defined with a generic arg. And I'm trying to extract type at position n of the generic arg to annotate return type of my method.

class MyClass[T: tuple[Any, ...]]:
    def __init__(self, t: T) -> None:
        self.t = t
    @property
    def first(self): # Missing return type annotation here
        return self.t[0]
    # Define second, third

Is there someway to do this or some other pattern to avoid this ?
Thanks

paper salmon
elfin junco
paper salmon
#

oh, i'm not sure about that then, there's TypeVarTuple but afaik you can't annotate a particular element with that...

trim tangle
paper salmon
#

derivative question, is it possible to overload an entire class?

trim tangle
#

what would that mean?

trim tangle
#

This also has the advantage of allowing MyClass[*tuple[()]] and MyClass[int] to exist (even though you can't use all of MyClass's methods on them)

paper salmon
paper salmon
# trim tangle what would that mean?

having looked at sqlalchemy's auto-generated overloads, i imagined something like this: ```py
@overload
class MyClass[T0]:
def init(self, t: tuple[T0]): ...
@property
def first(self) -> T0: ...
@property
def second(self) -> NoReturn: ...

@overload
class MyClass[T0, T1, *Tx]:
def init(self, t: tuple[T0, T1, *Tx]): ...
@property
def first(self) -> T0: ...
@property
def second(self) -> T1: ...```

trim tangle
#

No, that's not possible

trim tangle
#

thought the reference only presents the case that can be replaced with Self, so admittedly it's not helpful here

paper salmon
#

ye i was wondering how it would have differed from Self in their example, but otherwise TIL

#

neat that it recognizes when you pass insufficient elements for second/first

steep ether
# 
    # Default values for parameters
    fps: str = '1'
    start_time = None
    end_time = None
    crop_width = None
    crop_height = None
    crop_x = None
    crop_y = None

    # Prompt for custom settings
    if input("Want to set custom timing? (Yes/No): ").strip().lower() == "yes":
        try:
            start_time = int(input("Enter start time in seconds: "))
            end_time = int(input("Enter end time in seconds: "))
        except ValueError:
            print("Invalid timing input. Using default timing (full video).")

    if input("Want to set custom FPS? (default fps = 1) (Yes/No): ").strip().lower() == "yes":
        fps = input("Enter FPS (e.g., '2/3' or '30'): ").strip()

    if input("Want to crop the video? (Yes/No): ").strip().lower() == "yes":
        try:
            crop_width = int(input("Enter crop width: "))
            crop_height = int(input("Enter crop height: "))
            crop_x = int(input("Enter crop X offset: "))
            crop_y = int(input("Enter crop Y offset: "))
        except ValueError:
            print("Invalid cropping input. Skipping cropping.")

what will be the type hinting for start_time, end_time, crop_width, crop_height, crop_x, crop_y

trim tangle
steep ether
#

ohk

trim tangle
#

Though you probably don't need to add explicit annotations here

steep ether
#

i was thinking of that but wasnt sure, tysm

trim tangle
#

fps: str = '1' is unnecessary, just do fps = '1'

steep ether
trim tangle
#

You only need an explicit annotation on a variable if its type cannot be determined from context. Mostly with empty collections:

numbers = []  # type checker can't know what kind of list this is
numbers: list[int] = []  # now it can
barren owl
steep ether
#

i dont know about optional can you please explain it to me a bit further

#

ffmpeg_input = ffmpeg.input(original_video_location, **ffmpeg_input_args)
in line like these we already know that ffmpeg.input() -> Any, so i dont need to explicitly specify the type hint right ?

barren owl
steep ether
#

ohh so it will always consider NONE and other mentioned datatype

viscid spire
#

Optional is cringe, use a union

barren owl
#

yea like: user could input fps or not, so in case fps has no default value, it should be Optional

barren owl
steep ether
#

def createDir() -> Optional[str]: or def createDir() -> str | None:
both are same right ?

viscid spire
#

just cause it may be common on GitHub doesn't mean it's not cringe

steep ether
#

can you explain why union ?

#

if you dont mind

viscid spire
steep ether
# 
    try:
        os.makedirs(folder)
        print(f"Successfully Created: {folder}")
        return folder
    except OSError as e:
        print(f"Error: Could not create directory. {e}")
        return

folder is a str type

viscid spire
#

we talked about this before right?

#

use return None

#

it's a usable return value, so you should indicate it as such

steep ether
#

yep we did i just wanna cross check

viscid spire
steep ether
viscid spire
steep ether
#

def createDir() -> Union[str, None]: is this correct

viscid spire
#

yeah, but I would use | in recent versions

#

str | None

steep ether
#

i know im asking a lot of silly questions, but its my first time dealing with type hinting

#

ohh ohk

steep ether
viscid spire
#

No

#

replaces

steep ether
#

ic

steep ether
viscid spire
#

Union[str, None] => str | None

barren owl
viscid spire
#

I still don't like it then

steep ether
viscid spire
#

because you can have an optional arg that doesn't have a value of None

viscid spire
steep ether
#

ohhkkkk

#

start_time: Optional[int] = None
should i replace these as well with int | None ?

viscid spire
#

I would

steep ether
#

ok

viscid spire
#

Why give a special name to unions which are with None?

barren owl
viscid spire
fleet ember
trim tangle
#

However, you can use a hack like this:

from typing import TYPE_CHECKING

def _parse_obj(source: object, specification: object) -> object:
    ...

if TYPE_CHECKING:
    class Parser[T]:
        def parse(self, value: object, /) -> T:
            ...
else:
    class _ParserFactory:
        def __getitem__(self, typeform):
            return _Parser(typeform)

    class _Parser:
        def __init__(self, typeform):
            self.typeform = typeform

        def parse(self, source):
            return _parse_obj(source=source, specification=self.typeform)

    Parser = _ParserFactory()  


Parser[Literal[2, 4]].parse(some_int)  # ok
fleet ember
topaz flower
#

I have ```python
def findBodies(doc: FreeCAD.Document):
return cast(Sequence["PartDesign.Body"], doc.findObjects("PartDesign::Body"))

#

is it possible to make the "PartDesign.Body" a parameter as there are several different ones and the two strings are always the same except for the :: vs .

trim tangle
#

cast doesn't actually do anything at runtime, it's purely a type checking directive

restive rapids
topaz flower
#

@restive rapids it says T is not defined so maybe there's some syntax to quantify it somehow

restive rapids
#

yeah mb im eepy so didnt add it, edited

topaz flower
#

@trim tangle it looks like they call it a type parameter

trim tangle
#

Maybe Freecad provides a better way to do something like this? I'm trying to find the documentation for the FreeCAD.Document class but I haven't succeeded yet

#

||the freecad experience 💀||

topaz flower
rough sluiceBOT
#

freecad_stubs/FreeCAD-stubs/__init__.pyi line 586

class DocumentObject(FreeCAD.ExtensionContainer):```
trim tangle
#

apparently the Python classes are generated with PyCXX, so I got confused a bit

#

Is there some documentation on how these ::'d names are constructed? If it's just <module_name>.<class_name>, then what Lambda suggested is good

#

though maybe add a runtime check that t.__module__ is in a whitelist

topaz flower
# 
T = TypeVar("T")
def findObjects(t: type[T], doc: FreeCAD.Document) -> Sequence[T]:
    return doc.findObjects(t.__qualname__.replace(".", "::")) # type: ignore

findObjects(PartDesign.Body, doc)
#

this is what I have so far. The problem is that module 'PartDesign' has no attribute 'Body'.

#

it is a class though

#

PartDesign.Body() fixes that problem

trim tangle
topaz flower
#

yes I'm still confused

topaz flower
trim tangle
#

This is a runtime issue, so it shouldn't have anything to do with type checking

#

Is there some documentation for the PartDesign Python module? I'm getting this:

>>> import PartDesign
>>> dir(PartDesign)
['InvoluteGearFeature', 'SprocketFeature', 'WizardShaft', '_PartDesign', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'makeFilletArc']
>>>
topaz flower
#

my expectation is that the t: type[T] doesn't have to get evaluated since it's only for the type checker

trim tangle
#

type annotations are just for the type checker, so if you remove them, the code should have the same behaviour. I.e.: ```py
def findObjects(t, doc):
return doc.findObjects(t.qualname.replace(".", "::"))

findObjects(PartDesign.Body, doc)

trim tangle
#

it seems like the Body class advertises itself as being part of the PartDesign module, but it's not in there:

>>> [body] = doc.findObjects("PartDesign::Body")
>>> body
<body object>
>>> type(body)
<class 'PartDesign.Body'>
>>>
trim tangle
#

Any idea when the next typing-extension version is released? 👀 I want to try using TypeForm

trim tangle
#

@oblique urchin I have a question about the design of typing-extensions. It seems like the package serves two distinct purposes:

  1. Backporting brand new accepted typing features
  2. Giving access to experimental not yet accepted typing features
    I don't understand why items in both categories have the same "published status". For example, if Doc or TypeForm don't get accepted, they'll linger in typing_extensions forever. Worse, if a future PEP proposes something else called Doc or TypeForm, you either have to remove the old items (breaking the compatibility) or waiting on the items until they're accepted (and still breaking compatibility).
    Wouldn't it have been better to have a separate module (like typing_extensions.experimental) for stuff that's clearly experimental?
#

the documentation says

No typing PEP that affected typing_extensions has been rejected so far, so we haven’t yet figured out how to deal with that possibility.
this is a bit concerning 😄

oblique urchin
#

You're right that we'd be in trouble if a future PEP proposes to use the same name for an incompatible object. That doesn't seem too likely though.

#

Even if the semantics are different, the runtime behavior of the name is still likely to be compatible.

stable fjord
#

well, you can make things raise deprecationwarnings and the like - but ofc that'd introduce a hefty delay before it can get removed and then reused

trim tangle
#

that was essentially my worry, that e.g. PEP 727 may be rejected from the language (for example because it's deemed too verbose/occupying too much space/other reasons), but it is now de facto added to the language

#

though, of course, type checkers and IDEs don't have any such compatibility guarantees and may decide to not use typing_extensions.Doc

oblique urchin
trim tangle
#

that's fair

oblique urchin
#

also don't look for typing_extensions.IntVar

lunar dune
#

Maybe one day we'll find a use for that

#

you never know

empty moon
#

Is there anything like this in Pydantic where models are "selected" from enums with a tag?
Example in Rust with serde and serde_json:

image.png
#

Figured it out, Tagged Unions are used for this

frigid nova
#

I thought Python 3.14 and its deferred annotations would let us avoid forward refs when inheriting from a generic type, but it looks like it's not the case, and I can't find any relevant information in PEP 649:

# generic_base.py 
class Foo[T]:
    ...

# Works fine for a type annotation.
foobar: Foo[Bar] = Foo()

# But not when inheriting from the class.
class FooBar(Foo[Bar]):
    ...

class Bar:
    ...

% python3.14 generic_base.py
Traceback (most recent call last):
  File "/home/pawamoy/generic_base.py", line 8, in <module>
    class FooBar(Foo[Bar]):
                     ^^^
NameError: name 'Bar' is not defined
% python3.14 -V             
Python 3.14.0a0

Looking at the code, it seems obvious, because in the base class case, Foo[Bar] is runtime. But the [Bar] part really is typing information, and doesn't affect runtime, no?

rare scarab
#

note that unions are resolved left to right and uses the first successful parse.

#

so str | int would be considered invalid as everything could be interpreted as a str

empty moon
rare scarab
#

that's the |

empty moon
#

yes

oblique urchin
frigid nova
tranquil ledge
#

Oh . . .

#

I didn’t read the precursor message. Whoops. Just ended up repeating things. My bad.

pastel egret
#

An issue with using a submodule is that then if it was accepted, early adopters would need to migrate 3 times - first use typing_extensions.candidates or whatever, then typing_extensions once accepted, then finally typing once the min Python version covers usage. Not really desirable.

jolly cipher
jolly cipher
#

(smart mode is the default)

hardy linden
#

An API I'm working with exposes a "fetcher" function that I need to call over and over to return either new valid values, or a sentinel value that indicates that there's no more output. I'm not a huge fan of that behavior, so I'm wrapping it to be able to lazily iterate over the valid values.

In my wrapper, I just yield each valid value in turn; when I get to the sentinel value, I discard it, don't yield anything, and let the generator end. I'm annotating the fetcher function as Callable[ParamSpec, ReturnType], and the wrapper is returning Generator[ReturnType, None, None].

That all works, without any issues, but when a caller tries to use it, because the output of the generator is still ReturnType, unchanged, that means that the sentinel value is still being seen as one of the possible values produced by the generator, which causes some type-checking errors in the caller if I try to act as though the sentinels have been filtered out.

Here's my question: is there any way to annotate the return type of my wrapper as something like Generator[Difference[ReturnType, SentinelType], None, None], instead - or any other way to indicate that the sentinel value isn't a possible result from the generator?

I have a strong suspicion that this is not something Python's typing system currently allows for, given the long debates I've seen over things like Intersection types - but I wanted to double-check to make sure I wasn't missing something.

(In my exact use-case, the sentinel value happens to be None, and all valid values are non-None, so there might be some None-specific way to address this - but ideally, it'd be nice to have a way to annotate this that works for any arbitrary sentinel type.)

trim tangle
# hardy linden An API I'm working with exposes a "fetcher" function that I need to call over an...

You can do something like this ```py
from collections.abc import Iterator, Callable
from typing import reveal_type

def stinky_generator(foo: int, /, bar: str, *, baz: bool) -> int | None:
...

def unstinky[T, **P](fn: Callable[P, T | None]) -> Callable[P, Iterator[T]]:
def wrapper(*args: P.args, **kwargs: P.kwargs) -> Iterator[T]:
while True:
value = fn(*args, **kwargs)
if value is None:
break
else:
yield value
return wrapper

nice_generator = unstinky(stinky_generator)
reveal_type(nice_generator)

Type of "nice_generator" is "(foo: int, /, bar: str, *, baz: bool) -> Iterator[int]"

#

Putting the T | None in the parameter position essentially acts like a "difference" when figuring out the type variable

#

(**P is the 3.12 way of spelling ParamSpec, so if you need to support 3.11 or lower use typing/typing_extensions.ParamSpec)

hardy linden
trim tangle
#

!e

class Stinky:
    def __init__(self, start):
        self.x = start

    def generate(self):
        if self.x > 0:
            self.x -= 1
            return self.x + 1
        else:
            return None

stinky = Stinky(5)
it = iter(stinky.generate, None)
print(list(it))
rough sluiceBOT
trim tangle
rough sluiceBOT
#

stdlib/builtins.pyi lines 1463 to 1470

@overload
def iter(object: SupportsIter[_SupportsNextT], /) -> _SupportsNextT: ...
@overload
def iter(object: _GetItemIterable[_T], /) -> Iterator[_T]: ...
@overload
def iter(object: Callable[[], _T | None], sentinel: None, /) -> Iterator[_T]: ...
@overload
def iter(object: Callable[[], _T], sentinel: object, /) -> Iterator[_T]: ...```
trim tangle
#

For some reason it happens to have an overload specifically for your case 😛

hardy linden
#

...well, I may feel like an idiot, but I guess at least now I don't have to write those unit tests

#

Thanks, lol

median gulch
#

could mypy properly get a function decorator that executes itself and returns the value?

# 
from typing import *
T = TypeVar('T')
def as_value(func: Callable[[], T]) -> T:
    return func()
@as_value
def n():
    return 42
assert n == 42
#

couldn't get something similar to work with singleton class decorator that takes type[T] and returns T, so I resorted to a metaclass with a registry

stiff acorn
median gulch
#

how do I tell mypy a class implements another without inheriting?

#

I'm making my own typing.NamedTuple with additional methods

#

but I naturally can't inherit NamedTuple more than once, so I'm using typing.NamedTupleMeta which is the actual implementation, whereas NamedTuple is an empty class

#

imagine the following hierarchy:
NamedTupleMeta -> CustomNamedTupleMeta -> CustomNamedTuple -> DataStructure

where CustomNamedTupleMeta has regular metaclass business going on

where CustomNamedTuple is just a regular class as NamedTuple is

where DataStructure is a dev defined object

#

up until CustomNamedTuple, mypy gives proper lsp hints

#

at DataStructure the regular NamedTuple's lsp hints are gone

#

forget it, found my answer

stable fjord
#

is there a reason you're using NamedTuple instead of a dataclass?

median gulch
#

because the system that will consume the objects really cares about ordering and and iterating

#

but the input that must be transformed and organized are unordered and indexed by field name

#

the dev must know the values by name/context but the system only cares about ordering and strictness

#

imagine merging a dict and a namedtuple in behavior but it actually is just a tuple

stable fjord
#

I feel like it'd perhaps be more sensible to just define your own base class or protocol, and define all the dunders needed to make it work like you want

#

but idk, sounds messy/complicated

median gulch
#

does dataclasses preserve the order of the attributes defined when you get the field names?

stable fjord
#

I think so. Dicts also maintain order

median gulch
#

is this too complicated to you?

from typing import Any, NamedTupleMeta, SupportsIndex  # type: ignore

from pydantic import ConfigDict, TypeAdapter


# SEE typing.NamedTupleMeta implementation


class ModelMeta(NamedTupleMeta):
    __adapters: dict[type, TypeAdapter] = {}

    def __new__(cls, name, bases, attrs):
        nm_tpl = super().__new__(name, bases, attrs)
        if nm_tpl not in cls.__adapter:
            cls.__adapters[nm_tpl] = TypeAdapter(nm_tpl, config=ConfigDict(strict=False))
        return nm_tpl

    def _type_adapter(cls)->TypeAdapter:
        return cls.__adapters[cls]


class Model(metaclass=ModelMeta):
    def __getitem__(self, name: str|slice|SupportsIndex,/) -> Any:
        if isinstance(name, str):
            try:
                return getattr(self, name)
            except AttributeError as err:
                raise IndexError(err)
        return self.__getitem__(name)
stable fjord
#

you definitely need some comments and/or docstrings

median gulch
#

I see

#

the implementation of NamedTupleMeta fits on the screen and I understood it quickly, so I thought most people also could

#

doing this current project alone, so struggling with guessing what other people would find difficult

stable fjord
#

I think as soon as you start messing with metaclasses and dunders you will quickly lose a lot of people, and ModelMeta/Model aren't super descriptive names that you immediately understand what they're for. But if it's mostly just copying the behaviour of namedtuple/meta then you can expand your SEE comment to say that; and then describe what the classes do/should be used for in their docstrings. If you expect people to be reading and/or modifying the implementation of those dunder methods I'd probs pop some comments in there as well - but when given the context from your discord messages I can mostly figure them out without too much problem.

median gulch
#

wouldn't there be an influence of the parsing on it?

median gulch
stable fjord
stable fjord
median gulch
stable fjord
#

(also we're pretty fram from the topic of #type-hinting by now, heh)

median gulch
#

(yeah)

stable fjord
# median gulch is there a chance python would parse a class so that the attributes wouldn't be ...

no, "The order of the fields in all of the generated methods is the order in which they appear in the class definition." https://docs.python.org/3/library/dataclasses.html#module-contents

Python documentation
dataclasses — Data Classes

Source code: Lib/dataclasses.py This module provides a decorator and functions for automatically adding generated special methods such as__init__() and__repr__() to user-defined classes. It was ori...

median gulch
#

thank you

#

no more questions

stable fjord
#

happy to help, good luck!

topaz flower
errant ember
#

from datetime import timedelta

def tt(ss):
return timedelta(seconds=ss)

if name == "main":
seconds = 684500
kq = tt(seconds)
print(kq)

#

!e
from datetime import timedelta

def tt(ss):
return timedelta(seconds=ss)

if name == "main":
seconds = 684500
kq = tt(seconds)
print(kq)

rough sluiceBOT
hasty phoenix
#

What is the proper way to annotate class methods that may be inherited. E.g. this makes the type checker confiused, as factory returns a specific type that's not inherited:

from __future__ import annotations
class A:
    @classmethod
    def factory(cls) -> A:
        return cls()

class B(A):
    pass

b = B.factory()  # reveals as A but is a B instance
restive rapids
#

!d typing.Self
or a typevar T that you'd bind from cls: type[T], @classmethod def f[T](cls: type[T]) -> T: ...

rough sluiceBOT
#
typing.Self 

typing.Self```
Special type to represent the current enclosed class.

For example...
hasty phoenix
#

type[T] sounds cleaner. Thanks

trim tangle
# hasty phoenix `type[T]` sounds cleaner. Thanks

probably not, the Self version looks like this py class A: @classmethod def factory(cls) -> Self: return cls() and the typevar version looks like this ```py

after 3.12

class A:
@classmethod
def factory[T: "A"](cls: type[T]) -> T:
return cls()

before 3.12:

T = TypeVar("T", bound="A")
class A:
@classmethod
def factory(cls: type[T]) -> T:
return cls()

soft matrix
trim tangle
#

PEP 749 is scheduled for 3.14 for now

soft matrix
#

it seems to work fine

#

i thought i managed to convince them that that would be useful to use the mechanism for 749

trim tangle
#

!e

def f[T: "yes" + 42]():
    pass
rough sluiceBOT
trim tangle
#

yep

soft matrix
#

!e

def f[T: "yes" + 42]():
    pass

f.__type_params__[0].__bound__
rough sluiceBOT
trim tangle
#

oh

#

I honestly don't understand the proposed mechanism at all, never even tried

#

does it just change __annotations__ to a descriptor that executes the annotations when they are first accessed?

tranquil turtle
#

yes

iron vapor
#

What is the type hint for something like str | bool | int | ...? Basically the first param for typing.cast? I see the docs show it as type[_T@cast], but I can't seem to add that directly to my code, and I'm not sure if there's something else I should use.

#

I'm trying ot use typing.cast and want to make sure I type hint properly

cinder bone
#

If you're using typing.cast, you don't need to worry about it

iron vapor
#

Sorry, to clarify

#

The value is coming from a variable

#

Like this:

        cast_type = UserMeta.MetaKeys[self.key].value[2]
        return cast(cast_type, self._value)
#

I was thinking of doing a union, which would work

#

I wasn't sure if there's a more generic value to use though

cinder bone
#

Any is probably your best bet here

iron vapor
#

Pylance complains about any

cinder bone
#

In which case there's no reason to call cast anyways

cinder bone
#

pyright shouldn't complain unless you have some config or you're using basedpyright

iron vapor
#

Hm, the error changed, now it's saying Variable not allowed in type expression

#

So maybe it's my code

cinder bone
#

yeah, you shouldn't be using typing.cast here

iron vapor
#

Bah, not sure how to do this then

cinder bone
#

Just don't use typing.cast, return self._value directly

iron vapor
#

Ok, thanks, I'll google

#

Well, no, the problem is I need to cast the value

trim tangle
#

What does it mean to cast the value?

iron vapor
#

to convert types?

cinder bone
#

typing.cast doesn't do anything at runtime, if you're trying to change types at runtime

iron vapor
#

Oh... the way the docs read, I thought it literally cast

#

Oh, it's just doing the type checking cast

#

That's... useless? Well, maybe there's a use I don't see

cinder bone
#

it's literally just

def cast[T](ty: type[T], obj: object) -> T:
    return obj  # type: ignore
iron vapor
#

Yah, interesting

cinder bone
trim tangle
#

yeah

iron vapor
#

Yah, that makes sense

#

Ok, then I was doing the wrong thing from the start

trim tangle
#

There's no such thing as "casting to a type" in Python. typing.cast is just a shortcut for this:

# assume that `foo` is a str | None, but the programmer knows it's a `str` here
y: str = foo  # type: ignore
y = typing.cast(str, foo)

it also avoids some pitfalls that a blanked # type: ignore has, like not catching a typo (food instead of foo)

iron vapor
#

Ok, so then since it's basically str | int | bool, I can just do cast_type(self._value)

#

I guess I don't really know the typing for that either... it's a callable, but of specific types?.

trim tangle
#

What values do cast_type and self._value take?

#

Can you give some examples of what it should do maybe?

iron vapor
#

Cast type is literally str or int or bool. Not as in it's of type str, but the actual type str, as in str()

#

It's a situation where if x == 1, y is an int, if x == 2, y is a bool, etc

trim tangle
#

Maybe it's better to think of cast_type as a function that attempts to convert a value?

iron vapor
#

Right, a callable, but I was hoping to limit the types so I could set up the returns correctly. I tried Literal, but Literal[int] gets an error. I guess it'd be Literal["int"]?

trim tangle
#

the value int has the type of type[int]

iron vapor
#

Hm, literal is wrong, and callable is wrong

#

AH!

#

That should do it!

#

Thank you!

trim tangle
#

But whenever you want to use type[Something] as a parameter, in 90% of cases it's better to use something else. Like Callable[[object], int]

iron vapor
#

Ok, I'll look into the callable syntax more

#

I appreciate it!

#

I may end up restructuring my db anyway, but this is useful knowledge

iron vapor
#

Ok, another type hinting question. Is there a way to tell the static checker that in a certain if block, I know the type of a value will be a certain something? Something like

if x == 1:
    # here, y will always be a str
elif x == 2:
    # here, y will always be a bool
rustic gull
iron vapor
#

While I have some solutions I'll be looking at to try to break it apart, for now, it is what it is

rustic gull
# 
    if x == 1:
        return "something"
    if x == 2:
        return False


if __name__ == '__main__':
    b = a(1)
    b.strip()
    b = a(2)
    b.strip()

Mypy understands this, and will give the correct error

mypy_example.py:1: error: Missing return statement  [return]
mypy_example.py:10: error: Item "bool" of "str | bool" has no attribute "strip"  [union-attr]
mypy_example.py:12: error: Item "bool" of "str | bool" has no attribute "strip"  [union-attr]
Found 3 errors in 1 file (checked 1 source file)
iron vapor
#

Yah, that makes sense

#

I'll handle that tomorrow... I've been trying to create my own enum type for the last hour and my brain hurts now, time to sleep 😛

lament iris
#

Speaking of enums, why does this happen?

image.png
#

Code

from typing import reveal_type
from enum import Enum


class Foo(Enum):
    BAR = "bar"

reveal_type(Foo._value2member_map_)
print(Foo._value2member_map_) # {'bar': <Foo.BAR: 'bar'>}

# Shouldn't the reveal_type show Dict[str, Foo]?
oblique urchin
trim tangle
#

yep

trim tangle
oblique urchin
rough sluiceBOT
#

stdlib/enum.pyi line 175

_value2member_map_: dict[Any, Enum]  # undocumented```
trim tangle
#

oh right

lament iris
#

I came up with this horror lol

from enum import Enum, EnumMeta
from typing import Any, TypeVar, KeysView, ValuesView, reveal_type
from types import MappingProxyType

_ET = TypeVar("_ET", bound="EnumMeta")

class EnumExtras(EnumMeta):
    def values_to_keys(self: type[_ET]) -> dict[_ET, str]:
        assert isinstance(self.__members__, MappingProxyType)
        # return {v: k for k, v in zip(self.keys(), self.values())} # <- does work but gives a type error
        # return {v: k for k, v in self.__members__.items()} # <- also works, but gives a type error
        reveal_type(self.__members__) # Type of self.__members__ is Never (HUH????)
        return {v: k for k, v in zip(self.__members__.keys(), self.__members__.values())} # This works and gives no type error (somehow)

    def keys(self) -> KeysView[str]:
        return (self.__members__.keys())

    def values(self: type[_ET]) -> ValuesView[_ET]:
        assert isinstance(self.__members__, MappingProxyType)
        return self.__members__.values()


class SupportedConfigurationFormats(Enum, metaclass=EnumExtras):
    AOC_CONFIGURATION_FILE = ".advent"
    PYPROJECT_TOML = "pyproject.toml"
    # ...


print(SupportedConfigurationFormats.values()) # type error
print(SupportedConfigurationFormats.keys())
print(SupportedConfigurationFormats.values_to_keys()) # another type error
#

Nvm! Fixed it!

from enum import Enum, EnumMeta
from types import MappingProxyType
from typing import Any, KeysView, TypeVar, ValuesView, reveal_type

_ET = TypeVar("_ET")


class EnumExtras(Enum):
    @classmethod
    def values_to_keys(cls: type[_ET]) -> dict[_ET, str]:
        assert isinstance(cls, EnumMeta)
        return {v: k for k, v in cls.__members__.items()}

    @classmethod
    def keys(cls: type[_ET]) -> KeysView[str]:
        assert isinstance(cls, EnumMeta)
        return cls.__members__.keys()

    @classmethod
    def values(cls: type[_ET]) -> ValuesView[_ET]:
        assert isinstance(cls, EnumMeta)
        return cls.__members__.values()


class SupportedConfigurationFormats(EnumExtras):
    AOC_CONFIGURATION_FILE = ".advent"
    PYPROJECT_TOML = "pyproject.toml"
    # ...


print(SupportedConfigurationFormats.values())
print(SupportedConfigurationFormats.keys())
print(SupportedConfigurationFormats.values_to_keys())
rustic gull
#

any one here to talk with me

#

someone

median gulch
# iron vapor Right, a callable, but I was hoping to limit the types so I could set up the ret...

Yes. Type hints are just typing.GenericAlias or type objects, meaning type hints are not you signaling information to the interpreter, because it ignores hints unless they contain illegal python syntax. Type hints are just python expressions that return an object containing metadata consumable by static type checkers, which can be accessed at runtime through the inspect module. Mypy for instance, doesn't even evaluate your code, it retrieves the type hints from the ast and check them against their rules. So, yes, Literal[int] and Literal["int"] are different.

And the error you get happens because "int" is a constant value, meaning that knowing your variable is a string isn't enough for the type checker. You either need to tell it that string variable is constant with typing.Final[str], ignore the error, cast it to typing.Literal["int"], or just return typing.Literal["int"].

from typing import Final, Literal, TypeAlias, Any, Union, Never

regular_string: str = 'int'
apple_string: Final[str] = 'apple'
int_string: Final[str] = 'int'
Options: TypeAlias = Literal['int', 'str', 'float']

def unsafe_is_option(ty: Options) -> Union[Options, Never]:
  if ty in Options.__args__:
    return ty
  raise ValueError # raising error == returning Never

unsafe_is_option(regular_string) # typing error, runtime ok
unsafe_is_option(apple_string) # typing ok, runtime error
unsafe_is_option(int_string) # typing ok, runtime ok

There are syntactic sugars related to defining hints, but for the point of them being objects, this is the most literal syntax.

median gulch
#

On a note: int is a subclass of float, so int can be assigned to float, but never the contrary; even if somehow the type checker knows your float has no decimal points, like how dividing with / gives you a float, but diving with // gives you an int, but 10/2 is a float and 10//2 is an int even though numerically it shouldn't matter.

Yet, since int is a subclass of float, 5.0 == 5 is True at runtime, because python does care about numerical representation at number comparisons, so the type checker won't complain about int == float expressions.

trim tangle
#

which is stinky by itself, but to make things worse, mypy doesn't fully understand this fact

def f(x: float | str) -> None:
    if isinstance(x, float):
        print(x + 3.14)
    else:
        print(x + "!!!")

f(42)  # allowed by mypy, will explode at runtime
median gulch
# trim tangle Well, `int` is not really a subclass of `float` (`isinstance(42, float) is False...

Just double checked and you're right. Found the source of my misunderstanding: https://docs.python.org/3.11/reference/datamodel.html#numbers-number

Python docs says float are real numbers and int/bool are integral numbers, which fair enough.

>>> import numbers
>>> isinstance(42, numbers.Integral)
True

>>> isinstance(42, numbers.Real)
True

>>> isinstance(42.5, numbers.Integral)
False

>>> isinstance(42.5, numbers.Real)
True

>>> isinstance(42, float)
False
Python documentation
3. Data model

Objects, values and types: Objects are Python’s abstraction for data. All data in a Python program is represented by objects or by relations between objects. (In a sense, and in conformance to Von ...

#

I do remember reading somewhere that checking for instances of numbers.Number and such is not recommended. Can't remember why, but either way, I'm doubly sorry.

trim tangle
#

No need, it's a part of the type system that I think is not good. Making one type a virtual subtype of another as a lie for supposed convenience

#

Fun fact: int has an is_integer method for compatibility with floats

median gulch
# trim tangle which is stinky by itself, but to make things worse, mypy doesn't fully understa...

That's why I believe you either duck type or statically type. Having a implements alternative for isinstance would be great. I still don't know why can't we generate runtime-checkable Protocol objects. Even if they hard coded a rule where the class needed to be type hinted from top to bottom, like they hard coded types.SimpleNamespace despite the implementation being technically trivial. Obviously, it is way more complicated than that, it always is, but one can wonder.

trim tangle
#

!d typing.runtime_checkable

rough sluiceBOT
#
typing.runtime\_checkable 

@typing.runtime_checkable```
Mark a protocol class as a runtime protocol.

Such a protocol can be used with [`isinstance()`](https://docs.python.org/3/library/functions.html#isinstance) and [`issubclass()`](https://docs.python.org/3/library/functions.html#issubclass). This raises [`TypeError`](https://docs.python.org/3/library/exceptions.html#TypeError) when applied to a non\-protocol class. This allows a simple\-minded structural check, very similar to “one trick ponies” in [`collections.abc`](https://docs.python.org/3/library/collections.abc.html#module-collections.abc) such as [`Iterable`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterable). For example...
trim tangle
#

^but this does exist

#

And there's also the classes from numbers that will also support int and float and custom classes registered with these ABCs (numbers from numpy probably are)

cinder bone
trim tangle
#

3.12

#

int still doesn't have hex, so it's not fully compatible with float

median gulch
#

Or maybe

implements(int.__add__, 42.5) # True
implements(list.__add__, 42.5) # False
#

Because classes are hashable since type is always hashable. And Callable[[int], int] could be like ((int,),int) which is hashable too. So theoretically, could hash(signature(int.__add__)) == hash(signature((42.5).__add__))?

fluid jay
#

question on type hinting an iterator

class PostEmbedImageIterator(Iterator):
    def __init__(self, embed: dict):
        super().__init__()
        self._embed_images: list = embed.get("images")

    def __next__(self) -> EmbedTypeImage:
        try:
            item: dict = self._embed_images.pop(0)
            return EmbedTypeImage(item)
        except IndexError:
            raise StopIteration
...
match embed:
  case PostEmbedImageIterator():
      for image in embed:
          print(image)

PostEmbedImageIterator() produces EmbedTypeImage. i expect the LSP to catch this but it doesn't. is there a way to typehint the product of a custom iterator like this?

trim tangle
fluid jay
fluid jay
trim tangle
# median gulch Wheter it does return an int, is not the point. You can run `inspect.signature` ...

It might be easy for when you expect an int and the method signature specifies a return of int. But:

  • Some types are more complicated. What if you expect an int | str and the implementor has int? What about list[int | str] and list[int]? So now you'll need to implement an entire type checker at runtime, that would understand stuff like recursive types and variance. And it would do that in a single check that would seem to be cheap computationally.
  • Signature information can be missing or misleading. For example, contextlib.contextmanager preserves the annotations of a method. So here:
@contextmanager
def open_s3_file(s3: S3, path: str) -> Iterator[S3File]:
    ...
``` the `open_s3_file` function actually returns a `ContextManager[S3File]`, but inspecting its signature will say that it returns an `Iterator[S3File]`. and other decorators may not call `@functools.wraps` so the annotations will just be missing
- Would the check return `False` if the class doesn't have type annotations? For example, `int` and `float` don't have any type information at runtime, it's all in the stubs. (and there will be users of your library that don't want to use type annotations)
median gulch
trim tangle
fluid jay
trim tangle
#

I hope you don't have list.pop(0) in a loop in production code, because that's quadratic 👀

fluid jay
#

what i need is a FIFO queue
i have no idea if stdlib python has it

#

a quick google search doesn't turn it up
but i may be looking at the wrong place

trim tangle
median gulch
fluid jay
trim tangle
# 
def get_images(embed: dict) -> Iterator[EmbedTypeImage]:
    return map(EmbedTypeImage, embed.get('images', ()))
fluid jay
#

wait map returns an iterator

trim tangle
#

yes, map is a class whose instances are iterators 🙂

#

map used to return a list in Python 2

median gulch
fluid jay
#

i haven't been programming at all for a couple of months look at me

trim tangle
#

of course, Python does not respect its own convention and uses smashedtogethercase for some classes

fluid jay
#

😆

median gulch
fluid jay
#

the golfing starts kek

median gulch
#

pythonistas will say you should never assign a lambda function, but they didn't see anything

trim tangle
#

If you want a loop, then you can do ```py
def get_images(embed: dict) -> Iterator[EmbedTypeImage]:
images = embed.get('images', [])
for raw_image in images:
# imagine something else
yield EmbedTypeImage(raw_image)

median gulch
trim tangle
#

oh right

#

I too haven't been programming for a while

median gulch
#

no prob

trim tangle
#

I got confused because initially wanted to show that you don't need a linked list or other style of queue for this, you can just make an iterator and next it

# 
class PostEmbedImageIterator(Iterator):
    def __init__(self, embed: dict):
        self._embed_images = iter(embed.get("images", ()))

    def __next__(self) -> EmbedTypeImage:
        item = next(self._embed_images)  # may raise StopIteration
        # imagine something else
        return EmbedTypeImage(item)
#

which is similar in spirit to storing the full list and keeping a counter

#

yet another alternative might be reversing the list and popping from the front ```py
class PostEmbedImageIterator(Iterator):
def init(self, embed: dict):
self._embed_images = embed.get("images", ())[::-1]

def __next__(self) -> EmbedTypeImage:
    if not self._embed_images:
        raise StopIteration
    item = self._embed_images.pop()
    # imagine something else
    return EmbedTypeImage(item)
fluid jay
#

took a bit to get it but it's simple

median gulch
#

honestly, @fluid jay , if you want typing, try a TypedDict or a dataclass with a method

class DatabaseObject(TypedDict):
  images: Sequence[bytes]
  def serialized_images(self) -> Iterator[EmbedTypeImage]:
    for img in self.get('images', ()):
      yield EmbedTypeImage(img)
trim tangle
#

can't have methods on a typeddict

median gulch
#

really?

#

dataclasses ftw then

trim tangle
# median gulch really?

a typeddict just describes the shape of a dict, e.g. ```py
class Point(TypedDict):
x: int
y: int

p: Point = {"x": 4, "y": 5}

trim tangle
median gulch
#

or go full overengineering and subclass list[EmbedTypeImage] just to override __init__

#

so much for

There should be one-- and preferably only one --obvious way to do it.

fluid jay
#

all in all today's a win, got this problem solved 👌

#

thanks guys

median gulch
#

you're welcome! 🤗

stiff acorn
#

What's the correct approach to running mypy over a library that supports python 3.9 to 3.13?

#

For example, I have pytest running for each python version

#

Should I do the same for mypy?

#

That is, run mypy for each python version? Or run it for the lowest only (3.9)? Or run it for the highest only (3.13)?

grave fjord
#

Using tox

#

It's a hassle, so it's common to run it for the latest version only

#

Or latest version for each platform

stiff acorn
#

Have you ever encountered a case where mypy succeeds on one version but fails on another?

oblique urchin
stiff acorn
#

How do you deal with them?

jolly cipher
#

good to know! :)

oblique urchin
# stiff acorn How do you deal with them?

you could have different pieces of code for different Python versions (with if sys.version_info checks), or you could write code in such a way that it works on all Python versions, by avoiding features that were added later than your earliest supported version

stiff acorn
oblique urchin
stiff acorn
#

I don't mind it being slightly slower so guess I'll do that. Thank you!

cinder glen
verbal spoke
#

Weird question: I have a file that currently uses the older type hinting style for optionals Optional[T], but I'm wanting to convert it to the modern style T | None. The file I have is quite large, and doing those changes by hand would be incredibly tedious. Is there a simple way to do it? I've futzed around with regext but I keep running into weird walls on it. Was wondering if there was a tool or script I could use to do it for me. For context on the current regex I have:

old = r"\bOptional\[(.+?)\]"
new = r"\1 | None"

That runs into problems with nested Optional conditions:

# this
Optional[list[T]]
# turns into this
list[T] | None

Thoughts? Suggestions?

#

(apologies for jumping in the middle while you were typing, yield)

#

I hit enter before I looked

cinder glen
#

It's ok, no worries! What I was saying was not very important anyway.

oblique urchin
verbal spoke
#

I tried the autofixes, however from what I found, it won't fix it since the only rule that comes close just doesn't like type hints that default to None but don't have any optional noted

#

So like it'll correct:

ham: int = None

But since that stuff is already hinted, it won't convert

oblique urchin
verbal spoke
#

The only rule I could find for it is RUF013

oblique urchin
verbal spoke
#

Unless I'm missing something

#

Derp

#

I'm blind then, apologies

#

I appreciate it. Sorry for the dumb question!

severe musk
#

Argument of type "BufferedReader" cannot be assigned to parameter "data" of type "ReadableBuffer"
this is not good python design

#

they sound like the same thing

severe musk
#

i am trying to calculate crc for a binary opened file, and I didn't want to do fp.read() as that puts the whole file in a bytes object which can be wasteful

tranquil ledge
tranquil ledge
severe musk
#

Using the entire file contents

tranquil ledge
#

Seems like it needs the whole file contents in memory, then. Idk enough to say otherwise.

severe musk
#

ReadableBuffer sounds exactly like an object that can be called .read

tranquil ledge
#

Ah, that is confusing. I see how you got from a to b now.

oblique urchin
#

Possibly not the best name, it's more like a "bytearray protocol", but it's an established name by now

severe musk
#

I see, google sent me to this page but it wasn't too convincing because the type 'hint' isn't mentioned

oblique urchin
#

ReadableBuffer is specific to typeshed. The buffer protocol was initially C-only; PEP 688 introduced a Python visible way to use it, but it doesn't support distinguishing between readable and writable buffers.

tranquil ledge
#

Hmm. I have a bit of a tangential question: it seems that in my installations of python 3.12 and 3.13, the common builtin buffer types like bytes, bytearray, and memoryview aren't subclasses, virtual or otherwise, of collections.abc.Buffer, or don't pass the getattr check to match that "protocol". However, typing-extensions registers those three as virtual subclasses of its Buffer placeholder.

#

Am I missing a reason for that seeming lack of equivalent behavior? Or maybe is my installation borked?

#

Belay that, I might just be dumb. Never mind.

severe musk
#

did you check isinstance instead of issubclass

tranquil ledge
#

A bit late, but yes, eventually facepalm

severe musk
#

that's fine, cool that there is a base abstract class though

tranquil ledge
#

Aye. I like that it doesn't rely on subclass registration to make isinstance/issubclass work with the stdlib concrete implementations.

severe musk
#

My preference when making apis is BytesIO though

#

File oriented api

tranquil ledge
#

Fair. Seems flexible in some ways.

wary pulsar
#

Hi all. I tried asking this as general question but didn't get much so I'll ask here (slightly modified). I'm wondering why pyright gives the error:

error: Overload 1 for "myfn" overlaps overload 2 and returns an incompatible type (reportOverlappingOverload)

when I run it against this code:

from typing import Literal, overload

import numpy as np
import numpy.typing as npt

from typing import reveal_type

@overload
def myfn[T: np.generic](
    values: npt.NDArray[T],
    other_arg: int = 0,
    copy: Literal[False] = False,
) -> None: ...
@overload
def myfn[T: np.generic](
    values: npt.NDArray[T],
    other_arg: int = 0,
    copy: Literal[True] = ...,
) -> npt.NDArray[T]: ...
def myfn[T: np.generic](
    values: npt.NDArray[T],
    other_arg: int = 0,
    copy: bool = False,
) -> npt.NDArray[T] | None:
    output = values.copy() if copy else values
    # ... mutate output, maybe using other_arg ...
    # Nothing to return if not copying
    return output if copy else None

reveal_type(myfn(np.arange(10), copy=False))  # Correctly reveals as None
reveal_type(myfn(np.arange(10), copy=True))  # Correctly reveals as an ndarray
oblique urchin
#

only one of them (the one that gets picked if the parameter is omitted) should have it