#type-hinting

1 messages · Page 76 of 1

grave fjord
#

Can't even annotate Coroutine properly

trim tangle
grave fjord
#

Mostly have to settle for Callable[P, Coroutine[Any, Any, None]]

trim tangle
#

Ah

grave fjord
#

The send type is implementation specific

#

And you have to fit with async def ...(...): ... where it's a faff to specify

trim tangle
#

Actually, it's notable that a generator function's return type is properly specified

#

But coroutine functions are different for some reason

grave fjord
#

Guido said so

trim tangle
#

libraries could export their own coroutine type like asyncio.Coroutine, which would be a generic type alias

#

Side disappointment: no session types for generators

#

You can't type "first it receives an int, then a string"

#

Or "if it receives an int, it sends a string"

soft matrix
#

how would you track that?

#

the first one of those

trim tangle
#

That's indeed an interesting research problem

soft matrix
#

the second should be possible with conditional types

trim tangle
oblique urchin
#

that sounds like linear types, right?

trim tangle
#

That's called "session types"

soft matrix
#

but the call to send couldnt know if its been called before right?

trim tangle
trim tangle
#

In Rust session types were indeed implemented with this linearity

#

I have some weird idea with exceptions but it's 4am so I'll think about it tomorrow

soft matrix
#

it is tomorrow lemon_wink

oblique urchin
#

It's not really safe but it works fairly well in practice

#

I think mypy has a more limited version of this too ("partial types")

rose root
grave fjord
#

Like if your send type isn't optional you can't actually use it

green gale
#

how do type checkers like mypy and pyright statically analyze typing.Unions?

#

I've been trying to think through it but I'm unable to come up with a good solution.

rare scarab
#

I'm getting this error in mypy on a docstring typealias. Bug?

ContextLike: te.TypeAlias = """
    Context
    | t.Callable[[], Context]
    | t.Callable[[Template], Context]
"""

error: Invalid type comment or annotation
oblique urchin
oblique urchin
green gale
#

ah so it's like, an, uh... "opaque type"

#

I see

oblique urchin
#

why opaque?

green gale
#

that's lingo from another language with traits, that probably doesn't apply to python

#

but the wording sounds similar enough

rare scarab
green gale
#

but like, how would mypy determine that

gotten = some_dict.get("some key")
if gotten is not None:
    gotten.join([])

is valid? (this is some terrible code but whatever)

oblique urchin
#

mypy uses something called the binder for that but I've never looked into how it works

green gale
rare scarab
oblique urchin
#

in the type checker I wrote (pyanalyze) the equivalent internal structure is called a constraint. When you access a variable, it looks up all places where that variable may have been assigned to. In principle, the inferred type is the union of the types at all assignments. But with something like if x is not None, a constraint is placed on top of the original type. That's basically a transformation of the type, so if the original type is str | None, it filters out the None and gives you just str.

green gale
#

I might have to look at that code, is it open source?

oblique urchin
green gale
#

thanks, I'll take a look!

blazing nest
#

Think of a Venn diagram, the type that Union[...] creates is the middle of the two circles

wicked scarab
#

How do type checkers work (how can it know what type is something not at runtime)?

acoustic thicket
#

it trusts that your annotations are right

#

the types of literals, signatures of builtin and standard library functions etc are already known

wicked scarab
#

Do they parse the code manually?

#

To get the annotations and stuff

acoustic thicket
#

i believe so

wicked scarab
#

Oh I see, okay thanks

dull lance
#

I want to type hint any function with a json parameter (it can be anywhere in the parameter list, as a positional or keyword argument). Is there a more concise way to do it than this?

from typing import Any, Protocol, Union

class _AcceptsJSON_NoArgsKwargs(Protocol):
    def __call__(self, json: Any) -> Any: ...

class _AcceptsJSON_ArgsOnly(Protocol):
    def __call__(self, *args: Any, json: Any) -> Any: ...

class _AcceptsJSON_KwargsOnly(Protocol):
    def __call__(self, json: Any, **kwargs: Any) -> Any: ...

class _AcceptsJSON_WithArgsKwargs(Protocol):
    def __call__(self, *args: Any, json: Any, **kwargs: Any) -> Any: ...

_AcceptsJSON = Union[_AcceptsJSON_NoArgsKwargs, _AcceptsJSON_ArgsOnly, _AcceptsJSON_KwargsOnly, _AcceptsJSON_WithArgsKwargs]
trim tangle
dull lance
#

It is used in a decorator which validates the json from a user request against the provided schema, then passes that json to the function

#

it looks something like this:

def requires_json(schema: Any):
    def wrapper(view: _AcceptsJSON) -> Callable[..., Any]:
        @wraps(view)
        def wrapped_view(*args, **kwargs):
            json = request.json
            if json is None:
                abort_with_json(HTTPStatus.BAD_REQUEST, 'Missing JSON')

            try:
                validate(instance=json, schema=schema)
            except ValidationError:
                abort_with_json(HTTPStatus.BAD_REQUEST, 'Invalid JSON')

            return view(*args, json=json, **kwargs)
        return wrapped_view
    return wrapper
#

However, that function may receive other parameters (from other decorators that similarly obtain relevant values from the request), so I need to allow the input function to be as general as possible

trim tangle
soft matrix
#

The methods that intersect are the ones that can be accessed and used without narrowing but the original types are still there

dull lance
brisk heart
#

Wouldn't something like __call__(self, json: object, *args: P.args, **kwargs: P.kwargs) -> object:

trim tangle
#

But I'm not sure.

#

You can definitely do this with a TypeVarTuple if you're okay with positional args instead

dull lance
#

Actually I just realized that even my example above doesn't really do what I want, since it is still limited to positional arguments being passed as *args and keyword arguments being passed as **kwargs. So something like def my_func(a: int, json: Any) -> Any: ... won't match

dull lance
#

Examples of functions that should match:

def test1(a: int, json: Any) -> Any: ...
def test2(json: Any, b: int) -> Any: ...
def test3(a: int, json: Any, b: int) -> Any: ...
def test4(*, json: Any) -> Any: ...
def test5(json: Any, **kw) -> Any: ...
def test6(*, json: Any, **kw) -> Any: ...
def test7(a: int, *, json: Any) -> Any: ...
def test8(json: Any, b: int, **kw) -> Any: ...
def test9(a: int, *, json: Any, b: int, **kw) -> Any: ...
def test10(*, a: int, json: Any, b: int, **kw) -> Any: ...
#

basically I don't care where is json in the parameter list as long as it is not a positional-only argument (since I am calling the function in the wrapper using keyword arguments)

devout barn
#

is there a way to somehow typealias something which contains a ClassVar
ie how to make this valid :

Slots: TypeAlias = ClassVar[tuple[str, ...]]
# 
"ClassVar" is not allowed in this context
        - Pyright

pyright not happy

trim tangle
#

I don't think so

#

Why do you want this?

devout barn
#

so doing this works

Slots = ClassVar[tuple[str, ...]]


class Thing:
    __slots__: Slots = ()
#

that's strange pithink

dull lance
devout barn
#

could this be a bug in pyright?

#

because

Slots = ClassVar[tuple[str, ...]] # Okay

and

Slots: TypeAlias = ClassVar[tuple[str, ...]] # Not Okay!
dull lance
#

but I can't put json after *args to match a in test1 and test3

#

if that were possible I believe I can match all of them with a second protocol (def __call__(self, *args: P.args, json: Any, **kwargs: P.kwargs) -> Any:)

near night
#

Anyone know how to handle very big ints in python 2.7? I just need to multiply some stuff together, but int32 is not big enough

dull lance
dull lance
#

so now I have

class _AcceptsJSON_ArgsOnly(Protocol):
    def __call__(self, *args: Any, json: Any) -> Any: ...

class _AcceptsJSON_WithArgsKwargs(Protocol):
    def __call__(self, *args: Any, json: Any, **kwargs: Any) -> Any: ...

_P = ParamSpec('_P')
class _AcceptsJSON_Spec(Protocol[_P]):
    def __call__(self, json: Any, *args: _P.args, **kwargs: _P.kwargs) -> Any: ...

_AcceptsJSON = Union[_AcceptsJSON_ArgsOnly, _AcceptsJSON_WithArgsKwargs, _AcceptsJSON_Spec[_P]]

which is a good start

native oak
blazing nest
#

Anyone ever had similar issues with Pylance/Pyright?

unknown.png
#

!e ```python
import re

class Test:
def regex(self, pattern):
def decorator(func):
if isinstance(pattern, str):
pattern = re.compile(pattern)

        print(pattern)
        return func
    return decorator

@Test().regex('abc')
def test(): ...

brisk heart
#

is this something to do with every variable getting turned unknown?

rough sluiceBOT
#

@blazing nest :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 16, in <module>
003 |   File "<string>", line 7, in decorator
004 | UnboundLocalError: local variable 'pattern' referenced before assignment
blazing nest
#

Wut

brisk heart
#

ah wait

#

gotta declare nonlocal

blazing nest
#

OOooOooh, thanks!

#

I think I've accidentally been testing it without the if-statement

grave fjord
#

Really wish python declarations had the opposite default like typescript modules

soft matrix
#

whats that?

grave fjord
#

Everything defaults to nonlocal

#

In modules if there's no parent scope it's a syntax error

#

We don't talk about scripts....

#

You need to pick if it's a variable, constant or lettuce

trim tangle
#

Lettuce?

oblique urchin
#

let-bound variable?

grave fjord
#

The var keyword makes variables

#

The const keyword makes constants

#

The let keyword

#

Well, I'll let you decide what that does

#

I'd love a flag that enforced Final and autofixed it like eslint does

green gale
#

why it's used alongside const and whatnot in JS? dunno

grave fjord
#

Well in other functional programming languages let creates a thunk

green gale
#

right, but the name itself does read /sort of/ like english. why it means what it does in js/ts I don't know.

grave fjord
#

That's just another Eich to scratch I guess

trim tangle
#

I wonder why not mut

green gale
#

wdym by "default"?

#

the return type of a function that doesn't explicitly return a value is None

oblique urchin
#

type checkers will generally assume that a function without a return annotation returns Any

#

though pyright will infer the return type

green gale
#

what type checkers do/don't have inference?

#

personally I think inference is kind of a necessary tool if you don't want type checking to be a pain

oblique urchin
#

mypy doesn't infer return types

#

all type checkers have some amount of inference

green gale
#

are there any particular challenges posed by inferring return types?

#

or was it just a decision not to/hasn't been implemented

void panther
#

I recently found out pyright doesn't infer a list's type from what's appended to it, that was a bit surprising at first

blazing nest
rare scarab
#

I think pyright does have an internal Module["modulename"] type

#

It's not part of typing or typing_extensions though, so that sucks. and similarly named types.Module isn't generic.

soft matrix
#

types.Module isn't a thing

#

It's ModuleType

rare scarab
#

nice catch

#

it's still useless for this situation

soft matrix
rare scarab
#

I've seen that, yes.

#

I read typing-sig sometimes

#

new typevar syntax is what everyone is focusing on right now

acoustic thicket
#

@trim tangle pyright playground is down

unknown.png
#

ignore me

hollow osprey
#

Is there a way to annotate something with the common types of two Unions?

# 
a = Union[A, B, C]
b = Union[B, C, D]

c = Union[B, C]
``` not sure if i'm making any sense
#

I couldn't find any "Intersection" class/method in the typing docs

soft matrix
#

Intersection isn't a thing in typing yet

trim tangle
soft matrix
#

They thought site was down

acoustic thicket
#

yeah

trim tangle
#

ah ok

#

sometimes I forget to pay for the VPS lmao

#

but this time it's ok

soft matrix
slender timber
hollow osprey
slender timber
hollow osprey
#

oops wrong reply

#

I need it to be and, not or

slender timber
#

Yea its equivalent

trim tangle
#

I don't see how a TypeVar would help

#

A TypeVar is for linking two types together. If you want a type alias, that would be just A = Union[str, bytes]
As Gobot said, this is not possible to do automatically right now

hollow osprey
# 

a = Union[A, B, C]
b = Union[B, C, D]

c = Union[B, C] # what i need
trim tangle
#

can you give a more concrete example of why you need this?

hollow osprey
trim tangle
#

maybe there is a way to solve the bigger problem

hollow osprey
#

I think it'll be less work if I manually create a type alias and use it throughout

proud brook
trim tangle
#

that wouldn't make sense 🤔 there are voice channels which are not messageable

proud brook
#

I'm pretty sure all channel types are Messageables, well, maybe except VoiceChannel

soft matrix
#

VoiceChannel is messageable now

#

Cause there's the chat that vcs have

trim tangle
#

I guess, yeah

hollow osprey
#

Well that does solve all my problems then 😅

trim tangle
#

but

#

and aren't categories modelled as channels in discord.py?

soft matrix
#

Well this is only if you're using v2

hollow osprey
#

StageChannels are non-Messageable GuildChannels too right

#

Yep, I'm on V2

proud brook
trim tangle
#

Given: ```py
class Animal: ...
class Cat(Animal): ...

animals: list[Animal] = [Animal(), Animal()]
What's a good way of explaining why the following is okay:
animals.append(Cat())
``` even though list is invariant?

#

I'm thinking of some "assignability chain" like this: ```py
cat: Cat = Cat()
animal: Animal = cat

animals.append(animal)

proud brook
#

Isn't it okay because it's a subclass

trim tangle
#

yeah but list is invariant, and list[Cat] is not a list[Animal]

proud brook
#

Is Animal invariant

trim tangle
#

it's not generic

proud brook
#

Of course it is not generic

trim tangle
#

I think I don't understand you then

#

variance is a property of a generic type

proud brook
#

Well maybe Animal is generic

#

A generic Animal type

#

I'm not big on type hinting

proud brook
#

What is Key? You might want to type hint it that

#

Then, yes

#

Just with Key I think

trim tangle
#

Union[str, Key]

#

or yes, str | Key if you're on 3.10+

#

sorry, I'm bad at reading

#

I suppose

#

it will not work in type aliases though

#

You can extract a type that you're using frequently like this:

OnNewItem = Callable[[MenuItem], Result]  # this is a type alias

def order_pizza(big: bool, callback: OnNewItem) -> None:
    ...

def order_donuts(count: int, callback: OnNewItem) -> None:
    ...
#

because it's just assigning something to a variable, from __future__ import annotations doesn't impact it

#

so e.g. Input = str | Key will not work, you'll have to use Input = Union[str, Key]

#

therefore I'd personally use Union in codebases not yet on 3.10, to avoid being inconsistent

rare scarab
rare scarab
#

Vscode will properly highlight it

trim tangle
#

hmm

#

I guess

trim tangle
#

because you won't be able to [] it

rare scarab
#

You just have to put everything in quotes

trim tangle
#

oh wait, you will

#

yeah

oblique urchin
#

or you can just use Union[str, int] and not need quotes

trim tangle
#

yeah, I find the quotes a bit annoying

#

I don't think I have a rational argument against them, they're just not beautiful

rare scarab
#

It would also be nice if mypy supported multi line strings in type aliases. It makes making complex unions much easier

trim tangle
#

I think the maintainer already explained that it's silly to import TypeAlias from a 3rd-party library

#

Pyright devs seem toxic, so it's little surprise they refuse to fix their blatantly broken and PEP-noncompliant behaviour
Eric is just not wanting to supporting a very fringe use case that would be tricky to implement. This person goes on to say they were making that decision while overdosed on some anxiety medicine? And now Eric is "toxic" somethow??

rare scarab
#

That issue got a bit off topic with the retro game cartridge adapters

trim tangle
#

Also, I just checked. Beartype achieves O(1) type checking by checking only one element from a nested collection? Or did I miss something?

blazing nest
#

I don't get this issue though...

#

The author defends their own (and Mypy's) missing support for recursive types because it would be difficult to implement.. yet they complain that Pyright doesn't implement specific things which it reasons the same way.

#

They also go on about Mypy being so incredibly great because of how it implements into CI and then complain about Pyright not being great for CI. The thing is... they admit to not even trying to implement Pyright with their CI.

"I don't like X because I don't know how to do Y even though I have made no efforts into learning how"

trim tangle
#

I think they have pyright in the CI now

blazing nest
#

Pyright bad because it requires node but Mypy is good because it requires Python?!

rare scarab
#

How would I type a Callable that takes **kwargs: Any?

trim tangle
rare scarab
#

Any way to make the names of positional arguments not matter without using arg1, /, **kwargs? (requires py3.8)

void panther
#

__ prefix I believe

rare scarab
#

thanks

brisk heart
#

Is there any list of the changed internals of typing?

#

ideally 3.8 to 3.11

summer berry
#

I suppose the commit history is that.

brisk heart
#

it's just cause I keep trying to depend on inconsistent stuff

#

and it sucks

slender timber
#

typing changes a lot

#

typing_extensions is almost always a dependency for me

frigid shadow
# 
def my_json_file() -> Mapping[str, Union[int, str, List[int, str, List[...], Mapping[str, ...]], Mapping[str, ...]]]:
    with open('index.json', mode='r', encoding='utf-8') as file:
        ret = _load(file)
    return ret

why this is giving me an error, may someone help me please?

buoyant swift
#

List only takes 1 type. did you want to type each element in the list? that should probably be a tuple then

soft matrix
#

Mapping[str, ...] isn't a valid type

#

Realistically the correct thing to do here is use a TypedDict

#

!d typing.TypedDict

rough sluiceBOT
#
typing.TypedDict 

class typing.TypedDict(dict)```
Special construct to add type hints to a dictionary. At runtime it is a plain [`dict`](https://docs.python.org/3/library/stdtypes.html#dict "dict").

`TypedDict` declares a dictionary type that expects all of its instances to have a certain set of keys, where each key is associated with a value of a consistent type. This expectation is not checked at runtime but is only enforced by type checkers. Usage...
frigid shadow
#

thx

#

im new to python xd...

summer berry
#

Is there a way to annotate a dictionary based on the kwargs of a function? For example, a dict d should be of the same type as the kwargs of function f. I don't need it to be generic.

trim tangle
#

using Unpack and a TypedDict

summer berry
#

I think I get what you mean. Didn't think about it like that.

#

Still I would prefer a way to use the arg types directly from the function definition rather than defining a separate data structure for them

#

Are there technical reasons for why that'd be difficult for a typechecker to do?

#

Cause a feature like that seems like it would make working with kwargs easier.

trim tangle
#

This would require some kind of type-level typeof from TypeScript

summer berry
#

Well a function is a type, and that contains type information about its parameters, right?

trim tangle
#

there's currently no mechanism to extract the function type from a function

#

or from any other variable

summer berry
#

Okay, makes sense.

trim tangle
#

idk if it's actually hard to implement

soft matrix
#

It can't be that bad considering reveal type exists

summer berry
#

Stems from a situation like this ```py
def test_foo(self):
params = (
(dict(a=1, b=2), "12"),
(dict(a=3, b=4), "34"),
)

for kwargs, expected in params:
    with self.subTest(**kwargs):
        actual = foo(**kwargs)
        self.assertEqual(actual, expected)

Would be nice if the typechecker knew the `dict` in `params` had to match the parameters of the `foo` function, and that the string had to match its return type. That way it could also complain when a required arg is missing.
trim tangle
#

TypeScript elegantly solves this problem by not having kwargs 🙂

#

but you can get the parameters of a function ```ts
const foo = (bar: number, baz: string[]): void => {

}

type FooParams = Parameters<typeof foo>
// FooParams: [number, string[]]

summer berry
#

Yeah, I like how powerful that stuff is in TS

frigid shadow
#

whats the different between typing.Dict, typing.Mapping and typing.TypedDict?

void panther
#

Dict is a proper dict, Mapping is just the mapping interface, and TypedDict is a dict but allows you to type individual keys

frigid shadow
#

thx

void panther
#

dict and Mapping would be like list and Sequence, that may be a simpler example of a similar relationship

frigid shadow
#

im new to python xd

molten mason
# 
class Converter:
    @classmethod
    def convert(cls, record: DotRecord):
        values = {}
        for tup in record.items():
            values[tup[0]] = tup[1]

        return cls(**values)


class User(pydantic.BaseModel, Converter):
    user_id: int
    nickname: str

How can I type the return of convert to be whichever class I'm calling this "abstract" convert method from?
example, if I do User.convert(...) the return type will be User and so on.

summer berry
#

typing_extensions.Self is what you're after I believe

frigid shadow
#

mypy is giving me error error: Module "discord.ext.commands" has no attribute "Context" but theres no error when i import Context from discord.ext.commands, how to fix that? may someone help me please?

slender timber
#

If there is TypedDict then what is typing.Dict?

soft matrix
#

In newer versions it's basically just builtins.dict

tranquil cobalt
#

Hello, I don't know if this should be here, I'm reading and doing some study about deep learning with Python (François Chollet) and I see he represent any elements (more than one) in python with "()" like [(value, key) for (key, value) in word_index.items()] but in my work or other codes I see in the net I see [(value, key) for key, value in word_index.items()] what is the most correct way? I also see it in other places for the imports like from x import (a, b, c) both ways work but would like to know what is the most pythonic or PEP way

trim tangle
tranquil cobalt
restive warren
#

Is there a good way to type an argument as "one of two functions"? Right now I've got it type hinted as a callable, and the docstring explains what is expected

#

To clarify:

#

current:

def a(arg1, arg2):
  ...
def b(arg1, arg2):

def function(func: Callable[..., return]):
  ...
#

And I'd like:

grave fjord
#

You can use NewType to specify those a and b

restive warren
#

How would that look like?

trim tangle
restive warren
#

The function is basically a glorified wrapper around that func

#

It adds some logging and error handling

trim tangle
#

can you show an example maybe?

restive warren
#

Yeah it's from the bot's code base

#

One sec

rough sluiceBOT
#

botcore/utils/members.py lines 31 to 35

async def handle_role_change(
    member: discord.Member,
    coro: typing.Callable[..., typing.Coroutine],
    role: discord.Role
) -> None:```
restive warren
#

It's coro

trim tangle
restive warren
#

Ideally I'd like to document it as discord.Member.remove_role | discord.Member.add_role

#

It's a minor thing, but it helps the generated docs look better

viral bramble
#

I'd pass a partial 😄

trim tangle
#

but it does work with other functions, right?

restive warren
grave fjord
# 
AorBProtocol: TypeAlias = Callable[[T, U], Coroutine [Any, Any, object]

_A = NewType("_A", AorBProtocol)
_B = NewType("_B", AorBProtocol)

def _a(...):...
def _b(...): ...
a = _A(_a)
b = _B(_b)

async def needs_aorb(fn: _A | _B)-> ...:
    ...
restive warren
trim tangle
#

If it's so strongly coupled to those functions, I'd just make two functions, like safe_add_role and safe_delete_role

grave fjord
#

Or use an enum

restive warren
#

I've tried an enum, and that does work but it is even more difficult for the purposes of the docs

grave fjord
#

And have a dict of enum key to the function

restive warren
#

I'm trying out the NewType solution now

trim tangle
#

why not two functions? it's probably the simplest solution

restive warren
#

I'm not really sure, probably because someone was trying to simplify things

trim tangle
# trim tangle why not two functions? it's probably the simplest solution

basically, make the current function private (underscored) and do

async def safe_add_role(member: discord.Member, role: discord.Role) -> None:
    await _handle_role_change(member, discord.Member.add_role, role)

async def safe_delete_role(member: discord.Member, role: discord.Role) -> None:
    await _handle_role_change(member, discord.Member.delete_role, role)
#

alternatively, if all this function does is handle errors, you could make a context manager:

with suppress_role_errors(lemon):
    await lemon.add_role(roles.BANNED)
# 
@contextlib.contextmanager
def suppress_role_errors(member):
    try:
        yield
    except discord.NotFound:
        log.error(f"Failed to change role for {member} ({member.id}): member not found")
    except discord.Forbidden:
        log.error(
            f"Forbidden to change role for {member} ({member.id}); "
            f"possibly due to role hierarchy"
        )
    except discord.HTTPException as e:
        log.error(f"Failed to change role for {member} ({member.id}): {e.status} {e.code}")```
#

although it does require you to provide the member twice.

#

That seems like a strange function tbh...

restive warren
#

The newtype solution is really cool, but I'd probably get slapped if I tried PRing that in haha

trim tangle
#

yeah it sounds overengineered

restive warren
#

It's only role that's passed to coro (despite the wrong docstring)

trim tangle
#

I think two functions are easier to understand than a higher-order function or a newtype

restive warren
#

I'm probably going to leave it as is because I'm lazy and don't want to break the projects using this

#

This has been interesting, thanks for all your answers

trim tangle
restive warren
#

I think this just broke when being ported over /shrug

trim tangle
viral bramble
#

Pass log_info: str 😄

restive warren
#

I think it could work since it's actually role that the coro needs, and member doesn't get used

#

But also same breaking upstream projects problem

viral bramble
#

I wrote something similar recently - the purpose was to re-try the HTTP request with expo backoff on certain httpx types of errors - and I made the executor take a ready-made partial and extra info to include in error logs

#

but this fn in particular is highly coupled to the domain via errors types, anyway

#

I also like the idea of passing an enum, e.g. RoleOp.ADD | RoleOp.REMOVE, which makes sense if it's already coupled

#

since here the executor already knows about the domain, it might as well be responsible for choosing the right function

restive warren
#

Would that cause typing issues upstream if the function was not passed through the enum?

#

The function is from a third party library

#

So in userspace it's accessed through there

viral bramble
#

Sorry, I'm not sure what you mean

#

The callsite would be perform_op(RoleOp.ADD, role, member)
inside the function, you'd have e.g.

fn_map = {
  RoleOp.ADD: discord.add_role,
  RoleOp.REMOVE: discord.remove_role,
}
fn = fn_map[op]
restive warren
#

If the caller did perform_op(discord.remove_role, ...), would that create a typing warning

void panther
#

you could use the enum's value for the func instead of a dict

void panther
restive warren
#

Ah

trim tangle
#

yeah that's the point

#

I like the separate functions approach because there's just no way of using it wrong
But it does require an interface change

viral bramble
#

i.e. "just tell me what you want to do, not how"

#

on the other hand, if the value is the function, then you won't end up in a situation where you add a RoleOp but don't adjust the implementation in perform_op

blazing nest
#

Hm, discussions aren't enabled on the typing_extensions repository and I don't think this is an issue.

What is left for another release? 3.11 is feature frozen and there's been no activity for a month. The runtime support for generic typed dicts is a blocker for a project of mine.

#

I meant to ping @oblique urchin ^

oblique urchin
blazing nest
soft matrix
#

undecided if this is a typeshed bug or not```
Argument of type "Type[Games]" cannot be assigned to parameter "iterable" of type "Iterable[_T@get]" in function "get"
"iter" is an incompatible type
Type "(cls: Type[Games]) -> Generator[Games, None, None]" cannot be assigned to type "(self: Self@Iterable[_T_co@Iterable]) -> Iterator[_T_co@Iterable]"
Parameter name mismatch: "self" versus "cls"

#

im thinking this is because the iterable protocol surely doesn't need the first parameter to be called self for it to work in all cases right?

rare scarab
#

Will people still use typing-sig if the github has discussions?

soft matrix
#

no, the github already has discussions

rare scarab
soft matrix
#

an iterable metaclass

#

its just a faster version of enum

rare scarab
#

So basically this? ```py
class IterType(type):
def iter(cls):
yield something

soft matrix
# 
class EnumMeta(type):
    def __iter__(cls) -> Generator[Enum, None, None]:
        yield from cls._member_map_.values()

class Enum(metaclass=EnumMeta): ...
class Games(Game, Enum):
    TF2 = "Team Fortress 2", 440
    ...
#

i could rename cls here

#

but i think its better this way

trim tangle
#

interesting

#

technically, self is usually not marked as position-only

#

so if you declare

class Foo(Protocol):
    def bar(self) -> None:
        ...
```, you should be able to do ```py
def hmmm(foo: Foo) -> None:
    type(Foo).bar(self=foo)
``` unlike: ```py
class Foo(Protocol):
    def bar(self, /) -> None:
        ...
#

I wonder if that should be special cased.

soft matrix
#

thats what im wondering

trim tangle
#

I would make a discussion on typing

soft matrix
#

or just thinking it should be __iter__(self, /)

signal sentinel
#

The / represents placeholder for parameters right?

trim tangle
stray summit
#

anyone aware if there is a way to combine a generic with a union in such a way that the result is a untion of generics with the elements applied (Union[int, str] @ Set == Union[Set[int],Set[str]])

soft matrix
#

Might be possible using TypeForm (when it happens)

#

Currently I don't think it is

stray summit
#

TypeForm?

#

hmm, idea - maybe a bound covariant typevar will do

#

is it a mypy bug, that when given a typevar, isinstance no longer type guards?

class HasId(Protocol):
    id: str

HOST_OR_ID = TypeVar("HOST_OR_ID", HasId, str)
HOST_OR_HOSTS = Union[Set[HOST_OR_ID], Sequence[HOST_OR_ID], List[HOST_OR_ID], HOST_OR_ID]

def _id_from_host(host: HOST_OR_ID) -> str: 
    return host if isinstance(host, str) else host.id # errors that str cant have a id
hearty shell
#

Looks like it, in pyright this typechecks, so maybe mypy just has a problem with the "negative case" with typevars

#

if you invert it and use runtime_checkable it should work

soft matrix
stray summit
#

i resolved it by using only the union in the main function, now it all works out

frigid shadow
#

is that correct?

def main(x: T, y: V) -> List[Union[T, V]]:
  return [x, y]
hearty shell
#

Yes, that is also the default behaviour for pyright (not mypy though)

frigid shadow
#

im using mypy...

hearty shell
#

It is correct for both, It was just a side note

frigid shadow
#

oh

#

thx

hearty shell
#

Like main(x: T, y: T) -> List[T] main(3, "foo") will give you List[Union[int, str]] in pyright while it will give you List[object] in mypy

#

But using different typevars like you did will give you an union on both

frigid shadow
#

thx, im new to python so im noob :/

spice oak
#

is there a way to make mypy smarter

unknown.png
oblique urchin
spice oak
#

is there a way without that

#

because even vscode understands that hello returns a str

oblique urchin
#

there's no way to make mypy infer the return type if that's what you're asking

hearty shell
#

Yes, by using pyright instead, but type inference isnt that greate across the board

spice oak
#

ok thanks

trim tangle
#

If you're making a library, you shouldn't rely on inference for interfaces (arguments, return types, attribute types etc.). PEP 484 doesn't constrain inference in any way, so you will end up with different types in different type checkers

vivid lance
#

hint: my word is 4 letters and has 2 o’s in the middle

rare scarab
#

if the answer isn't type, how is this related?

plain dock
#

but it could be bool, which is a little related

rare scarab
#

I will guess noop

desert delta
#

can anyone explain why mypy doesn't like this?

from typing import Literal

Options = Literal['a', 'b']

def foo(x: Options):
    print(x)

def bar() -> None:
    for opt in 'a', 'b':
        foo(opt)
vivid lance
#

L

#

it was cool

#

book was a good idea tho

desert delta
rare scarab
desert delta
rare scarab
#

It's just manually defining the type of the list

#

You could also do for opt in typing.cast(list[Options], ['a', 'b']):

desert delta
#

yeah but I mean the failure of mypy to inspect a collection to check that the values are all members of my Literal

#

is that documented somewhere

rare scarab
#

pyright fails on it too

fathom river
#

is there a way I can achieve something similar to this?

def decorator(...):
    @overload
    def inner(fn: Callable[[str, str], int]):
        ...
    @overload
    def inner(fn: Callable[[int, int], str]):
        ...
    def inner(fn):
        # signature checking and whatnot
        ...
    return inner
rare scarab
#

they'll be some overhead

#

But using the typing.overload does no runtime checking

#

either way, you should o the checking inside the final function.

#

Are you trying to do type checking, or type coercion?

fathom river
#

type checking

rare scarab
#

pydantic is good for both things.

#

!pip pydantic

rough sluiceBOT
fathom river
#

hmm, alr, thanks

hallow flint
#

So decorator() -> InnerType where InnerType is a protocol that has your overloaded __call__

sterile python
#

Let's say I need to narrow type, is NewType with TypeGuard good practice to do this?

LeapYear = NewType('LeapYear', int)


def is_leap_year(year: int) -> TypeGuard[LeapYear]:
    ...
trim tangle
#

In this regard, a wrapper class is better if you want a stronger guarantee

cedar sundial
#

Is NewType used commonly? It just seems like an extra step to wrap items around which could easily be forgotten

trim tangle
#

I don't personally use it

#

I think Jelle mentioned using it for different types of IDs

sterile python
sterile python
#

I often see this practice in Haskell, but function and newtype instead of class

spice oak
#

is there a way to change the refactor category of error messages in pylint

trim tangle
#

in Haskell newtype actually makes sense because you can export the type but not the ability to construct it

sterile python
trim tangle
#

I meant that a module can export the type ProjectId, so e.g. an outside user can do foo :: ProjectId -> ProjectId.

#

But not necessarily export the constructor, so that an outside user can't do ProjectId (-42)

#

unlike with Python's NewType, where you have to export either neither or both

sterile python
#

I was not thinking in this way, that I can do

module Projects (ProjectId) where

To keep strictness of newtype, thanks you

blazing nest
spiral fjord
#

If you have TypedDicts etc that are used in multiple places in your code, is there a common way that projects use to place these in your project structure?

viral bramble
#

where I work - usually a domain or types module close to where they are used

spiral fjord
#

Thanks

grave fjord
oblique urchin
#

reopened

grave fjord
#

Thanks!

oblique urchin
grave fjord
#

A bit emojish for my taste, but setting the None here seems the best choice

oblique urchin
soft matrix
#

weird i noticed that today

summer berry
grave fjord
#

Ah I never look at the news filename 🤦

#

And github crops it out

summer berry
#

I suppose you're on mobile then?

grave fjord
#

Yep

#

Does that filename go anywhere? I thought it all gets deleted when incorporated into the release notes?

summer berry
#

I'm not familiar with the contrib process for cpython but I presume that is the case.

#

Not sure if their tooling relies on a certain name format that this would interfere with

#

Apparently it's just a nonce to prevent name collisions

grave fjord
#

Well I put a note on the PR, thanks for catching it

summer berry
#

np

grave fjord
#

I assumed bedevere/news would check that

dull lance
#

Is there a good way to automatically type hint functions that forward keyword parameters of an inner function? For example:

class MyClass:
    def my_func(self, *, x: int, y: bool) -> float:
        print('my_func', x, y)
        return x / 2

class MyWrapper:
    def __init__(self, obj: MyClass) -> None:
        self._obj = obj

    def wrapper_func(self, a: str, **kwargs):
        print('wrapper_func', a)
        return self._obj.my_func(**kwargs)
#

How would I type hint wrapper_func so that its signature would be automatically updated when my_func is updated?

#

I know parameter objects are a thing, but they are more verbose than simply passing keyword arguments

#

So I would prefer using keyword arguments unless it's not possible to type hint them in a sustainable manner

pastel egret
#

You might be able to create some sort of @wraps(MyClass.my_func) decorator, which returns my_func, so external code typechecks correctly. But that doesn't validate the wrapper itself..

dull lance
#

Hmm, something like

P = ParamSpec('P')
PDummy = ParamSpec('PDummy')
R = TypeVar('R')
def dec_wrapper_func(my_func: Callable[Concatenate[MyClass, P], R]):
    def wrapper(wrapper_func: Callable[Concatenate[MyWrapper, str, PDummy], R]) -> Callable[Concatenate[MyWrapper, str, P], R]:
        def inner(self: MyWrapper, a: str, *args: P.args, **kwargs: P.kwargs):
            print('wrapper_func', a)
            return my_func(self._obj, *args, **kwargs)
        return inner
    return wrapper
#

and then

class MyClass:
    def my_func(self, *, x: int, y: bool) -> float:
        print('my_func', x, y)
        return x / 2

class MyWrapper:
    def __init__(self, obj: MyClass) -> None:
        self._obj = obj

    @dec_wrapper_func(MyClass.my_func)
    def wrapper_func(self, a: str, **kwargs):
        pass
#

Or did I overcomplicate this?

#

Not sure whether there is a good way to pull the logic from the decorator back to the body of the instance function

dull lance
# 
def dec_wrapper(my_func: Callable[Concatenate[MyClass, P, R]):
    class _MyWrapper:
        def __init__(self, obj: MyClass) -> None:
            self._obj = obj

        def wrapper_func(self, a: str, *args: P.args, **kwargs: P.kwargs) -> R:
            print('wrapper_func', a)
            return my_func(self._obj, *args, **kwargs)

    return _MyWrapper

MyWrapper = dec_wrapper(MyClass.my_func)
#

This looks cleaner

summer berry
#

This seems loosely related to what I was wishing for a few days ago #type-hinting message dealing with kwargs is annoying when it comes to typing

dull lance
#

pyright doesn't seem to be able to capture the function signature 😐

unknown.png
#

However, it can do so in the first solution

unknown.png
#

not sure why this is the case

lethal mauve
#

hello

dull lance
glacial pollen
#

Is there a quick and dirty way to provide a minimal stub for a package that doesn't provide one? Even if it's not a particularly useful stub

brisk heart
#

mypy's stubgen perchance?

brisk heart
#

How do I subscript typing.Self?

# 
def __get__(self: Self[Concatenate[typing.Any, P], T], instance: typing.Optional[object], owner: type) -> Self[P, T]: ...
hearty shell
#

I think that is disallowed by the pep

brisk heart
#

shucks, I have trouble doing this even with typevars

hearty shell
#

Yeah

Note that we reject using Self with type arguments, such as Self[int]. This is because it creates ambiguity about the type of the self parameter and introduces unnecessary complexity:

brisk heart
#

well I have unnecessary complexity because they don't allow it now 😕

hearty shell
brisk heart
#

how the heck do I make a __get__ that works like classic method binding

hearty shell
dull lance
#

Can't provide more information without more specifics on what you're doing though

trim tangle
#

what is your use case?

brisk heart
brisk heart
trim tangle
#

I think this particular usage wouldn't really work even with HKT because child classes can have a different number of generic paramters.

class Foo(Generic[T]):
    def bar(self: Self[int]) -> Self[str]:
        ...

class Baz(Foo[float]):
    # what should `bar` do? (what is `Self[str]`?)

class Duck(Foo[tuple[A, B, C]], Generic[A, B, C]):
    # what should `bar` do? (what is `Self[str]`?)
#

(out of context but: Optional[object] is the same as object)

brittle socket
#

Is there a more specific type hint than str for regex string literals i.e. r"\d.*"? Please ping me if you reply

trim tangle
green gale
#

typing.Pattern[str] works for a compiled regex (re.compile()) afaik, but that's not what you showed. typing.Match also corresponds to re.match()

trim tangle
#

It's not a regex literal, it's a "raw string". It creates a normal string object, but is more readable with backslashes

trim tangle
brittle socket
green gale
trim tangle
green gale
trim tangle
#

If you want to signal intent like that, accept a re.Pattern object

brittle socket
#

Oh I see

green gale
#

!e

print(r"Hello,\nworld!")
rough sluiceBOT
#

@green gale :white_check_mark: Your eval job has completed with return code 0.

Hello,\nworld!
green gale
#

this is a perfectly valid string

brittle socket
trim tangle
#

and not, for example, a function accepting a str and returning a bool

green gale
#

I mean, if you really need to pass it to re.match or something similar, take a re.Pattern, sure.

brittle socket
#

The function does text splitting based on the received pattern. I also use re.search and re.match with it

brittle socket
#

Because taking a re.Pattern forces the caller to import re themselves

trim tangle
#

You can provide flags to re.compile

#

e.g. case insensitivity

brittle socket
#

Ok...so you mean this gives better control to the caller?

trim tangle
#

yes

green gale
#

I'm a firm believer in the idea that you should restrict the input type as much as possible rather than forcing the user to handle the results of your function, so that's my own philosophy, I guess

#

so like, if the user is the one to compile it, they have to ensure it's a valid regex

brittle socket
#

Indeed. I like that it would force them to ensure that. It's not that case when I'm just taking a string

uneven ginkgo
#

How do I type hint a json encodable object/variable?
I am having trouble doing Dict[str, Any] or Any
Is there another way to type hint it?

dull lance
brittle socket
#

Thank you @trim tangle and @green gale 🙂

rustic gull
#

How do you type hint an argument that takes either a mmap or an open() (file-object) in binary mode? I know BinaryIO works for typehinting the file-object but not sure if it applies for the mmap too

oblique urchin
rustic gull
#

i'll try that thanks!

uneven ginkgo
dull lance
#

yes

#

you essentially define the structure of the object in the subclass

uneven ginkgo
#

The keys in that json_obj are keys that i am not able to know beforehand

#

i just need to type hint that the json_obj is a json encodable object

oblique urchin
dull lance
#

then yeah you can just use dict[str, Any], although I think JSON objects can also be a list

dull lance
# dull lance However, it can do so in the first solution

!e

from __future__ import annotations

from typing import Any, Callable, Generic, Optional, TypeVar, overload
from typing_extensions import ParamSpec, Protocol, Self

_TOwner = TypeVar('_TOwner', contravariant=True)
_P, _R = ParamSpec('_P'), TypeVar('_R', covariant=True)
class _UnboundMethod(Protocol[_TOwner, _P, _R]):
    def __call__(__self, self: _TOwner, *args: _P.args, **kwargs: _P.kwargs) -> _R: ...

class _BoundMethod(Protocol[_P, _R]):
    def __call__(__self, *args: _P.args, **kwargs: _P.kwargs) -> _R: ...

class _ForwardParams(Generic[_TOwner, _P, _R]):
    def __init__(
        self,
        fn: _UnboundMethod[_TOwner, Any, Any],
        ref_fn: Callable[_P, _R],    # So that _P and _R can be inferred automatically
    ) -> None:
        self._fn = fn

    @overload
    def __get__(self, obj: None, owner: type[_TOwner]) -> Self: ...
    
    @overload
    def __get__(self, obj: _TOwner, owner: type[_TOwner]) -> _BoundMethod[_P, _R]: ...

    def __get__(self, obj: Optional[_TOwner], owner: type[_TOwner]) -> Self | _BoundMethod[_P, _R]:
        if obj is None:
            return self
        
        def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> _R:
            return self(obj, *args, **kwargs)

        return wrapper
    
    @overload
    def __call__(__self, self: _TOwner, *args: _P.args, **kwargs: _P.kwargs) -> _R: ...
    
    @overload
    def __call__(__self, *args: _P.args, **kwargs: _P.kwargs) -> _R: ...
    
    def __call__(__self, *args, **kwargs):
        return __self._fn(*args, **kwargs)

def forward_params(ref_fn: Callable[_P, _R]):
    def wrapper(f: _UnboundMethod[_TOwner, Any, Any]) -> _ForwardParams[_TOwner, _P, _R]:
        return _ForwardParams(f, ref_fn)
    return wrapper


class RefClass:
    def ref_fn(self, x: int, y: float) -> bool:
        return x == y

class MyClass: 
    @forward_params(RefClass().ref_fn)
    def my_fn(self, *args, **kwargs):
        return RefClass().ref_fn(*args, **kwargs)
rough sluiceBOT
#

@dull lance :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 4, in <module>
003 | ModuleNotFoundError: No module named 'typing_extensions'
green gale
#

we should add that

uneven ginkgo
#

the json_obj should be type hinted in a way that it should be able to be json encoded by json.dumps
it can be anything from a string to a dictionary. as long as it can be json encoded

uneven ginkgo
dull lance
uneven ginkgo
dull lance
#

I see this in the source code

unknown.png
#

so it would be something like Union[dict[str, Any], list[Any], tuple[Any, ...], str, int, float, bool, None]

spiral fjord
#

And Ideally those Any's would refer back to the type itself, but idk if recursive typing is possible.

trim tangle
#

in Pyright it is possible:

# 
JSON = Union[dict[str, "JSON"], list["JSON"], tuple["JSON", ...], str, float, bool, None]
#

It does get quite complex with custom decoders/encoders though, since they can accept or return other objects.

dull lance
tranquil turtle
#

is there a difference between Any and Any | T?

uneven ginkgo
spiral fjord
#

You can use the normal types in 3.10+

tranquil turtle
#

list and List mean exactly same thing

trim tangle
spiral fjord
#

or even earlier

uneven ginkgo
#

oh damn, had no idea

#

ty!

#

wait, so i dont have to import from typing anymore??

spiral fjord
#

You'll have to import less things

trim tangle
#

yeah

uneven ginkgo
#

got it!

trim tangle
#

and some stuff like Iterator are now supposed to be imported from collections.abc

#

(there are more details in PEP 585)

trim tangle
spiral fjord
#

The union was also not needed if it used |

tranquil turtle
#

i suggest adding type-variable factory to typing:

from typing import t

def f(x: t.T, y: t.T) -> t.T:
    return x or y

possible implementation:

class _TVarFactory:
    def __getattr__(self, varname: str) -> TypeVar:
        return TypeVar(varname, covariant=varname.endswith('_co'), contravariant=varname.endswith('_contra'))
t = _TVarFactory()
dull lance
green gale
rough sluiceBOT
#

@green gale :warning: Your eval job has completed with return code 0.

[No output]
spiral fjord
#

noice

rough sluiceBOT
#

@dull lance :warning: Your eval job has completed with return code 0.

[No output]
dull lance
#

!e

from __future__ import annotations

from typing import Any, Callable, Generic, Optional, TypeVar, overload
from typing_extensions import ParamSpec, Protocol, Self

_TOwner = TypeVar('_TOwner', contravariant=True)
_P, _R = ParamSpec('_P'), TypeVar('_R', covariant=True)
class _UnboundMethod(Protocol[_TOwner, _P, _R]):
    def __call__(__self, self: _TOwner, *args: _P.args, **kwargs: _P.kwargs) -> _R: ...

class _BoundMethod(Protocol[_P, _R]):
    def __call__(__self, *args: _P.args, **kwargs: _P.kwargs) -> _R: ...

class _ForwardParams(Generic[_TOwner, _P, _R]):
    def __init__(self, fn: _UnboundMethod[_TOwner, ...], ref_fn: Callable[_P, _R]) -> None:
        self._fn = fn

    @overload
    def __get__(self, obj: None, owner: type[_TOwner]) -> Self: ...

    @overload
    def __get__(self, obj: _TOwner, owner: type[_TOwner]) -> _BoundMethod[_P, _R]: ...

    def __get__(self, obj: Optional[_TOwner], owner: type[_TOwner]) -> Self | _BoundMethod[_P, _R]:
        if obj is None:
            return self

        def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> _R:
            return self(obj, *args, **kwargs)

        return wrapper

    @overload  # Unbound
    def __call__(__self, self: _TOwner, *args: _P.args, **kwargs: _P.kwargs) -> _R: ...

    @overload  # Bound
    def __call__(__self, *args: _P.args, **kwargs: _P.kwargs) -> _R: ...

    def __call__(__self, *args, **kwargs):
        return __self._fn(*args, **kwargs)

def forward_params(ref_fn: Callable[_P, _R]):
    def wrapper(f: _UnboundMethod[_TOwner, ...]) -> _ForwardParams[_TOwner, _P, _R]:
        return _ForwardParams(f, ref_fn)
    return wrapper

class RefClass:
    def ref_fn(self, x: int, y: float) -> bool:
        return x == y

class MyClass: 
    @forward_params(RefClass().ref_fn)
    def my_fn(self, *args, **kwargs):
        return RefClass().ref_fn(*args, **kwargs)

print('unbound result', MyClass.my_fn(MyClass(), 1, 2.1))
print('bound result', MyClass().my_fn(1, 2.1))
rough sluiceBOT
#

@dull lance :white_check_mark: Your eval job has completed with return code 0.

001 | unbound result False
002 | bound result False
brittle socket
#

I have disable_error_code = attr-defined in my mypy.ini and it works fine. But I want it to be specific to a class in a file. How can I achieve that? I tried this as instructed in the docs (https://mypy.readthedocs.io/en/stable/config_file.html#examples) but it didn't work (still showed attr-defined errors):

[mypy-file_name.class_name.*]
disable_error_code = attr-defined
#

The x of this y is that the members it's complaining about the definition of are defined dynamically through setattr

#

Please ping me if you reply

rare scarab
#

Why not put # typing: ignore on the class?

oblique urchin
brittle socket
rare scarab
#

Consider adding typing?

brittle socket
#

Wdym?

rare scarab
#

type what you expect

brittle socket
#

I can't, the members in question are defined with setattr

rare scarab
#

unless it's too dynamic or relies on mixins

#

you don't define it in the class?

brittle socket
#

Yeah, in init

rare scarab
#

Can you post it

brittle socket
# 
class B:
  Marker: TypeAlias = Literal["intro_marker", "chapter_marker", "header_marker"]

  default_markers: Dict[Marker, re.Pattern[str]] = {
    "intro_marker": re.compile(r"..."),
    "chapter_marker": re.compile(r"..."),
    "header_marker": re.compile(r"...") }

  def __init__(self, markers: Dict[Marker, re.Pattern[str]] | None=None):
    if markers is None:
      markers = {}
    for marker, default_pattern in B.default_markers.items():
      setattr(self, marker, markers.get(marker, default_pattern))

It's the members defined by setattr that trigger mypy's attr-defined

#

And so every line where e.g. self.intro_marker is used, mypy flags it with attr-defined

rare scarab
#

so your types would be ```py
intro_marker: re.Pattern[str]
chapter_marker: re.Pattern[str]
header_marker: re.Pattern[str]

brittle socket
#

Isn't that already the case in default_markers?

rare scarab
#

default_markers won't be used to look up the intro_marker attr

brittle socket
rare scarab
#

in fact, you could just do something like ```py
class B:
intro_marker = re.compile(r"...")
chapter_marker = re.compile(r"...")
header_marker = re.compile(r"...")

def init(self, markers):
for marker in "intro_marker", "chapter_marker", "header_marker":
if marker in markers:
setattr(self, marker, markers[marker])

#

if you don't set the self.attr, it will look it up the class var

#

by that, I mean class vars are the shared default value for all objects of that class.

brittle socket
#

In that solution, markers isn't optional anymore

#

The intent was forcing the caller to use those exact keys were they to provide markers

#

Which is achieved with the combination of a dictionary and Literal

#

With that snippet, they 1) have to provide markers and 2) can give it irrelevant keys or lack one of those 3, which the class absolutely relies on

rare scarab
#

but yours has defaults it uses, and only uses the ones provided as defaults

#

the important part is the classvar

brittle socket
#

I use get which would take the value if it's provided, otherwise returns its 2nd arg as the default

rare scarab
#

if you don't setattr, it will use the one provided by the class

brittle socket
#

In your version? If setattr is not called, then that member is just not defined 🤔

#

In mine, setattr is always called

#

And you never use your class vars?

#

Yeah that example is confusing me more than anything tbh

rare scarab
#

!e ```py
class B:
intro_marker = "intro"
chapter_marker = "chapter"
header_marker = "header"

def init(self, markers=None):
if not markers:
markers = {}
for marker in "intro_marker", "chapter_marker", "header_marker":
if marker in markers:
setattr(self, marker, markers[marker])

b = B()
print(b.intro_marker)

rough sluiceBOT
#

@rare scarab :white_check_mark: Your eval job has completed with return code 0.

intro
rare scarab
#

!e ```py
class B:
intro_marker = "intro"
chapter_marker = "chapter"
header_marker = "header"

def init(self, markers=None):
if not markers:
markers = {}
for marker in "intro_marker", "chapter_marker", "header_marker":
if marker in markers:
setattr(self, marker, markers[marker])

b = B({"intro_marker": "intro2"})
print(b.intro_marker)

rough sluiceBOT
#

@rare scarab :white_check_mark: Your eval job has completed with return code 0.

intro2
rare scarab
#

!e ```py
class B:
intro_marker = "intro"
chapter_marker = "chapter"
header_marker = "header"

def init(self, markers=None):
if not markers:
markers = {}
for marker in "intro_marker", "chapter_marker", "header_marker":
if marker in markers:
setattr(self, marker, markers[marker])

b = B({"intro_marker": "intro2"})
print(B.intro_marker)
print(b.intro_marker)

rough sluiceBOT
#

@rare scarab :white_check_mark: Your eval job has completed with return code 0.

001 | intro
002 | intro2
rare scarab
#

@brittle socket Does that make sense? ^

brittle socket
#

I'm surprised those class vars were reached at all, but my brain is mush rn, I'll grok this tomorrow :)

oblique urchin
blazing nest
#

Ooh! Awesome, thanks

brittle socket
# rare scarab <@458440277548335125> Does that make sense? ^

Oh, now it does yeah. Defaults are handled by class vars, and members are created if values are provided.
I suppose the surprising part was that first print(b.intro_marker) returning intro which is a member call syntax returning a class var

#

!e

class B:
  intro_marker = "intro"
  chapter_marker = "chapter"
  header_marker = "header"

  def __init__(self, markers=None):
    if not markers:
      markers = {}
    for marker in "intro_marker", "chapter_marker", "header_marker":
      if marker in markers:
        setattr(self, marker, markers[marker])
    
  def show_marker(self):
      print(self.intro_marker)

a = B()
a.show_marker()
rough sluiceBOT
#

@brittle socket :white_check_mark: Your eval job has completed with return code 0.

intro
brittle socket
#

Apparently you can use class variables with self. TIL

oblique urchin
brittle socket
#

I thought you had to the class name or cls for class variables

#

that self was reserved for per-object-members

brittle socket
# 
class B:
    intro_marker = re.compile(r"...")
    chapter_marker = re.compile(r"...")
    header_marker = re.compile(r"...")

    def __init__(self,  markers: dict[str, re.Pattern[str]] | None=None):    
        if markers is not None:
            for marker in "intro_marker", "chapter_marker", "header_marker":
                if marker in markers.keys():
                    setattr(self, marker, markers[marker])

@rare scarab Good? 🙂

rare scarab
#

yeah.

#

You could also call self.__dict__.update({})

brittle socket
#

No idea what that does. I'll look it up

rare scarab
#

it's basically bulk setattr for an object (without slots)

#

this isn't really type related at this point though

brittle socket
#

Indeed. Thank you for this educational ride 🙂

rustic lagoon
#

hi, I'm trying to type hint some monads. I was just talking about my issue in #help-cupcake, and it turns out my issue might be related to a bug. I'm wondering how to work around this issue

rustic gull
#

related to the above this , given some class A, is it possible to express: "some type B that inherits A (but not A itself, because then C which also inherits A would also count, and only B is valid here)"?

rustic lagoon
#

Which I'm not super happy about

rustic gull
rustic lagoon
#

That's true, but that's me overwriting the type of self

#

to Monad[Any]

#

In the implementation it's more restrictive

#

And the type checker is able to infer the correct type

rustic gull
#

yeah I saw

rustic lagoon
# 
from methods import fold
from monad import Callable, Er, Ok, Result, ResultType

StrDict = Result(str, KeyError)


@StrDict.binds
def format_with(format: str, d: dict[str, str], keys: list[str]) -> str:
    vals = [d[k] for k in keys]
    return format.format(*vals)


def main():
    f_string = "I like, {0} and {1}!"
    my_dict = {"color": "purple", "language": "Haskell", "city": "Warsaw"}
    strings = [
        format_with(f_string, my_dict, ["city", "color"]),
        format_with(f_string, my_dict, ["language", "city"]),
        format_with(f_string, my_dict, ["color", "language"]),
        format_with(f_string, my_dict, ["name", "food"]),
    ]
    for s in strings:
        match s.value:
            case Ok(v):
                print(v)
            case Er(e):
                print("No such key:", e)

    get_key: Callable[
        [dict[str, str], str], ResultType[dict[str, str], KeyError]
    ] = Result(dict, KeyError).binds(lambda d, k: {**d, k: my_dict[k]})

    collect: Callable[
        [list[str]], ResultType[dict[str, str], KeyError]
        ] = lambda keys: fold(get_key, Result(dict, KeyError)({}), keys) # type: ignore

    collected = [
            collect(["city", "color"]),
            collect(["language", "city"]),
            collect(["color", "language", "city", "food"]),
            collect(["name", "food"]),
            ]
    for c in collected:
        match c.value:
            case Ok(v):
                print(v)
            case Er(e):
                print("No such key:", e)


if __name__ == "__main__":
    main()

So the fold method needs some tweaking since it doesn't 100% work rn, but the functionality is there

# 
from functools import reduce
from typing import Callable, Iterable, TypeVar

from monad import Monad

T1 = TypeVar("T1")
T2 = TypeVar("T2")


def fold(
    func: Callable[[T1, T2], Monad[T1]], seed: Monad[T1], iter: Iterable[T2]
) -> Monad[T1]:
    return reduce(lambda acc, x: acc.apply(lambda a: func(a, x)), iter, seed)

this is the fold method btw

median ledge
#

is it possible to overload a method with optional, mutually-exclusive arguments?

from typing import overload
class Foo:
    @overload
    def bar(self, x: int = None):...
    
    @overload
    def bar(self, y: int = None):...

    def bar(self, x: int = None, y: int = None):
        if x and y:
            raise TypeError("cannot pass both x and y")
        ...
summer berry
#

I believe you can if you make them keyword-only arguments

#

If you allow them to be positional then it's ambiguous

summer berry
median ledge
#

yeah, that works, thanks!

summer berry
#

You're welcome

rare scarab
# 
@overload
def bar(self, *, x: int): ...
@overload
def bar(self, *, y: int): ...
median ledge
#

Hmm, if I do this, I lose type hints for the arguments

rare scarab
#

passing args via a.bar(y=3)?

median ledge
#

wait, I'm being dumb

#

all is good :D

runic sleet
#

can I not subscript like such in Python 3.7?

def func() -> Generator[str]:
  ...
#

I get a

E   TypeError: 'ABCMeta' object is not subscriptable
soft matrix
#

Nope

#

You need to import future.annotations

#

If you want it to be subscriptable at runtime use the typing version

rare scarab
#

is that collections.abc.Generator?

#

(is that a type?)

#

Use ```py
from typing import Generator
def func() -> Generator[str, None, None]:
...

runic sleet
#

so I'm not sure if I can actually use the one from typing

rare scarab
#

I think the one from typing is technically an alias

runic sleet
#

here's the full signature

from __future__ import annotations
from collections.abc import Generator, Callable
from typing import Sized


class Length:
    def __init__(self, length_like: int | Sized):
        self._length: int = 0
        self._call: Callable[None, int] | None = None

        if isinstance(length_like, int):
            self._length = length_like
        elif isinstance(length_like, Sized):
            self._call = length_like
        else:
            raise TypeError(f'Length must be an int or a Sized object, not {type(length_like)}')

    @property
    def value(self) -> int:
        if self._call is not None:
            return self._call.__len__()
        return self._length


class SizedGenerator(Generator):
    def __init__(self, gen: Generator, length: int | Sized):
        """
        Generator with fixed size.

        Args:
            gen: Base Generator
            length: Length of iterator as int, or Sized object that implements __len__
        """
        super().__init__()
        if not isinstance(gen, Generator):
            raise TypeError(f"Expected Generator, got {type(gen)}")
        self._gen = gen
        self._length = Length(length)
        self._index = 0

    def send(self, *args, **kwargs):
        if self._index > self._length.value:
            self.throw(IndexError(f"Index out of range: {self._index}. Length defined as {self._length}."))
        self._index += 1
        return self._gen.send(*args, **kwargs)

    def throw(self, *args, **kwargs):
        return self._gen.throw(*args, **kwargs)

    def __len__(self) -> int:
        return self._length.value

-----

def yield_all(self, timeout: float = None) -> SizedGenerator[TaskResult]:
  ...
#

so essentially everything works with 3.7 except that TypeError: 'ABCMeta' object is not subscriptable thing

#

I'm kind of torn between getting rid of that subscript (and getting less type hinting in my current 3.10 environment) vs. just dropping 3.7 support pithink

brazen jolt
#

your SIzedGenerator isn't generic

#

it's just a subclass of Generator, without any generic type vars added

#

maybe consider ```py
class SizedGenerator(Generator[T, None, None]):
...

#

the __future__.annotations does give you the freedom to use non-generic collections.abc.Generator, but since your class itself isn't generic, that doesn't help you here

rare scarab
#

You should be able to use typing.Generator for this.

runic sleet
#

is that what a real generator made with def and yield becomes? A generic?

brazen jolt
#

if a class is generic, it will support class getitem, allowing for MyClass[str] without getting MyClass isn't subscriptable error

runic sleet
brazen jolt
#

yes

#

you can name the type variables whatever you want

rare scarab
#

Is there a reason it's a class and not a regular function?

runic sleet
#

the SizedGenerator?

rare scarab
#

Yes.

runic sleet
#

I want to be able to give a generator a __len__

brazen jolt
#

you may want to be decreasing that length each time the generator yields a value though

#

since at that moment, the generator should no longer keep track of any previous values, that's for the most part the point of generators

#

so the length there should probably be more like items remaining thing

runic sleet
brazen jolt
#

I mean, it kind of would, but with generators, you can't go back to a previous index

runic sleet
brazen jolt
#

so it probably doesn't make much sense to keep track of length as the original length, but as the remaining length, i.e. the original length - current index

brazen jolt
runic sleet
#

I'll switch that over, thanks

brazen jolt
#

I mean, depending on your use, this may still be what you want, it's just quite odd since yeah, if the generator was already used, the length immediately becomes incorrect if you wanted to use it as an indication on how many times the generator will yield a next value

runic sleet
#

I mainly just wanted a generator that when passed to tqdm would actually show the total amount correctly for a progress bar

#

apparently it worked previously since tqdm just checks len once at the start

#

but I should probably make __len__ work correctly anyways in case it's used for something else

rare scarab
#

tqdm has a total kwarg

green gale
#

is there a way to type hint x and y to be any types that can be added to one another? and then I have no idea how I would write the return type, either.

def add(x, y):
    return x + y
acoustic thicket
#

ignore T its unused

green gale
#

ahhh that's much more complex than I would have liked

#

thank you!

hearty shell
#

What is fun about operators is that is also not a full definition, a full definition would require being able to identify a subtype relationship in the overloads which is not possible atm

green gale
#

I wish there was a simpler syntax for saying "this argument must have a method with this signature" without a protocol

little hare
#

what am i doing wrong??

# 
import disnake

from disnake.ext import commands


bot = commands.InteractionBot()

@bot.slash_command(name='asdf')
async def cmd(itr: disnake.ApplicationCommandInteraction, opt:int):
    await itr.send(opt)
unknown.png
#

pyright detects an error

#

the problem is

#

mypy is not

#

(itr.send takes a string)

#

??

#

now its worse

acoustic thicket
# little hare

what happens if you add a from disnake import ApplicationCommandInteraction

little hare
#

oki i got it all to work

#

annoyingly

#

but it does now

brittle socket
# 
error: Type aliases inside dataclass definitions are not supported at runtime  [misc]
        TextOrTable : TypeAlias = str | list[dict]

mypy complains about this, but umm why is this bad? I'm supposed to define type aliases outside of the class even if they're strictly relevant to it?

void panther
#

won't the dataclass try to use it as a field?

brazen jolt
#

I think what you want is ClassVar[TypeAlias]?

#

since otherwise as Numerlor said, it would assume TextOrTable is a field with default value of str | list[dict]

brittle socket
#

Ohh I see. Types can be default values?

brazen jolt
#

any object can be set as a default value

#

the important thing here though is the way dataclasses distinguish between fields and class variables, which happens through type-hints. If a variable has a type-hint set, it's generally a field, if it's just x = 5 without any type-hints, it's a class variable. The only exception here is if the type-hint is ClassVar, in which case it will be treated as a class variable even though there is a type-hint

brittle socket
#

I didn't know about the type-hint-less case of x = 5 being considered a class var. Got it, thanks!

#

I'm wondering if it's really worth it to import them from there instead of typing like everything else

summer berry
#

It'd be worth it since importing them from typing is deprecated.

#

If you only support Python 3.9+ then just import them from collections.abc

#

They're equivalent

brittle socket
#

Right...but this feels weird

from typing import Any, Literal, TypeAlias
from _collections_abc import Iterator, Callable
summer berry
#

That's just how it is

brazen jolt
brittle socket
#

I'm on 3.10.4

#

(not importing __future__.annotations though, as I haven't seen a need for it yet)

# 
things: list[dict]
things2: Iterator[Any]

things2 = map(lambda p: p["VALUE"], things)
things2 = filter(lambda p: p != "undesirable", things)

What's the right way to handle typing here? I feel like I'm cheating with things2: Iterator[Any]

summer berry
#

So you wouldn't even need to annotate things2

brittle socket
brazen jolt
#

consider TypedDict

brittle socket
#

It would be

things: list[dict[str, str | list[dict[str, str | list[dict]]] | dict]]
#

And as you see, it's not even all of it

#

I'm not familiar with TypedDict I'll look it up

brittle socket
# brazen jolt consider `TypedDict`

things is actually a dict returned by a lib (simplify-docx), and TypedDict could work as far as the specific set of keys, but the values' types are heterogenous

#

TypedDict declares a dictionary type that expects all of its instances to have a certain set of keys, where each key is associated with a value of a consistent type.

summer berry
#

Your example looks like it might be a recursive type. Is that the case?

brittle socket
#

Oh, interesting. Not sure actually. Is there some utility to get the full type of an object?

#

If not, I'll just dive in manually (tomorrow)

#

But say it is recursive, how can it help?

summer berry
#

Because it would likely be easier to annotate

# 
ThingType = list[dict[str, str | ThingType]]
brittle socket
#

Wait you can do that 😮

summer berry
#

Well, at least pyright supports that

brittle socket
#

I keep hearing about pyright around here. Should I use that instead of mypy + pylint?

summer berry
#

I don't know. I've not used mypy that much.

green gale
#

I've personally found pyright a lot nicer, even if mypy is the "de-facto standard"

summer berry
#

pyright seems to keep up with the latest features better

green gale
#

I haven't use pylint too much, so not sure about that one

summer berry
#

pyright isn't a complete replacement for pylint

#

pyright doesn't concern itself with formatting

#

But it does check some stuff like undefined variables, unused imports, etc.

brittle socket
#

Heh, I'll just use all 3

soft matrix
rustic gull
#

Not sure if I'm misunderstanding the documentation but I was looking at TYPE_CHECKING and it states: The first type annotation must be enclosed in quotes, making it a “forward reference”, to hide the expensive_mod reference from the interpreter runtime.
I was looking at discord.py source code and they're doing this:

from typing import TYPE_CHECKING
if TYPE_CHECKING:
    import datetime
...
    @property
    def created_at(self) -> datetime.datetime:
        """:class:`datetime.datetime`: Returns the member's creation time in UTC."""
        return snowflake_time(self.id)

How come this doesn't cause any error? But if I run something similar:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    import datetime

class A:
    def a(self) -> datetime.datetime:
        print("called")

a = A()
a.a()
void panther
#

do they have an annotations future import at the top?

rustic gull
#

Yeah I just found some mypy documentation and they do have a from __future__ import annotations on another file I guess that's enough

void panther
#

Yes, it'll turn every annotation into a string like that

rustic gull
#

If I kept scrolling a bit more on the mypy page 1 minute after I asked the question, but thanks this clears things up

brisk heart
pastel mantle
#

Is from __future__ import annotations the way to go moving forward?

green gale
#

yes

#

it's becoming the default soon (3.11? not sure)

pastel mantle
#

Cool

summer berry
#

That decision was reverted for 3.11

green gale
#

was it confirmed for a different version?

soft matrix
#

nope

#

its eternally deferred

summer berry
#

I don't think so. They need to figure out some things due to community feedback at the final hour

green gale
#

ugh

pastel mantle
#

Is there a tracking issue for this somewhere? I can check when I get to my computer

soft matrix
pastel mantle
#

Yea the second link is the last I’ve read about this.

summer berry
modern grotto
#

How do i handle this? I know that a RuleType-like object will be returned, but since the original thing may also return a Response, it's showing typeIssue

Type "dict[Unknown, Unknown] | Response" cannot be assigned to type "RuleType"
    "Response" is incompatible with "RuleType"
sterile python
modern grotto
modern grotto
sterile python
modern grotto
#

ok

sterile python
# modern grotto ok

TypeGuard would be good if you would like to exactly check is dict being RuleType

def is_ruletype(mapping: Mapping) -> TypeGuard[RuleType]:
    return isinstance(mapping.get("rule"), str)  # If RuleType is a dictionary with rule string key


response = function()
if is_ruletype(response):
    ...

And cast would be good if you exactly sure that returned dict is RuleType

response = function()
typing.cast(response, RuleType)
...  # Using response as RuleType
modern grotto
#

Interesting, thanks for the info!

cedar sundial
#

When a typed function has an annotation for a Literal and I pass in one of those strings, I get a mypy error [arg-type]. Am I doing something wrong?

Preprocessors = Literal['test', 'test1', 'etc']
def flush(self, ppt: Preprocessors) -> None: ...

ppt = "test"
some_class.flush(ppt)  # Incompatible type "str"; expected Literal[...]
soft matrix
#

You either need to annotate ppt as Final or just pass the string directly into flush

acoustic thicket
cedar sundial
#

Thank you both

fervent sierra
#

Hello, everyone

I have multiple implementations of my_module, which I select during load time by setting PYTHONPATH to point to one of those implementations.
In my code, I use it like so: from my_module import foo, bar .
What would be a sensible, practical way to specify that all implementations of my_module should adhere to some interface?

#

I have a feeling that I might be outside of the realm of the type checker by having multiple modules doing the same thing. I can tell pyright to take one of them into consideration when checking the entire project, but it seems I can't tell it about all the implementations... not without some clever tricks, anyway

trim tangle
#

Why not provide an object (which could be a module) conforming to some interface at runtime?

#

(i.e. "dependency injection")

fervent sierra
#

I usually find that dependency injection is just trying to take a job meant for load time and jam it into runtime. Here is some of my reasoning:
1 - DI adds extra complexity because it adds more code to do the "loading" in runtime;
2 - it adds the potential for things to not break immediately on program startup, but later during runtime when I might not be watching;
3 - everybody knows what PYTHONPATH does, but no one will know how my DI system works and how to configure it;
4 - I think it would be easier to create a new implementation if it's just a python module, rather than a specific DI entity (more so if the type checker could help)

My specific use case is that I have multiple implementations of a @global_cache decorator. One doesn't cache at all, another is just @lru_cache, another is using redis, etc

brittle socket
#

Can you assign default values to keys in a TypedDict?

oblique urchin
brittle socket
#

So, even though you know you're expecting a fixed set of keys and consistent values types, if you need default values then TypedDict is just not the right tool?

fervent sierra
oblique urchin
brittle socket
oblique urchin
signal sentinel
#

What's NotRequired?

oblique urchin
#

But I'm still not clear what "default values" means. Can you explain how that would work exactly?

brittle socket
#

Oh, interesting. I'll look up PEP655

#

Sure, sec

oblique urchin
#

note that the TypedDict "constructor" (SomeTD(a=3, b=2)) just returns a dict at runtime

#

would you want it to insert default values for those keys?

brittle socket
# 
class B: # pylint: disable=too-few-public-methods
    Marker: TypeAlias = Literal["intro_marker", "chapter_marker", "header_marker"]

    intro_marker = re.compile(r"...")
    chapter_marker = re.compile(r"...")
    header_marker = re.compile(r"...")

    def __init__(self, markers: dict[Marker, re.Pattern[str]] | None=None):
        if markers is not None:
            self.__dict__.update(markers) # type: ignore[arg-type]

This is what I'm currently doing. if no markers is provided, the class's use of e.g. self.intro_marker defers to its class-var default value.
The caller can individually or totally overwrite those default values by providing a markers dict.
A type checker enforces that a provided markers's keys are within the set defined by Marker's Literals.

#

So I have a set of fixed keys ("intro_marker", "chapter_marker", "header_marker"), and consistent value types (re.Pattern[str]), which points to using a TypedDict, but then I lose the handy mechanism for default values (re.compile(r"..."))

acoustic thicket
#

how about def __init__(self, markers: dict[Marker, re.Pattern[str]] = {"intro_marker": ..., ...})

#

oh wait no (that won't allow overwriting individually)

brittle socket
#

Then I can't use self.__dict__.update(markers) as it is there

#

Not that it's a restriction per-se, but I think it's handier than what I'd have to do with default params in __init__

acoustic thicket
#

NotRequired is the solution here, i think

brittle socket
# 
class B:
  class Markers(TypedDict):
    intro_marker: NotRequired[re.Pattern[str]]
    chapter_marker: NotRequired[re.Pattern[str]]
    header_marker: NotRequired[re.Pattern[str]]

    def __init__(self, markers: Markers | None=None):
      if markers is None:
        markers = {}
      self.intro_marker = markers.get("intro_marker", re.compile(r"..."))
      self.chapter_marker= markers.get("chapter_marker", re.compile(r"..."))
      self.header_marker= markers.get("header_marker", re.compile(r"..."))

I think the TypedDict version would look like this. Not sure which is better. I think I still lean towards the handiness of self.__dict__.update of the 1st

brazen jolt
brittle socket
#

I believe so. I just got exposed to PEP655 as @oblique urchin suggested, but I think total=False still makes sense here

rustic lagoon
#

i have a question whether something is a bug or a feature

def sums_ints(a: int, b: int) -> int:
    return a + b


def main():
    add_two = lambda x: sums_ints(x, 2)


if __name__ == "__main__":
    pass

in this example, pyright sees add_two as having the type (x: Unknown) -> int, but it should be able to infer that the type of x is int from the context of the lambda.
one one hand, it would require the type checker to infer type from the usage of a variable, rather than its declared type, which could be contrary to the whole Explicit is better than implicit python zen line.
however it's able to do that when lambda is passed as an argument in things like map, and if i do inferred = list(map(lambda s: s + s, ["this", "was", "inferred"])) the is correctly inferred to be list[str]

dull lance
#

try using functools.partial, I think pyright can correctly infer the input types in that case

rustic lagoon
#

ok, it does do that, neat

# 
from functools import partial
from itertools import chain
from typing import Iterable, TypeVar

T = TypeVar("T")


def sums_ints(a: int, b: int) -> int:
    return a + b


def merge(a: Iterable[T], b: Iterable[T]) -> list[T]:
    return list(chain(a, b))


def main():
    add_two = partial(sums_ints, b=2)
    print(add_two(5))
    merge_ints = partial(merge, [1, 2, 3])
    result = merge_ints([4, 5, 6])
    print(result)


if __name__ == "__main__":
    main()

it even works for generics, nice

#

although it's a little unfortunate how doing something like

merge_ints = partial(merge, a=[1, 2, 3])
result = merge_ints([4, 5, 6])

does not get caught by the type checker, only errors at runtime

acoustic thicket
#

why would it error pithink

rustic lagoon
#

run it

spiral fjord
#

!e

from functools import partial
from itertools import chain
from typing import Iterable, TypeVar

T = TypeVar("T")


def sums_ints(a: int, b: int) -> int:
    return a + b


def merge(a: Iterable[T], b: Iterable[T]) -> list[T]:
    return list(chain(a, b))


def main():
    add_two = partial(sums_ints, b=2)
    print(add_two(5))
    merge_ints = partial(merge, [1, 2, 3])
    result = merge_ints([4, 5, 6])
    print(result)


if __name__ == "__main__":
    main()```
rough sluiceBOT
#

@spiral fjord :white_check_mark: Your eval job has completed with return code 0.

001 | 7
002 | [1, 2, 3, 4, 5, 6]
rustic lagoon
#

if you assign kwargs first, and then args, if the position of the kwargs you assigned is not bigger than the length of the args you are passing to the partial function, there's an error

#

!e

from functools import partial
from itertools import chain
from typing import Iterable, TypeVar

T = TypeVar("T")


def sums_ints(a: int, b: int) -> int:
    return a + b


def merge(a: Iterable[T], b: Iterable[T]) -> list[T]:
    return list(chain(a, b))


def main():
    add_two = partial(sums_ints, b=2)
    print(add_two(5))
    merge_ints = partial(merge, a=[1, 2, 3])
    result = merge_ints([4, 5, 6])
    print(result)


if __name__ == "__main__":
    main()
rough sluiceBOT
#

@rustic lagoon :x: Your eval job has completed with return code 1.

001 | 7
002 | Traceback (most recent call last):
003 |   File "<string>", line 25, in <module>
004 |   File "<string>", line 20, in main
005 | TypeError: merge() got multiple values for argument 'a'
sterile python
spiral fjord
#

ah, they passed a as both a kwarg and an arg

rustic lagoon
#

yeah, it would be great if the type checker shouted at me if i did that. cause while i might be aware of the problem now, it could be easy to forget in more complex code

runic sleet
#

how would I type hint the return type of this decorator catlost

def sized_generator(length: int | Sized | None = None):
    def deco(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            return SizedGenerator(func(*args, **kwargs), length=length)

        return wrapper

    return deco
oblique urchin
rustic lagoon
#

i'm guessing type checkers haven't implemented PEP 673 yet considering it's a 3.11 feature right?

#

Cause I really wish I could use the Self type

#

And now that I know it's gonna exist it's quite frustrating :v

oblique urchin
rustic lagoon
#

oh hell yeah

oblique urchin
#

generally "3.11 feature" doesn't mean much for new type system features, type checkers have their own schedule and these things get backported in typing-extensions

rustic lagoon
#

I suppose so. I remember having to wait for structural pattern matching to get implemented into mypy for a while after 3.10 release

#

i've switched to pyright since though

#

it's much easier to get working with nvim's lsp

soft matrix
#

Getting it into mypy is painful

oblique urchin
soft matrix
#

It seems like it should be fine now apart from weird bugs I can't reproduce

#

I'll take another look tomorrow

median ledge
#

Say, in file b.py, I import a.pyi and do smth like:

# b.py
from a import (c.foo)

How would I hint in a.pyi that it has a c object, with a foo member, without import c?
Basically, the a.py file would have smth like

import c
#

I don't care about the foo type except to remove errors, since I'm only using it as a type hint

rustic lagoon
#

cool, i guess i'm not doing that

unknown.png
#

lol

rustic lagoon
#

Damn, I'm quite happy with the way I've done those monads

from functools import partial
from typing import Iterable, TypeVar

T1 = TypeVar("T1")
T2 = TypeVar("T2")

from maybe import Just, Maybe, Nothing
from result import Err, Ok, Result


@Result[str, KeyError].capture(KeyError)
def format_with(format: str, d: dict[str, str], keys: Iterable[str]) -> str:
    vals = (d[k] for k in keys)
    return format.format(*vals)


@Maybe[str].binds
def sometimes(ask: str) -> str | None:
    if ask == "do nothing":
        return None
    else:
        return ask


def main():
    f_string = "I like, {} and {}!"
    my_dict = {"color": "purple", "language": "Haskell", "city": "Warsaw"}
    strings = (
        format_with(f_string, my_dict, ("city", "color")),
        format_with(f_string, my_dict, ("language", "city")),
        format_with(f_string, my_dict, ("color", "language")),
        format_with(f_string, my_dict, ("name", "food")),
    )
    for s in strings:
        match s.inst():
            case Ok(v):
                print(v)
            case Err(e):
                print("No such key:", e)

    def add_k_v(source: dict[T1, T2], d: dict[T1, T2], k: T1) -> dict[T1, T2]:
        return {**d, k: source[k]}

    add_from_my = Result[dict[str, str], KeyError].capture(KeyError)(
        partial(add_k_v, my_dict)
    )

    empty = Ok[dict[str, str], KeyError]({})
    collected = (
        empty.fold(add_from_my, ("city", "color")),
        empty.fold(add_from_my, ("language", "city")),
        empty.fold(add_from_my, ("color", "language", "city", "food")),
        empty.fold(add_from_my, ("name", "food")),
    )
    for c in collected:
        match c.inst():
            case Ok(v):
                print(v)
            case Err(e):
                print("No such key:", e)

    asks = ("do nothing", "do something")
    for ask in asks:
        match sometimes(ask).inst():
            case Just(v):
                print(v)
            case Nothing():
                print("Nothing")


if __name__ == "__main__":
    main()
#

it's not super clean cause i need to use that inst() method to let the type checker know that the return type is gonna be only one of the subclasses

#

actually, i could change the type hints for the bind method to an explicit union

#

that way i could match for them without the type checker saying the matches are non exhaustive, without having to run that awkward inst() method

median ledge
#

quick one: if I have a package structured like this:

.
└── foo
    ├── __init__.py
    └── bar.py

with this is the init:

from foo import bar

then bar is a variable in foo, right?
So, ```python
import foo
print(foo.bar)

should be valid?
#

Pylance complains that bar is not a known member of module

oblique urchin
summer berry
#

If you add an __all__ maybe the error goes away?

oblique urchin
#

from . import bar should work too

median ledge
#

Alright, thanks!

brittle socket
brittle socket
median ledge
uneven maple
brittle socket
#

Hm, apparently Pylance includes Pyright and the latter suggests only keeping the former

oblique urchin
brittle socket
#

I just run this every now and then (defined in ~/.bashrc)

lint() {
 echo "----------MYPY----------"
 mypy --show-error-codes --pretty --show-column-numbers --show-error-context "$1";
 echo "----------PYLINT----------"
 pylint --output-format=colorized "$1";
 echo "----------PYRIGHT----------"
 pyright --pythonplatform=Linux "$1";
}
blazing nest
# runic sleet how would I type hint the return type of this decorator <:catlost:77004958475576...

I love overly typed code!

IT = TyperVar('IT')
ST = TypeVar('ST')
RT = TypeVar('RT')
P = ParamSpec('P')

def sized_generator(length: int | Sized | None = None) -> Callable[[Callable[P, Generator[IT, ST, RT]]], Callable[P, SizedGenerator[IT, ST, RT]]:
    def deco(func: Callable[P, Generator[IT, ST, RT]]) -> Callable[P, SizedGenerator[IT, ST, RT]]:
        @wraps(func)
        def wrapper(*args: P.args **kwargs: P.kwargs) -> SizedGenerator[IT, ST, RT]:
            return SizedGenerator(func(*args, **kwargs), length=length)

        return wrapper

    return deco
#

Don't worry, I've done worse

green gale
#

I think I have a fundamental misunderstand of how TypeVars work... I expected this to be very, very wrong because "hello" and 8 have different types, but it's not? Does TypeVar not have to be the same type across multiple usages?

from typing import TypeVar

T = TypeVar("T")

def test(x: T, y: T) -> list[T]:
    lst = []
    lst.append(x)
    lst.append(y)
    return lst

test("hello", 8)
acoustic thicket
#

they're the same type

#

object

#

its funny

green gale
#

what the hell

#

i think i've gone mad

#

so that is valid (as in, type hinted correctly) code???

#

oh god

acoustic thicket
#

yeah, but you could put a bound=str in the typevar

pastel egret
#

When they don't match, the type checker will try checking parent classes to find a common superclass yeah.

green gale
#

still, that is... not what I expected TypeVar to do

#

is there something that does what I expect it to do? (I think I just don't want it to check parent classes)

pastel egret
#

It'll fail then if you assign it to list[str] or whatever.

tranquil turtle
#

I'm going to study it again

green gale
#

!d typing.TypeVar

rough sluiceBOT
#
typing.TypeVar 

class typing.TypeVar```
Type variable.

Usage:

```py
T = TypeVar('T')  # Can be anything
S = TypeVar('S', bound=str)  # Can be any subtype of str
A = TypeVar('A', str, bytes)  # Must be exactly str or bytes
```  Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function definitions. See [`Generic`](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") for more information on generic types. Generic functions work as follows:
green gale
#

I should have just checked the docs, that first example has it

#

that's inconvenient and much less powerful than I had hoped

#

I was looking for something akin to Rust generics, I'll look through the rest of the docs for typing

trim tangle
#

so TypeVars are the "closest thing" to generics

#

guess who forgot to pay for their VPS...

trim tangle
rustic lagoon
#

i use neovim

#

and pyright is the open source type checker that pylance uses

#

though my fonts are doing something ugly with the border, and that's why it has that ugly gray thing around it

#

or maybe it's my lsp settings going whack?

#

either way, too lazy to fix that

rustic lagoon
#

the theme i was using before was just borked it seems

unknown.png
#

cattpuccin works great however

rustic lagoon
#

is there a convenient way of type hinting a non empty list?

soft matrix
#

Size cannot be encoded into a list

green gale
soft matrix
#

If you want that use a tuple

green gale
#

I kinda wanted something TypeVar like but only checks the "top" type (not the type theory term, but rather jus, not looking at parent types)

soft matrix
#

Is that a lower bound?

brazen jolt
#

TypeVars automatically use upper bounds

rustic lagoon
brazen jolt
#

yeah, you probably meant lower bound, in which case I think you can use Final

green gale
green gale
rustic lagoon
#

i was thinking of inheriting from list and restricting that in the new dunder method

brazen jolt
#

oh, then it's just ```py
T = TypeVar("T", bound=str)

rustic lagoon
#

but that would not be obvious to the type checker :v

brazen jolt
#

this will limit the type-var to just str and it's subtypes

green gale
rough sluiceBOT
#
**PEP 593 - Flexible function and variable annotations**

Link

Status

Accepted

Python-Version

3.9

Created

26-Apr-2019

Type

Standards Track

green gale
green gale
soft matrix
#

Annotated doesn't help that much though

green gale
#

there's no way to specify that the length must be greater than 0? sad, that's annoying

#

I want my dependent typing lemon_angrysad

void panther
#

what could I use to have something like this typed

templates = {
    type1: callable_that_takes_instance_of_type1,
    type2: callable_that_takes_instance_of_type2,
}

I'm currently using the dict but I don't think that can get me the specific types?

brazen jolt
#

So, something like: dict[Type[T], Callable[[T], object]]?

#

@void panther

void panther
#

yeah

#

It's also a static definition instead of being created at runtime if that helps

brazen jolt
#

Hm, I suppose you could define a protocol

soft matrix
brazen jolt
# 
class MyDct(Protocol[T], Mapping[Type[T], Callable[[T], object]]):
    ...
soft matrix
#

You'll get a warning T is unused

brazen jolt
brazen jolt
soft matrix
#

But the types are different aren't they?

#

Afaik the relationship between the two isn't type-able

soft matrix
brazen jolt
#

Oh yeah right, hmm maybe you could just make a non-generic protocol with custom getitem,etc then

void panther
#

I think I'll just have a wrapper function for accessing the dict then, and hope I don't mess up in the dict itself

sterile python
# rustic lagoon is there a convenient way of type hinting a non empty list?

You can try to use other languages experience:

class NonEmptySized(Sized[T]):
    head: T
    rest: Sized[T]
 
    def __init__(self, head: T, *rest: T) -> None:
        self.head = head
        self.rest = rest

    @overload
    def __getitem__(self, index: Literal[0]) -> T:
        return self.head
 
    @overload
    def __getitem__(self, index: int) -> Maybe[T]:
        ...
sterile python
rustic lagoon
#

btw, this type error is so dumb

unknown.png
oblique urchin
rustic lagoon
#

i suppose. the underscore thing is what cmp snippets wrote for me :v

#

i just went with it

#

but pyright is still upset at me that i have a variable of unknown type that i check the type of in the next line over

#

i mean, i kinda get it, but it's still funny

rustic lagoon
#

I have a bit of an issue rn. I have the Maybe protocol, that's never gonna be an instance on its own, and i'm wondering if there's a way to let the type checker know that the only instances of it are gonna be Just or Nothing subclasses

# 
from src.maybe import Just, Maybe, Nothing


def main():
    def foo_or_bar(fbar: str) -> Maybe[str]:
        if fbar in ("foo", "bar"):
            return Just(fbar)
        return Nothing()

    match foo_or_bar("foo"):
        case Just(v):
            print(f"{v}!")
        case Nothing():
            print("Aw shucks!")


if __name__ == "__main__":
    main()
#

this has pyright say

Pyright: Cases within match statement do not exhaustively handle all values
  Unhandled type: "Maybe[str]"
  If exhaustive handling is not intended, add "case _: pass"
oblique urchin
rustic lagoon
#

but aliases can't take generic type vars

#

i've done a hack where i have type hinted the inst() method to return the Nothing | Just[T] union

# 
    match foo_or_bar("foo").inst():
        case Just(v):
            print(f"{v}!")
        case Nothing():
            print("Aw shucks!")
#

this is fine

#

but kinda ugly too

oblique urchin
rustic lagoon
# 
class Maybe(Monad[T1], Unwrappable[T1], Foldable[T1], Protocol[T1]):
    def apply(self, f: Callable[[T1], "Maybe[CO]"], /) -> "Maybe[CO]":  # type: ignore
        ...

    def fold(self, f: Callable[[T1, T2], "Maybe[T1]"], l: list[T2]) -> "Maybe[T1]":  # type: ignore
        ...

    def inst(self) -> "Just[T1] | Nothing":
        ...

    @classmethod
    def binds(cls, f: Callable[P, T1 | None], /) -> Callable[P, "Maybe[T1]"]:
        @wraps(f)
        def inner(*args: P.args, **kwargs: P.kwargs) -> "Maybe[T1]":
            try:
                match f(*args, **kwargs):
                    case None:
                        return Nothing()
                    case val:
                        return Just(val)
            except Nothing:
                return Nothing()

        return inner

maybe also has a class method declared in it which is clearer than using...

rustic lagoon