type-hinting

[viral birch]

code that suggests the type of variables that aren't checked by the compiler/interpreter of the language, but rather a third party tool that can catch type errors (ex: passing an int to a function that expects a str)

[slender timber]

Is NotRequired going to be a part of 3.11?

oh ok

it will be

[rustic gull]

Speaking of NotRequired... I've got a bit of a chonker of a question, is it alright if i paste it in here?

[trim tangle]

of course

If there's a lot of code:

!paste

[rough sluice]

Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

[rustic gull]

Cheers, don't want to spam it
Q: Can I place a bound on the values of a TypedDict?

context:
I would like my classes to be generic in the "schema" of their json serialized form.
I define what JSON looks like, and I define a base class:

JsonType = Dict[str, Any] # simplified
SerializedSchema = TypeVar("SerializedSchema", bound=Type[JsonType])

class Serializable(Generic[SerializedSchema]):
  def serialize(self) -> SerializedSchema

Trouble is, pyright gives me a GeneralTypeIssues warning when I try to use it:

class SerializedUserSchema(TypedDict):
  name: str

class User(Serializable[SerializedUserSchema]): # Type "SerializedAliasable" is incompatible with bound type "JsonType" for type variable
  ...

I suppose the problem is that trying to use JsonType as if it were a protocol 🥴

[trim tangle]

The correct way would be SerializedSchema = TypeVar("SerializedSchema", bound=JsonType)

however, a SerializedUserSchema is not really a subtype of Dict[str, Any]

because in a dict, you can add arbitrary string keys.

why do you need the dict restriction?

a Mapping works though:
https://pyright-playground.decorator-factory.su/?gzip=H4sIAPqPC2MC_22QwUrEMBCG73mKYfeSQM0DLKygCIKwp1UvZZFpMtVgmpQkCvXpTbrNKtWcwvz_fPPP9MEPoLy1pJLxLkrsFJhh9CHBAcfRuFfWF0-ayr9Kj9NIzxgauCdHwahmrug7o1IDN25i7CF6V2qwr5w2pjCLJ3bMPWjNF-mjeqMBs2kh8s1a2zTQ-Q-n95UoGFMWY4TqxM4SX4K06_aT2DEATT3EqvBIthdwdQ1rc7GWJ6VcDyH9lAFnG78sO8MdDrSDvB2rTcXKf8dr_8PkaABb8O9smRjok9C-5FMTnxFC_oQW4oK_Rf13gnHpvGtGfQNjYLM61gEAAA%3D%3D

[rustic gull]

I'm pulling the classic one here, trying to present a solution instead of the problem 😆 Is there anything that comes to mind just looking at the problem - placing a bound on the values of a TypedDict?

I'd like it to be Dict to represent a json form of the class - a single primative type like int isn't really json
I'm seeing the same warning in that playground link 🤔

[trim tangle]

wdym?

yeah, it complains if you provide an int

[rustic gull]

oh whoops, i misread, you're right

[trim tangle]

the trick is to use Mapping instead of dict

it's a read-only interface, so it won't let you mess up the typeddict

[rustic gull]

Yeah looks good thank you so much! 😁

The next problem is how I type JsonType, this is how I'm actually doing it:

JsonPrimatives = Optional[Union[int, float, str, bool, datetime]]
JsonPrimatives = Union[JsonPrimatives, Iterable[JsonPrimatives], Mapping[str, JsonPrimatives]]
JsonType = Mapping[str, JsonPrimatives]

With this definition of JsonType i get this error on the class User line...
Type "object" cannot be assigned to type "int | float | str | bool | datetime | Iterable[int | float | str | bool | datetime | None] | Mapping[str, int | float | str | bool | datetime | None] | None"

[trim tangle]

sounds like pyright is not smart enough

maybe you could open a discussion/issue on the pyright repo?

I would personally just leave it with no bound

[tranquil turtle]

Prim**>i<*tives

[rustic gull]

Thanks 👍

[plucky shoal]

ya cannot sum time and timedelta objects

[rustic gull]

sad times, i guess I will until i learn more about it... Thanks so much for all your help 😁

[rustic gull]

works for me on python 3.9:

>>> from datetime import datetime, timedelta
>>> datetime.now() + timedelta(minutes=1)
datetime.datetime(2022, 8, 28, 17, 21, 3, 719187)

[trim tangle]

!e

from datetime import datetime, timedelta, timezone

print(datetime.now(timezone.utc) + timedelta(days=5))

[rough sluice]

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

2022-09-02 16:20:49.533440+00:00

[trim tangle]

yeah you can

well, this is a datetime object, not a time object

[plucky shoal]

wow

is datetime.now(timezone.utc) the equivalent to datetime.utcnow() ??

[trim tangle]

no

[plucky shoal]

what would be the equivalent in order to perform the operation?

[trim tangle]

utcnow quite confusingly returns a naive datetime (i.e. without timezone info), whereas datetime.now(timezone.utc) returns a timezone-aware object

[trim tangle]

I don't think there's really a different way to do this

anyway, this is probably not the right channel for datetime stuff

[plucky shoal]

they r almost the same, I think I have to format it in order to look equal

[plucky shoal]

why it does not gives unsuported operand?

[rustic gull]

Because the only operand I gave was minutes=1 which is valid for timedelta

[soft matrix]

Thanks!

[uneven ginkgo]

Are context managers type hinted as Generators or iterators?

@contextmanager
def temporary_finding(
    apih: SwatAPI,
    obj_payload: List[Dict[str, Any]],
) -> Generator[List[Finding], None, None]:

or

@contextmanager
def temporary_finding(
    apih: SwatAPI,
    obj_payload: List[Dict[str, Any]],
) -> Iterator[List[Finding]]:

[tranquil turtle]

Look at @contextmanager signature

[trim tangle]

I think any iterator will work

[boreal ingot]

If you have a function taking a callback, and your function doesn't use the return value, should you type hint it as None or object?

[oblique urchin]

object, so that users can pass callables that happen to return something

[boreal ingot]

That's fair, I have always used None because it felt like the natural thing to do, but I guess I shouldn't

[trail kraken]

Is object preferred over typing.Any in this case?

[oblique urchin]

yes, only use Any if you really need to

[trail kraken]

What are the rationales for using object? The only things that come to mind are shorter and not having to import typing.

Are there common examples where you have to use Any?

[oblique urchin]

Any generally indicates that you cannot give a precise type because the type system isn't powerful enough or you haven't figured out the exact type required. Any is inherently unsafe

object explicitly indicates that all Python objects are accepted. In the callback case, it makes the implementation safer because it won't allow you to do random things with the return value of the callback

[trail kraken]

So... Any is like a todo placeholder, whereas object can be used to indicate "anything, and I don't care about its interface"?

[boreal ingot]

Basically yes

Any can also be used when the type system just isn't powerful enough for what you need

[trail kraken]

Sigh... I probably have some grep-ing to do.

[idle nimbus]

where do i put typy hints for for loops?

[heady venture]

for the variable?

[idle nimbus]

yes

[heady venture]

it should be above the loop

i: int
for i in range(10):
    ...

[idle nimbus]

200

[soft matrix]

generally you shouldnt have to do this

[trim tangle]

Yeah, it's kinda pointless

[tranquil turtle]

It it useful in weird situations when type inference doesnt work

[indigo locust]

Questing about type hinting callables

Callable[[Event], None]

This is to match a event that takes in an Event object and return nothing. (Event is a custom class)

If I want this to apply to both functions and methods, do I to write

Callable[[Event], None] | Callable[[Self, Event], None]

[fierce ridge]

no, the first parameter "goes away" when a method is bound to an instance. example:

class EventHandler:
    def __call__(self, event: Event) -> None: ...

the signature of EventHandler.__call__ is (self: EventHandler, event: Event) -> None but the signature of an instance method EventHandler().__call__ is (event: Event) -> None

the python instance method binding system is honestly one of the most clever and elegant designs i have ever seen in a piece of software

it's so much better imo than magic scoping rules and special placeholder variable names like in ruby and javascript

[indigo locust]

Agreeeeeed

[fierce ridge]

at least, it's elegant if your objects are going to be "hash tables in a trenchoat" anyway

but it's definitely harder to learn for newbies, at least compared to ruby

(javascript this is a whole other matter)

[indigo locust]

But the tl;dr is that static analysis tools automatically make the considerations for self (or cls) as needed

[fierce ridge]

yes, but in this case, the first parameter literally goes away

[indigo locust]

Specifically with regard to matching to Callable[Params, Return]

[fierce ridge]

that might be a slightly different story

how are you using ParamSpec here?

if you have something annotated with Callable[Params, Return], then yes, the "non-self" parameters of a bound instance method will match Params

[indigo locust]

I'm not, thankfully. I was just using that notation as a turn of phrase

[fierce ridge]

but the unbound method will not match

this is the distinction between str.replace("foo") and ("foo").replace()

it happens to work on strings specifically but it's not the same thing in general, and this self-binding business is why

[tranquil turtle]

You dont need brackets around str literal

[fierce ridge]

!e ```python
print("foo".replace("o", "_"))

[rough sluice]

@fierce ridge :white_check_mark: Your 3.11 eval job has completed with return code 0.

f__

[fierce ridge]

huh, i never thought to try that before

i know you need it for number literals

[tranquil turtle]

No, you need brackets or one space around integer literal
Its because 1.foo parses like 1. float literal and foo name

Valid:
'\n'.join(strs)
1.0.__add__
1 .__add__
().meth
[].meth
{}.meth
....meth

Invalid:
1.__add__

[fierce ridge]

huh, i never considered why it didn't work for integer literals

thanks for the demo

[brisk hedge]

Is the following a valid way to use ParamSpec? Pyright doesn't seem to complain...

T = TypeVar("T")
P = ParamSpec("P")
R = TypeVar("R")

Decorator: TypeAlias = Callable[[Callable[P, R]], T]

class Foo:
  ...

def deco() -> Decorator[[int, int], int, Foo]:
  def inner(fn: Callable[[int, int], int]) -> Foo:
    ...
  return inner

[oblique urchin]

looks right to me

[brisk hedge]

Just strange being able to pass [int, int] to aliases

pretty damn convenient, though

[soft matrix]

one thing that i do find weird about this stuff is the different representations of ParamSpec vs Callable the first shows parameters as a tuple/like pep 692 but callable has always shown it as using []

[brisk hedge]

Pyright generic inference just being inadequate here?

from typing import *

T = TypeVar("T")
U = TypeVar("U")

class Wrapper(Generic[T, U]):
    def __init__(self, fn: Callable[[T], U]):
        self.fn = fn
    
    def call(self, t: T) -> U:
        return self.fn(t)

def id(t: T) -> T:
    return t

Wrapper(id).call(1)
# Argument of type "Literal[1]" cannot be assigned to parameter "t" of type "T@foo" in function "thing"
#  Type "Literal[1]" cannot be assigned to type "T@foo"

[fierce ridge]

seems like it

[soft matrix]

i think inferring that isnt possible

[fierce ridge]

if it makes you feel better, mypy does something weird here too

[soft matrix]

you need to specialise wrapper

[fierce ridge]

you'd think that it should be able to figure out that T and U should be unified

https://mypy-play.net/?mypy=latest&python=3.10&flags=show-error-codes%2Cstrict&gist=f1160cb3b20e7934ba3d04e9fa5f5f52

[brisk hedge]

explicit type arguments work in this case

but won't for e.g. generic functions

or generic return values from decorators

Okay, so it doesn't unify types, fine, but you can't even pass generic values?

from typing import *

T = TypeVar("T")
U = TypeVar("U")

class Wrapper(Generic[T, U]):
    def __init__(self, fn: Callable[[T], U]):
        self.fn = fn
    def __call__(self, t: T) -> U:
        return self.fn(t)

def deco(fn: Callable[[T], U]) -> Wrapper[T, U]:
    return Wrapper(fn)

@deco
def foo(t: T) -> T:
    return t

def bar(t: T) -> T:
    return foo(t)
# Expression of type "T@foo" cannot be assigned to return type "T@bar"
#  Type "T@foo" cannot be assigned to type "T@bar"
#
# Argument of type "T@bar" cannot be assigned to parameter "t" of type "T@foo" in function "__call__"
#  Type "T@bar" cannot be assigned to type "T@foo"

Ouch

[soft matrix]

that does seem slightly more broken

mypy seems fine with it fwiw

[runic tapir]

https://github.com/python/mypy/pull/13557 is a swing at the first one.

[fierce ridge]

is it not possible to have narrowing on a non-final typeddict?

oh because then you don't know if it's the typeddict itself or it's subclass

that is going to be a super cool feature!

[brittle socket]

def filter_n(n: int, pred: UnaryPred[T], iterable: Iterable[T]) -> Iterator[T]:

    it = iter(iterable)
    i = 0
    while i < n:
        e = next(it)
        if pred(e):
            i += 1
            yield e

Do not raise StopIteration in generator, use return statement instead Pylint(R1708:stop-iteration-return)
So pylint complains about the e = next(it) because it can raise in a generator context.
How am I supposed to fix this? 😕

[soft matrix]

Can't you just iterate over iterable using a for loop?

But you can use the two argument form of next and then catch the None value

Or you could just turn off this error cause it seems pointless

[brittle socket]

I'm using while i < n because I'm not going through the whole iterable

[soft matrix]

You can use a break to achieve the same thing

[brittle socket]

Oh, indeed. Is that better practice?

[soft matrix]

I'd say so, seems much more pythonic to me

[sick notch]

https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.hist.html returns "matplotlib.AxesSubplot or numpy.ndarray"

However, I get an error when using the type-hinting

matplotlib.AxesSubplot

Any idea what I should do?

[sick notch]

It works when I type hint it as a numpy.ndarray but in my code, when printing type(), I get matplotlib.axes._subplots.AxesSubplot so I'm pretty sure it's the first object type (AxesSubplot) that is returned

[soft matrix]

sounds like you need to update the stubs for the function

[sick notch]

How can i update the stubs?

[soft matrix]

what ide do you use?

[tranquil turtle]

pip install types-<libname> -U

[brittle socket]

Is this what you had in mind?

def filter_n(n: int, pred: UnaryPred[T], iterable: Iterable[T]) -> Iterator[T]:
    i = 0
    for e in iterable:
        if pred(e):
            yield e
            i += 1
            if i == n:
                break

It does get rid of next and so the pylint warning, but...idk, feels a bit clunky with 3 levels of indentation

Oh wait, I have an idea

[brazen jolt]

you can always avoid the additional indent level by just doing if not pred(e): continue

also, why not just use normal filter

with enumerate?

[brittle socket]

i = 0
for e in filter(pred, iterable):
    yield e
    i += 1
    if i == n:
        break

bit better I think

Oh lol, yeah I can enumerate on this filter

[brazen jolt]

enumerate works with any iterable

[trim tangle]

yield from map(itemgetter(0), zip(filter(pred, iterable), range(n)))

Ok that's a bit cursed. What about

for e, _ in zip(filter(pred, iterable), range(n)):
    yield e

oh wait

[brazen jolt]

that's better yeah

[trim tangle]

!d itertools.islice

[rough sluice]

itertools.islice

itertools.islice(iterable, stop)``````py

itertools.islice(iterable, start, stop[, step])```
Make an iterator that returns selected elements from the iterable. If *start* is non-zero, then elements from the iterable are skipped until start is reached. Afterward, elements are returned consecutively unless *step* is set higher than one which results in items being skipped. If *stop* is `None`, then iteration continues until the iterator is exhausted, if at all; otherwise, it stops at the specified position. Unlike regular slicing, [`islice()`](https://docs.python.org/3/library/itertools.html#itertools.islice "itertools.islice") does not support negative values for *start*, *stop*, or *step*. Can be used to extract related fields from data where the internal structure has been flattened (for example, a multi-line report may list a name field on every third line). Roughly equivalent to:

[brittle socket]

def filter_n(n, pred, iterable):
    for i, e in enumerate(filter(pred, iterable)):
        yield e
        if i == n - 1:
            break

[trim tangle]

islice(filter(pred, iterable), n)

[brittle socket]

Oh

[trim tangle]

anyway... this seems quite far removed from type hinting 👀 how did it start

[brazen jolt]

at this point, I probably wouldn't even make this into a function

[trim tangle]

yeah you'd probably use islice with a genexpr

[brittle socket]

Yeah this is sexier

[brittle socket]

This started with a pylint warning on my use of next but it derailed a bit. Alright, thank you all 🙂

[pure raptor]

say why can I do list[ParentT] = [ChildT()] when list is invariant

[hallow flint]

type checkers are clever! they can infer the type of an expression based on what type is expected

remember, the unsafety that invariance solves is when the same object is sometimes treated as list[parent] and sometimes treated as list[child]. this isn't the case here: [child()] only ever gets treated as list[parent]

so e.g. type checkers will complain about:

x = [child()]
y: list[parent] = x

since x will be inferred as list[child].

the same object has been treated as a list[child] (when using the name x) and treated as list[parent] (when using the name y). this is unsafe, because there are things you can do to list[parent] that you can't do to list[child] such as add a parent to the list.

[pure raptor]

ah alright thanks for the explanation

[sick notch]

PyCharm

[sick notch]

ERROR: Could not find a version that satisfies the requirement types-matplotlib (from versions: none)
ERROR: No matching distribution found for types-matplotlib

I get this :/

[brisk heart]

matplotlib doesn't have any iirc

your best bet is using something like https://github.com/wearepal/data-science-types

or the pyright one

https://github.com/microsoft/python-type-stubs/tree/main/matplotlib

[fierce ridge]

is there any framework for using typing.Annotated to attach units of measurement to values?

x: Annotated[float, units.meter]
y: Annotated[float, units.meter]
a: Annotated[float, units.degree]

[soft matrix]

yes astropy does this

[fierce ridge]

sweet let me look up how they do it

i dont think i want to depend on astropy but i can at least get an idea of what they do

ideally it would also include a suite of predicates to check preconditions

Meters: TypeAlias = Annotated[float, units.meter]
Degrees: TypeAlias = Annotated[float, units.degree]

@units.check(auto=True)
def fwd(x: Meters, y: Meters, a: Degrees) -> tuple[float, float, float]:
    return tuple(pyproj.Geod(ellps='sphere').fwd(x, y, a))

(i'm just imagining things here)

[soft matrix]

they just have

class Distance:
    def __class_getitem__(cls, params):
        return Annotated[cls, params]
```or something

[fierce ridge]

huh, that's interesting

[soft matrix]

which isnt as useful as what youre describing

[fierce ridge]

well that's also useful because you could have Distance[int] and Distance[float]

[trim tangle]

at least in TypeScript it's theoretically possible to make a type-level unit system
after all, it's just a map of different dimensions being assigned a rational number
like {distance: 1/2, mass: 1, time: -1} is m^(1/2) kg / s

you'll just need to implement addition, subtraction, multiplication and division of rational numbers on the type level

[fierce ridge]

it's possible in python too with TypeGuard but the problem is that preserving units after any operation requires asserts or casts

that is, if you intend to enforce rules like "distance can't be <= 0"

(which arguably isn't a universal property anyway, since you can subtract a non-negative distance from something else)

[soft matrix]

https://docs.astropy.org/en/stable/units/type_hints.html

[fierce ridge]

x: Quantity = 2 * u.km

wow

[soft matrix]

yeah thats a cool astropy feature

but it doesnt know that x is Quantity[u.km] at type checking time which is a bit sad

[nocturne siren]

Can't seem to write e=mc^2 with that

[fierce ridge]

i guess this is one of the interesting use cases where "variable annotation" diverged from "type hinting"

and/or where a user-friendly interface was prioritized over type safety

ideally you'd re-unify them somehow

runtime functions for unit conversions but static annotations for unit declarations

[soft matrix]

well until python is typescript i think thats not possible

[nocturne siren]

I don't think it's really that possible even with typescript

I don't think typescript's typing does dimensional analysis

[fierce ridge]

basically * u.km could be an alias for typing.cast

how would typescript help?

maybe i didn't understand fix error's example

the only language i know of with built-in dimensional analysis is F#

[soft matrix]

can you not do this #type-hinting message?

[fierce ridge]

maybe i don't understand the example

can you not do that in python too?

the problem there (and in python) is that you're no longer able to just work with floats

[soft matrix]

i thought ts could handle floats fine as everything is just a number

[fierce ridge]

wait, i'm confused

[nocturne siren]

I think I get the concept, you map units to numbers

But the thing is __mul__ and __add__ or whatever likely need to be super complicated

[fierce ridge]

oh, you are talking about doing dimensional analysis, but not necessarily attaching units to actual numbers

i'm sure there are already libraries for this

like udunits or whatever

[spice oak]

hey, i am new to python and I am trying to use django pylint. This is working very well. However I always get the following error: "Django was not configured. For more information run pylint --load-plugins=pylint_django --help-msg=django-not-configured". In order to fix it in github actions and in vscode i need to provide the following flag: "--django-settings-module=your.app.settings [..other options..] <path_to_your_sources>". Since I am new to Django I dont know which path to provide....

[sick notch]

but why does Pandas say it returns a class that does not exist then? I'm a bit confused

(cf #type-hinting message @brisk heart)

[brisk heart]

cause that's just what your typechecker thinks ig

[sick notch]

ig?

wdym

[rare scarab]

You may need to hide your import in if TYPE_CHECKING:

Related stackoverflow: https://stackoverflow.com/questions/63783154/how-to-type-hint-a-matplotlib-axes-subplots-axessubplots-object-in-python3

[sick notch]

Not sure I understand

[sick notch]

Interesting, I might just do that

I don’t understand you first message though

[rare scarab]

That was my initial impression, thinking maybe matplotlib had some stubs in typeshed. That was not the case.

[hallow flint]

yeah, i think pandas just uses types from matplotlib even though they don't really exist. i think this is because the types originated with microsoft who do have some basic stubs for matplotlib (linked above by ashlen)

e.g. see discussion here https://github.com/pandas-dev/pandas-stubs/pull/33#issuecomment-1186272498

[rustic gull]

Hey guys, what suggestions do you have for pycharm?

Mypy is the fist thing I'll install

I tried searching, but stuff like quokka popped up, and not really python stuff

[short sapphire]

173: error: Argument 1 to "CameraController" has incompatible type 
"Callable[[Arg(Event, 'event'), Arg(Source, 'source'), DefaultArg(Optional[float], 'timestamp'), DefaultArg(Optional[int], 'command_id'), KwArg(Any)], None]";
expected
"Callable[[Arg(Event, 'event'), Arg(Source, 'source'), DefaultArg(Optional[float], 'timestamp'), DefaultArg(Optional[int], 'command_id'), KwArg(Any)], None]"

I feel like mypy is making fun of me

[trim tangle]

can you show the code?

These types do seem the same. But it's possible that the Source and Event are different between the two types. Can you show the imports?

[short sapphire]

Oh, sorry, must have missed this, It was an absolute import in place of a relative one 😳 PyCharm auto-imported it the wrong way

[trim tangle]

pycharm bad

jk, I once spent 3 hours debugging an issue that occurred because of an auto-import

granted, it was in JavaScript.

[short sapphire]

yea, this won't be the first or last time :/ sorry for the spam, i didn't want to open up a help thingy, I was just aggravated at the information density of that message 😅

[trim tangle]

the message is presented weirdly indeed

sounds like mypy doesn't bother formatting the arguments nicely

and the fact that the types are represented exactly the same

[short sapphire]

Yea, it wasn't formatted at all, i just made it so it's obvious they're the same. Thanks for the reply, now to finish fighting pylint with mypy and commit that. Cya

[nocturne siren]

Anyone tried ruff? Ping me if you have comments on it

[heady flicker]

Heard of it but not sure if it's worth it at this point for type hinting.

According to its github, It barely supports the stuff flake8. And flake8 is a weird choice for a type checker, imho :)

Great project and initiative in general though.

[brisk heart]

all of these projects with lackluster features whose only thing is that they're written in rust are getting kinda lame :/ If I wanted speed I would be writing my entire project in it

[tranquil turtle]

Flake8 isnt even a typechecker

[fierce ridge]

i think the idea is that the dev tooling can still benefit from being fast. but i think rust extension modules (using the hpy interface?) would be a lot more interesting than standalone rust applications

that and people just like doing things "in rust"

i think there's alot of benefit to dev tooling being as fast as possible

[heady flicker]

There definitely is! But it's the second priority. Being a good fully featured tool is the first priority.

I see a lot of tools that claim to be much faster simply because they do less stuff. Tortoise, for example. If you compare it with sqlalchemy.

However, pydantic v2 is exciting. Rewriting an already great library in Rust sounds pretty dope.

[signal sentinel]

How do you use the collections.abc.Coroutine? Anyone mind giving an example please

[brazen jolt]

do you understand how collections.abc.Generator works?

[signal sentinel]

No, I haven't used it

[brazen jolt]

well, most generators generally look like this: ```py
def count() -> Iterator[int]:
x = 0
while True:
yield x
x += 1

in here, count would have a type of collections.abc.Generator[int, None, None]

the first generic argument here is the type this generator is yielding

so that's the int

this is how most generators will work, and it will usually be sufficient

[signal sentinel]

Got it. And the second and third?

[brazen jolt]

however generators do have some extra things that they can do

specifically, you can send values to a generator, and you can also return a value from the function as a whole

those are the 2 other generic values

[signal sentinel]

I know a little on what generators are, but the documenting for the abc.Coro doesn't specify what the arguments it takes are

[brazen jolt]

so as an example, let's take a look a this: ```py
def foo() -> Iterator[int]:
sent = yield 0 # We yield 0, but also take in a sent value
while sent >= 0:
sent = yield round(sent)
return "Done"

in this case, the type of this generator could be this for example: collections.abc.Generator[int, float, str]

that is because the sent type is float, and the return type will be str

and you can use it like this:

[pastel egret]

Most of the time, really you don't need to annotate with Coroutine, just use Awaitable (which is the equivalent to Iterable for generators). The send and yield types aren't actually particularly important, since they're all internal to the async lib you're using. Unless you're writing that, you shouldn't need to care.

[signal sentinel]

Isn't there something like typing.Callable for coros?

[brazen jolt]

my_gen: collections.abc.Generator[int, float, str] = foo()
x = my_gen.send(2.5)  # send here acts like next, and gives you the yielded value, but it also sends another value back
y = my_gen.send(-1)
try:
    z = my_gen.send(5)
except StopIteration as exc:
    ret = exc.value

[pastel egret]

Yep, Awaitable is anything that can be await-ed, with the value being the final result.

[brazen jolt]

and the thing about a Coroutine is, that originally, python didn't have await keyword, so we were using yield instead

[pastel egret]

Coroutine[YieldT, SendT, RetT] inherits from Awaitable[RetT].

[brazen jolt]

which means coroutines supported the same things as generators, and you could send values to them, or return from them

you can still create a coroutine with using yield with a special decorator, I think it's asyncio.coroutine

but it's deprecated and it's possible that it's not a thing anymore, I'm not sure

but yeah, in vast majority of cases, Awaitable will be sufficient for you

[pastel egret]

I think it was removed. More importantly, there's objects that implement __await__() directly.

[brazen jolt]

as every Coroutine is an Awaitable

Coroutine just allows you to also specify the return and send types

if you want the whole function signature, instead of just it's return type, you can still use typing.Callable, you'd just do: Callable[[int, str], Awaitable[int]]

that's for a function that takes int and str as parameters, and gives you an int once awaited

[pastel egret]

Note that there's a special case in typing. async def func() -> T: implicitly means that the return type is wrapped in Coroutine[Any, Any, T]. So you don't need to use it in your function definitions, only if you want a callable coroutine or are doing an async generator.

[brazen jolt]

you could also use Callable[[int, str], Coroutine[int, None, None]]

[pastel egret]

The return type's the last one in Coroutine actually.

So here's an example then.

async def get_size(url: str) -> int:
    data = await something(url)
    return len(data)

int_funcs: list[Callable[[str], Coroutine[Any, Any, int]]] = []
int_funcs.append(get_size)

Usually I'd just do list[Callable[[str], Awaitable[int]]] though, no need to specifically indicate a coroutine.

[rustic gull]

I have a base class Serializable:

TSerialized = TypeVar("TSerialized")
class Serializable(Generic[TSerialized]):
    def serialize(self) -> TSerialized: ...

And a container class that can hold and serialize a Serializable:

TItem = TypeVar("TItem", bound=Serializable)
TItemSerialized = TypeVar("TItemSerialized")
class Container(Generic[TItem, TItemSerialized]):
    def __init__(self, item: TItem):
        self.item = item

    def serializeItem(self) -> TItemSerialized:
        return self.item.serialize()

If I create a Container, pyright is able to infer TItem from the constructor parameter:

class MyItem(Serializable[int]): ...

myItemContainer = Container(MyItem())
reveal_type(myItemContainer.item) # MyItem

By pyright is not able to infer TItemSerialized based on the inferred TItem:

reveal_type(myItemContainer.serialize()) # Unknown, should be int

Is there any way to let pyright infer this kind of... Transitive generic parameter? Without providing the 'inner' generic to the constructor?
I can't define Container as Generic[TItem[TItemSerialized]], because pyright doesn't understand higher types... Which I suppose might be the crux of my problem!

[opal dagger]

How do we type hint a tuple response types?

[rustic gull]

To give a Tuple of finite length, give each item as a generic type parameter, eg to return a Tuple with 2 ints and a str, you would write: Tuple[int, int, str].
To hint a variadic Tuple, give a single generic type parameter followed by ..., such as: Tuple[int, ...]

[trim tangle]

do you really need the TItemSerialized?

Could you perhaps return Serialized[TItem]?

Or maybe get rid of TItem? And just put Serializable[int] into TItemSerialized

can you perhaps explain the use case, or how you're going to use this class?

[rustic gull]

What would Serialized represent here?

I have a multiplayer game, where the player has inventories, each storing a single type of item, like Weapon, Ship, etc.
In places, I manipulate the JSON-serialized forms of items, and I'd like to statically type those serialized forms with TypeDicts (thanks again for your help with that bit!)

class SerializedInventory(TypedDict, Generic[TSerializedItem]):
  items: List[TSerializedItem]

class Inventory(Serializable[SerializedInventory], Generic[TItem, TSerializedItem]):
  def __init__(self, initialItems: List[TItem]):
    self.items = initialItems

  def serialize(self): SerializedInventory[TSerializedItem]:
    return [i.serialize() for i in self.items]

class SerializedWeapon(TypedDict):
  damage: int

class Weapon(Serializable[SerializedWeapon]): ...

weaponsInventory: Inventory[Weapon, SerializedWeapon] = Inventory([Weapon(), Weapon()])
serialized = weaponsInventory.serialize()
serialized[0]["name"] = "Super mega awesome laser" # ERROR: SerializedWeapon has no field "name"

in places I'll make references to items in a user's inventory, so would still like the TItem parameter
The ideal case would be if I didn't need to type hint weaponsInventory - it'd be fantastic if pyright could infer that since I have an inventory of Weapon, the serialized form of an item must be SerializedWeapon.

After this rubber duck debugging (i just got your profile picture lmao ), I suppose the solution here is to have helper types/aliases, like class WeaponInventory(Inventory[Weapon, SerializedWeapon]), and use those instead...

[trim tangle]

hmm perhaps you want something called "Higher Kinded Types"

which is not available in Python

[chrome lynx]

I'm using mypy for type checking, pytest for unit tests, and tox to run everything automatically. If I just run mypy mymodule mytests it works fine, if I run tox, mypy complains: error: Cannot find implementation or library stub for module named "pytest"

Tox uses the same version as I do... Feels like it should be something simple but no idea what's going on.

Started having the issue since I import pytest

Hmm, probably just need to add pytest to the dependencies

Yup that worked

[rustic gull]

Yeah I think so too 😔
Oh well, thanks so much for all your help as always! 😁

[twin lantern]

how do i annotate a __new__

or make the class a type

[spiral fjord]

classes are a type

[twin lantern]

undefined name

[spiral fjord]

You can either from __future__ import annotations or make the typehint a string.

[twin lantern]

how do i use the future annotations

[spiral fjord]

just importing it should be enough

[twin lantern]

oh it just works

ye

thank you

[rustic gull]

Alternatively, you can wrap the hint in quotes to defer type resolution :)

[tranquil wave]

how might i manually signal something to a type checker?

like in terms of low level stuff with typing

[void panther]

signal what?

[tranquil wave]

in this case i want to signal a union

is it even possible?

[void panther]

what does signal mean here? Mark some object as being an union or something else?

[tranquil wave]

yeah just telling the type checker that somethings a union

but without union ofc

[void panther]

I don't believe you can, though you may be able to do something with if typing.TYPE_CHECKING

[acoustic thicket]

typing.cast maybe?

[weak hull]

is there any way to annotate a function separately from its a definition? something like

def foo(*, a: int, b: int):
    # only the annotations
    ...

def foo(**kwargs):
    # the actual definition
    super().__init__(**kwargs)

[acoustic thicket]

type stub files are the closest i can think of

[weak hull]

seems I can do it like this

@typing.no_type_check
def foo(**data):  # type: ignore
    return data["a"] * data["b"]


def foo(*, a: int, b: int):
    ...

just need to get pyright to shut up about a and b not being accessed

[hallow flint]

if TYPE_CHECKING can work well for this; also makes it pretty clear what the intent is

[heady flicker]

I feel like no_type_check is more useful for metaprogramming or for making exceptions for frameworks that respect typehints such as fastapi.

As @hallow flint suggested, you should use:

if typing.TYPE_CHECKING:
    def foo(*, a: int, b: int):
        ...
else:
    def foo(**data):
        return data["a"] * data["b"]

[hallow flint]

(also the ordering in the no_type_check example is wrong)

[heady flicker]

Are you sure? I think that's his intent

I.e. The less specific case is the real case.

[hallow flint]

yes, so in OP's post, at runtime the not specific actual implementation is getting clobbered by the fake typing only definition

[heady flicker]

Oh, you're right.

[heady flicker]

Remember that type stubs (.pyi files alongside your .py files) are also an option

[nocturne dew]

How should I annotate my wrapper class, this wrapper class is used inside a decorator function

def log_call(obj: Callable[..., Any]) -> Callable:
    if type(obj).__name__ == 'function':
        return logger(obj)

    class Wrapper(obj):

src/log_call/__init__.py:42: error: Variable "obj" is not valid as a type
src/log_call/__init__.py:42: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
src/log_call/__init__.py:42: error: Invalid base class "obj"
Found 2 errors in 1 file (checked 1 source file)

https://github.com/Agent-Hellboy/log_call/blob/main/src/log_call/__init__.py#L42

[rough sluice]

src/log_call/__init__.py line 42

def __init__(self, *args, **kwargs):```

[heady flicker]

You're dynamically defining a class so my first intuition would be to refactor your system to be less complex first.

But let me fix what I can fix:

P = ParamSpec("P")
R = TypeVar("R")

def log_call(obj: Callable[P, R]) -> Callable[P, R]:
    if inspect.isfunction(obj):
        return logger(obj)

This is a bit better.

However, that's not solving your core issue. Let's try to use a Protocol for that

[nocturne dew]

Thanks man

[heady flicker]

from typing import Callable, ParamSpec, TypeVar
import inspect

P = ParamSpec("P")
R = TypeVar("R")

def logger(func: Callable[P, R]) -> Callable[P, R]:
    @functools.wraps(func)
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
        log_stmt = get_log_stmt(func, args, kwargs)
        logging.info(f"{log_stmt}")
        return func(*args, **kwargs)
    return wrapper


def log_call(obj: Callable[P, R]) -> Callable[P, R]:
    if inspect.isclass(obj):
        class Wrapper:
            def __init__(self, *args: P.args, **kwargs: P.kwargs) -> None:
                self.wrapped_obj = obj(*args, **kwargs)

            def __getattr__(self, attr: str) -> Any:
                attr_value = getattr(self.wrapped_obj, attr)
                return logger(attr_value) if callable(attr_value) else attr_value

        return Wrapper # type: ignore
    else:
        return logger(obj)

Next time you need to check whether something is a function (or use introspection for anything else in python), I suggest using inspect.

1. It is properly typehinted on typeshed so pyright/mypy will type check it quite well
2. It will decrease the chance of error
3. It's quite easy to use

this implementation is problematic in terms of isinstance checks. You can use __subclasscheck__ to fix that.

For some reason, both mypy and pyright didn't handle inheritance well.

[nocturne dew]

Why __getattr__ instead of __getattribute__? As the attribute lookup starts from __getattribute__ if it is implemented?

I have no idea about __subclasscheck__?
Can you give me an example when it can be problematic?

why did you create a wrapped_obj and use __getattr__?
I will incorporate your introspection and typing hinting changes.

[heady flicker]

1. __getattr__ is called after an unsuccessful lookup on the wrapper. I.e. If we want to get access to the original object, we can do: log_call(ObjType).wrapped_obj. It's not ideal but I'll need to know your specific use cases to provide a better implementation. I guess if you want to fully mimic the object, __getattribute__ is gonna be better.
2. My implementation would be problematic in the following cases: isinstance(log_call(ObjType), ObjType) # -> False and log_call(ObjType).__dict__ # -> wrapper dict, not object dict
3. __subclasscheck__ allows you to override how isinstance works. I.e. It's called on isinstance checks and returns a bool.

[nocturne dew]

There are no specific usecase, this can be used on the server(maybe others) monitoring scripts.
Thanks for your suggestion. I will try to improve this.

[nocturne dew]

Ohh, one more thing I am going to support 3.7, 3.8,3.9. are ParamSpec and TypeVar available in 3.7?

[soft matrix]

youll need typing_extensions for ParamSpec

[heady flicker]

Just do pip install typing_extensions and import newer things from there and you're all set

[nocturne dew]

Yes, i get this 👍

[weak hull]

yeah you were right, but I just switched it to if TYPE_CHECKING branches as I should have

[azure kernel]

def function(arg : int) -> int:
    return int - 1

is this correct

I always forget if the colon is after or before the arrow

[hallow flint]

the annotation is correct

although you probably meant to do return arg - 1 🙂

but hey that's what type checkers are form

maybe a way to remember is "everything after the colon is the function body, everything before is the signature"

[solid light]

Also should strictly be arg: int and not arg : int

[azure kernel]

oh yes sorry

[hallow flint]

( arg : int and arg: int are both perfectly valid python code; it's just a matter of style preference. most people choose the second one )

[rustic gull]

Guys, anyone using VSC and mypy?

hints get recognized withing the same folder level, but now when I'm importing from an upper folder

• main
• package
  +- file1
  +- file2

Like, file1 and file2 hints work well when mixing, but they won't work for main, unless I've a stub

[brisk hedge]

have you added an empty __init__.py in package?

[rustic gull]

I did

https://github.com/removed-usr/temp-repo-for-sharing-issues/blob/main/README.md

With or without it, it doesn't seem to matter

Hi, is this graph: Graph = Graph(azure_settings) some type of type hint?

[soft matrix]

yes

but thats probably an unnecessary in most cases

[rustic gull]

Thx, I'm working with Graph API and found this in repo examples 🙂 from witych PEP it is? 🙂

[soft matrix]

!pep 526

[rough sluice]

**PEP 526 - Syntax for Variable Annotations**

Link

Status

Final

Python-Version

3.6

Created

09-Aug-2016

Type

Standards Track

[rustic gull]

I also tried with MYPYPATH and .mypy.ini btw

[sinful oxide]

have you tried doing import mod.foo

[runic sleet]

What do you mean hints don't work well?

[rustic gull]

Same error, basically, but now it adds the same error for mod.foo

[rustic gull]

I'm having problems with subpackages/modules/directories

[rustic gull]

I've the code here

[sinful oxide]

what if you add __all__ = ["simple_sum", "simple_mul"] in mod/foo.py

[rustic gull]

I did it, doesn't work (added in on the very beginning)

Now I'm getting python errors related to python haha

[rustic gull]

Great name, btw

[sinful oxide]

btw have you tried changing .sum() to .simple_sum()

[rustic gull]

I did, I didn't update the files in github I guess, I've tried many stuff since I tried the github files haha

Update them

Thanks btw

If anyone could prove a simple working example, I think that would be better

[rustic gull]

@brisk hedge @sinful oxide

So guys, I solved it, I think you both said it

my package was indeed missing __init__.py

Then, how? If you suggested it?
So, you will see, my package had __init__,py...

Using a , instead of ., thank you so much for helping me. 💖

[near prairie]

from typing import Any, Type, TypeVar

from jsf import JSF
from pydantic.main import BaseModel


# How do I denote that T must be of type BaseModel
# But that the function will return an initialsed version of the given `klass` type (T)
T = TypeVar("T")
def c(klass: Type[T], **override: Any) -> T:
    fake = JSF(klass.schema())
    initialised_klass: T = fake.generate()
    for attribute_name, attribute_value in override.items():
        setattr(initialised_klass, attribute_name, attribute_value)
    return initialised_klass

I wrote the question as a comment, hoping for some guidance :D

oh, by using bound:

from typing import Any, Type, TypeVar

from jsf import JSF
from pydantic.main import BaseModel


T = TypeVar("T", bound=BaseModel)
def c(klass: Type[T], **override: Any) -> T:
    fake = JSF(klass.schema())
    initialised_klass: T = fake.generate()
    for attribute_name, attribute_value in override.items():
        setattr(initialised_klass, attribute_name, attribute_value)
    return initialised_klass

[brisk heart]

thought the convention was cls rather than klass in python 🤔

[rare scarab]

klass is java

[stoic warren]

ran into an issue with a class im writing, it boils down to this: ```py
T = TypeVar("T")
S = TypeVar("S")

@dataclass
class Foo(Generic[T, S]):
t: T

def foo(self, s: S) -> S:
    return s

x = Foo(5).foo("hello")
``` here, mypy infers that Foo(5) has type Foo[int, <nothing>] so complains that <nothing> != str on .foo. similarly, pyright infers it as Foo[int, Any], which at least type checks but isnt very useful

is this just a limitation of python typing? in a stronger typed language (eg Haskell) i'd expect that Foo(5) : Foo[Int, a], and that type variable can be monomorphised later

[trim tangle]

I had a long and unfruitful discussion with the pyright maintainer about this in a different case, seems like we didn't quite understand each other

basically no, there are no "polymorphic objects"

Well, apart from generic functions and protocols with generic methods, kinda.

[stoic warren]

ugh, thats very disappointing. i feel like that kills most uses of Generic with multiple arguments right?

i was trying to write some type-safe Haskell style parser combinators but without that its not very feasible :(

[trim tangle]

hmm well, there's stuff like Mapping

[trim tangle]

You can make a generic function that returns something, as a bit of a hack. I suppose.

[stoic warren]

oh yeah of course

[stoic warren]

ill try that, not sure if i can do everything i need to with just functions

dont any PEPs discuss this behaviour?

[trim tangle]

I don't think so

[trim tangle]

you can make a discussion on https://github.com/python/typing

[stoic warren]

will do :) thanks for the help

[heady flicker]

That's not a first time with Eric. 90% of discussions with him end up like that. But I guess he has too much on his plate so he probably needs to manage his time well.

[heady flicker]

Share the thread, please. Maybe we can revive it. Last time I had an argument with Eric, I was able to convince him by calling up a lot of devs that agreed with me.

[oblique urchin]

In fairness he often writes very detailed and helpful responses on the pyright issue tracker

Though I'd agree he can be a bit too fixed in his views

[heady flicker]

True-True. His replies definitely paint his views well and leave little place to misunderstanding.

[heady flicker]

Such discussions should go to the typing-sig mailing list, I think.

[oblique urchin]

No, GH discussions are a good first stop

The mailing list is a better fit for a developed proposal

[heady flicker]

Okie, thanks

[trim tangle]

https://github.com/microsoft/pyright/discussions/1380

I might have want for a bit of a rant there...

[heady flicker]

Actually, here I rather agree with Eric.

[trim tangle]

so how would you implement what I want in that case?

[heady flicker]

I don't think that case is common enough to be supported.

[trim tangle]

yeah I guess that's true

[heady flicker]

Lambdas are one of those "badly typed" things in python.

Like it's really hard to properly typehint how a metaclass works

[trim tangle]

Well it's not about lambdas. You could do the same with normal functions

[heady flicker]

It was also pretty hard to typehint dicts before we got typeddicts

[heady flicker]

Assign them to a variable?

[trim tangle]

oh, you're not talking about the thread?

[heady flicker]

I am talking about that thread

Give a specific example. That thread had too many

[trim tangle]

Well, for example:

@dataclass
class Lens(Generic[S, T, A, B]):
    getter: Callable[[S], A]
    setter: Callable[[S, B], T]

def foo(ab: tuple[A, B]) -> A:
    return ab[0]

def bar(ab: AB, c: C) -> tuple[C, B]:
    return (c, ab[1])

first: Lens[tuple[A, B], tuple[C, B], A, C] = Lens(foo, bar)

[trim tangle]

or this

[trim tangle]

hmm actually, you can just remove the second parameter can't you?

@dataclass
class Foo(Generic[T]):
    t: T

    def foo(self, s: S) -> S:
        return s

then it's just a class with a generic method.

but I assume there's something more involved in the real code

[trim tangle]

this all started with Haskell which lets you do:

data Lens s t a b = Lens (s -> a, s -> b -> t)

first :: forall a, b, c. Lens (a, b) (c, b) a, c
first = Lens (\(a, _) -> a) (\(a, b) c -> (c, b))

[heady flicker]

In this case, the problem is with the lack of a default generic type. I think someone has already proposed it somewhere (found it: https://peps.python.org/pep-0696/)

[trim tangle]

how would a default generic type solve it?

[heady flicker]

The type of S would stop being "nothing"

[trim tangle]

Basically I want to be able to later say:

first_int_str: Lens[tuple[int, str], tuple[bool, str], int, bool] = first

[heady flicker]

Otherwise, if he doesn't want it to be anything, then type checkers are right

[heady flicker]

Okay. So you want to use one of haskell's features essentially. Why?

What's the use case where your proposal is more readable or significantly more optimized than what we already have?

[trim tangle]

I was working on a library which made extensive use of immutable values (for a good reason). It's a bit of a pain to transform nested immutable values though.

Pointer = Pair[Rectangle, Pair[Circle, Triangle]]

def scale_pointer_circle(pointer: Pointer) -> Pointer:
    return Pair(pointer.left, Pair(Circle(pointer.right.left.x, pointer.right.left.y, pointer.right.left.radius * 2), pointer.right.right))

whereas with a mutable equivalent it would be

def scale_pointer_circle(pointer: Pointer) -> Pointer:
    pointer.right.left.radius *= 2

So in Haskell you can construct so called "lenses" which are a convenient notation for transforming nested values. They work something like this:

def scale_pointer_circle(pointer: Pointer) -> Pointer:
    return (right + left + circle_radius).change(lambda r: r * 2)

the issue is that it's impossible to type left and right in Python

I suppose you could reduce the mess a bit with specialized methods ```py
def scale_pointer_circle(pointer: Pointer) -> Pointer:
return pointer.with_right(pointer.right.with_left(pointer.right.left.with_radius(pointer.right.left.radius * 2)))

[soft matrix]

What should the default be?

[heady flicker]

It depends on the author and his intent :)

[heady flicker]

This sounds like a perfect solution, tbh.

That's how pandas does things and that's how we did things on my prior workplace where we worked with immutable objects

[trim tangle]

but that's kinda horrifying

although... the correct solution would probably be to go use Haskell instead 😄

[heady flicker]

Exactly :D

[trim tangle]

* leaves Python Discord *

[stoic warren]

yeah, in my case i'm trying to replicate a Parser a b over some token type a and some return type b. in a parser like choice : [Parser a b] -> Parser a b, the type of the token type is completely polymorphic, even when choice is instantiated with concrete parsers. i cant see a way to replicate this behavior with a python class currently, because unbound type variables are erased to <nothing>

and i think i agree about using haskell instead, python may not be designed for this :) but i'd love to be able to define type-safe parsers, like ```py
int_p = regex("[0-9][1-9]*").map(int)

vec_p = (text("Vec(") >> int_p)
.then(text(",") >> int_p << text(")"))
.map(Vec)
``` and the vec_p parser is known to produce a Vec2 type

[fierce ridge]

def between(
    x: T,
    lo: T,
    hi: T,
    include_left: bool = False,
    include_right: bool = False,
) -> bool:
    a = (x >= lo) if include_left else (x > lo)
    b = (x <= hi) if include_right else (x < hi)
    return a and b

should i use a covariant, invariant, or contravariant T here?

i'm thinking that maybe i should use either invariant T, or a much more generic protocol like SupportsComparisons

[trim tangle]

variance doesn't really exist in function definitions

[fierce ridge]

as in, i should be fine with an invariant typevar then?

[trim tangle]

yes

[fierce ridge]

isn't that overly-restrictive? maybe a protocol is what i want then

[trim tangle]

Well, you need to be able to compare all three

actually this is kinda tricky

I would use a protocol that supports comparisons with itself and use a bound typevar

class Ord(Protocol):
    def __lt__(self, other: Self) -> bool: ...
    def __le__(self, other: Self) -> bool: ...

O = TypeVar("O", bound=Ord)

[fierce ridge]

ah, that's interesting

from typing import Protocol, TypeVar

T = TypeVar('T')
T_contra = TypeVar('T_contra', contravariant=True)

class SupportsLessThan(Protocol[T_contra]):
    def __lt__(self, other: T_contra) -> bool: ...
    def __lte__(self, other: T_contra) -> bool: ...

class SupportsGreaterThan(Protocol[T_contra]):
    def __gt__(self, other: T_contra) -> bool: ...
    def __gte__(self, other: T_contra) -> bool: ...

def between(
    x: T_contra,
    lo: SupportsLessThan[T_contra],
    hi: SupportsGreaterThan[T_contra],
    include_left: bool = False,
    include_right: bool = False,
) -> bool:
    a = (lo <= x) if include_left else (lo < x)
    b = (hi >= x) if include_right else (hi > x)
    return a and b

mypy didn't like this one...

main.py:21: error: Unsupported left operand type for <= ("SupportsLessThan[T_contra]")  [operator]
main.py:22: error: Unsupported left operand type for >= ("SupportsGreaterThan[T_contra]")  [operator]
main.py:23: error: Returning Any from function declared to return "bool"  [no-any-return]
Found 3 errors in 1 file (checked 1 source file)

(it complained about using invariant T and told me to use contravariant)

[soft matrix]

whats lte?

[fierce ridge]

meanwhile this worked

from typing import Protocol, TypeVar

T = TypeVar('T')
T_contra = TypeVar('T_contra', contravariant=True)

class SupportsLessThan(Protocol[T_contra]):
    def __lt__(self, other: T_contra) -> bool: ...
    def __lte__(self, other: T_contra) -> bool: ...

class SupportsGreaterThan(Protocol[T_contra]):
    def __gt__(self, other: T_contra) -> bool: ...
    def __gte__(self, other: T_contra) -> bool: ...

def between(
    x: T_contra,
    lo: SupportsLessThan[T_contra],
    hi: SupportsGreaterThan[T_contra],
    include_left: bool = False,
    include_right: bool = False,
) -> bool:
    a = lo.__lte__(x) if include_left else lo.__lt__(x)
    b = hi.__gte__(x) if include_right else hi.__gt__(x)
    return a and b

[soft matrix]

isnt the dunder le?

[fierce ridge]

oh

LOL

[soft matrix]

and ge

[fierce ridge]

hah that fixed it

from typing import Protocol, TypeVar

T = TypeVar('T')
T_contra = TypeVar('T_contra', contravariant=True)

class SupportsLessThan(Protocol[T_contra]):
    def __lt__(self, other: T_contra) -> bool: ...
    def __le__(self, other: T_contra) -> bool: ...

class SupportsGreaterThan(Protocol[T_contra]):
    def __gt__(self, other: T_contra) -> bool: ...
    def __ge__(self, other: T_contra) -> bool: ...

def between(
    x: T_contra,
    lo: SupportsLessThan[T_contra],
    hi: SupportsGreaterThan[T_contra],
    include_left: bool = False,
    include_right: bool = False,
) -> bool:
    a = (lo <= x) if include_left else (lo < x)
    b = (hi >= x) if include_right else (hi > x)
    return a and b

thank you 😅

i'm still having a hard time wrapping my head around the variance here

[trim tangle]

oh I just figured it out

it was quite puzzling lol

[fierce ridge]

https://mypy-play.net/?mypy=latest&python=3.10&flags=show-error-codes%2Cstrict&gist=8a1b8fd850dfabbc41fa4f92ca8e743f if anyone wants to mess with this in mypy-play

[trim tangle]

https://pyright-playground.decorator-factory.su/?gzip=H4sIACXAGGMC_5WSwW6EIBCG7z7F3NTE-gCk7rG99NCkpJemMaijkrBgBmx3376Ia6xuD5YTzHz_zM9AS-YM7jpI3YE8D4YcvJJxpjYqA34d8F1QFHEolkMS8ziNeFkb7Uhs4rdYnMG8-RIkhXYFpxHTKKqVsBbexmHqYl_QWt4LnSztPhb9Z8oi8KvBFspSubJMLKo2A-N6JAYLl8LDCSpjFIM8z39L8KBkZ-mZUDikI666_7vqDrua-ArdN6JOQoHLymYhoAy7G-TqdGZ6yf662R6TulZjg35orWPBhX_TJ6EsbvMku_4OWJ0HdvoOiTLwWMAlBdluigN6yZz22cBXE99LOO350OwmmPKLgNCNpH0foRuofgAOTvrkvAIAAA%3D%3D

[brazen jolt]

that's one heck of a link, lol

oh it's gzipped

[fierce ridge]

wait how do i actually run something in pyright-playground again? there's no "run" button

[trim tangle]

It runs continuously

try ```py
x: str = 42

[fierce ridge]

too hip for me

[brazen jolt]

it's quite nice as it feels like an IDE imo

[trim tangle]

a very bad IDE lmao

[brazen jolt]

yeah lol

[trim tangle]

I used a gzip link, idea is stolen from the TS playground

this way I don't need to store any state

[brazen jolt]

yeah, that is nice

[fierce ridge]

its a good idea

[trim tangle]

the links are enormous though

[brazen jolt]

might be worth it to just integrate some paste service though

[fierce ridge]

like gzip or python discord paste?

[brazen jolt]

something like paste.gg has a public api

[trim tangle]

but then I need to worry about storage and GDPR and not accidentally deleting stuff...

[brazen jolt]

you wouldn't

paste.gg would

[trim tangle]

oh I'll just save on that service

interesting

[fierce ridge]

instead of ?gzip you'd have ?pydispaste or ?gist or whatever

[brazen jolt]

yeah

[trim tangle]

I'm working with the Google Drive API right now so I might use that 😂

[brazen jolt]

lol

[soft matrix]

you have gist support dont you?

[trim tangle]

I don't

oh wait

[soft matrix]

its just a bit hard to use

[trim tangle]

I do yes

I forgot what I implemented

and github issues

[brazen jolt]

gh issues?

[trim tangle]

https://pyright-playground.decorator-factory.su/?issue=microsoft/pyright/3913

[brazen jolt]

what if there were more codeblocks?

[trim tangle]

IIRC it picks the first one

[brazen jolt]

that's interesting, not sure how many people will use it though

[trim tangle]

yeah it's kinda... random

I implemented because I could I suppose

[brazen jolt]

I remember making a discord bot like that like 2 years ago, it had so many completely pointless features, oh well, it was fun to make

[trim tangle]

did it have a todo list?

[brazen jolt]

oh yeah

it had reminders that supported timezones

(before discord's timestamps)

[trim tangle]

only on Earth though?

[brazen jolt]

fair point

lmao

[trim tangle]

like, Earthly timezones

[brazen jolt]

but I had a stardate converter

(from star trek)

[trim tangle]

Imagine going all the way to the moon to find that your favourite discord bot doesn't work

[brazen jolt]

it did support relative timing I suppose

like, in x minutes

anyway, probably way too off-topic

[sick notch]

Anyone would have the answer to this stackoverflow? https://stackoverflow.com/questions/73475146/pyodbc-has-a-pyi-file-but-mypy-doesnt-see-the-stub-file

I have the same error but no one answered

[oblique urchin]

the linked issue seems pretty clear, they put the stub in the wrong place

also doesn't sound like there's a py.typed file

[sick notch]

What’s that?

[sick notch]

So I should do what? I’m not very good with the notion of stub files

I know what they look like but that’s all

[sacred spindle]

have a question regarding typing and numpy (keep in mind I'm using numpy 1.20 so I don't have access to the numpy mypy plugin)

def my_func() -> np.ndarray:
    a = []
    for i in range(10)
        a.append(i)
    b = np.asarray(a)  # type-error here saying incompatible type assignment between np.ndarray and List[Any]

so I then try to specify the type of a as numpy.typing.ArrayLike but then I get a complaint about how that doesn't have an append method. Any suggestions?

[soft matrix]

is arraylike not a protocol?

[fierce ridge]

i think it's a really complicated Union... but List[Any] should be a valid ArrayLike as far as i know

[fierce ridge]

you might need to enable the mypy numpy plugin... i don't know if any of the numpy typing stuff works without it, i've never tried

[sacred spindle]

here is what ArrayList is: "Union[Union[int, float, complex, str, bytes, generic], Sequence[Union[int, float, complex, str, bytes, generic]], Sequence[Sequence[Any]], _SupportsArray]" has no attribute "append"

numpy mypy plugin is available in 1.21+

I'm stuck on 1.20 for ... reasons

[soft matrix]

yeah ok so it should work, this looks like a mypy bug?

[fierce ridge]

list is a subclass of Sequence, so that should be okay... i think

[sacred spindle]

(this is on python 3.7 in case that makes a difference)

[soft matrix]

can you send the full error?

[sacred spindle]

error with ArrayLike

Item "Sequence[Union[int, float, complex, str, bytes, generic]]" of "Union[Union[int, float, complex, str, bytes, generic], Sequence[Union[int, float, complex, str, bytes, generic]], Sequence[Sequence[Any]], _SupportsArray]" has no attribute "append"

[soft matrix]

without the annotation

i mean if it isnt complaining about annotating it, it should be fine to pass to the function np.asarray

which is weird

i think the most sure fire way mypy would accept this would just be py def my_func() -> np.ndarray: b = np.asarray([i for i in range(10)])but id be pretty confident that you arent just doing that

[sacred spindle]

actual copy/paste of my code here..

        z = []
        for cr in uv:
            cr_int = np.int32(np.rint(cr))
            z.append(dsm[cr_int[1], cr_int[0]])

...

        z = np.asarray(z)

results in

...321: error: Incompatible types in assignment (expression has type "ndarray", variable has type "List[Any]")

[soft matrix]

oh i see

just change the name of the assignment of one of the variables

/shrug this would be fine with pyright but

[sacred spindle]

but putting the annotation....

        z: npt.ArrayLike = []
        for cr in uv:
            cr_int = np.int32(np.rint(cr))
            z.append(dsm[cr_int[1], cr_int[0]])
...
         z = np.asarray(z)

results in ...

error: Item "int" of "Union[Union[int, float, complex, str, bytes, generic], Sequence[Union[int, float, complex, str, bytes, generic]], Sequence[Sequence[Any]], _SupportsArray]" has no attribute "append"
...
error: Item "Sequence[Sequence[Any]]" of "Union[Union[int, float, complex, str, bytes, generic], Sequence[Union[int, float, complex, str, bytes, generic]], Sequence[Sequence[Any]], _SupportsArray]" has no attribute "append"
error: Item "_SupportsArray" of "Union[Union[int, float, complex, str, bytes, generic], Sequence[Union[int, float, complex, str, bytes, generic]], Sequence[Sequence[Any]], _SupportsArray]" has no attribute "append"

[soft matrix]

        z = []
        for cr in uv:
            ...

        y = np.asarray(z)```

is what im suggesting

[sacred spindle]

got it

thanks

that looks to have worked, thanks!

[rare scarab]

I was looking at typing#1216 today. Are there any proposals or ideas on how that would work?
https://github.com/python/typing/issues/1216

it was apparently dropped from PEP 646 to be covered later.

[soft matrix]

the original draft had a Map special form

are you familiar with asyncio.gather's signature?

if Map was added its massive set of overloads would simply become

Ts = TypeVarTuple("Ts", bound=Awaitable[Any])
                      # ^^^^^^^^^^^^^^^^^^^^ pretend this exists
def gather(*aws: *Ts) -> Map[list[asyncio.Future[T]], Ts]: ...

i think

[oblique urchin]

except we still can't have heterogeneous lists 🙂

[soft matrix]

oh yeah

[sacred spindle]

speaking of TypeVar I just ran into an issue which is clearly breaking my understanding of it...

from typing import TypeVar

class Polygon:
    pass

class Circle(Polygon):
    pass

class Square(Polygon):
    pass

class Length:
    pass

T = TypeVar("T", bound=Polygon)

def makePolygon(shape:str) -> T:
    if shape == "circle":
        return Circle()
    elif shape == "square":
        return Square()
    else:
        raise NotImplementedError

running mypy on this I get the following errors:

typevar_example.py:21: error: Incompatible return value type (got "Circle", expected "T")
typevar_example.py:23: error: Incompatible return value type (got "Square", expected "T")

Shouldn't TypeVar bound capture subclasses?

[soft matrix]

that sucks

[trim tangle]

A typevar is assigned to a particular type eventually. So e.g. if you "solve" T to be Square, that function won't fit

[oblique urchin]

the signature of makePolygon doesn't really make sense because there's nothing to tell the type checker what T is

generally a TypeVar should appear in both the arguments and the return type

[trim tangle]

actually pyright should complain about a typevar appearing only once

[trim tangle]

so imagine you had this:

def copy_polygon(poly: T) -> T:
    if random.choice() > 0.5:
        return Circle()
    else:
        return Square()
``` this would surely not be right

[sacred spindle]

https://tenor.com/view/good-burger-some-of-these-gif-12933391

sorry I'm being slow to respond, I'm having one of these moments...

[trim tangle]

yeah pyright says this is no bueno

which makes sense, as Jelle said, a typevar exists to "connect" two types together

so using it only ones is kinda grey territory

[sacred spindle]

(I'm working on a machine I don't typically work on, should probably make sure pyright is actually working )

[trim tangle]

are you using pyright or mypy?

[sacred spindle]

mypy

[trim tangle]

hmmm mypy doesn't complain

well you can use this thing if you don't want to install it locally

[sacred spindle]

definitely complains for me

[trim tangle]

oh I mean about the typevar appearing only once

[sacred spindle]

for context I inheritted a codebase that's in pretty good shape for the most part, I think they used TypeVar inappropriately ... (but I've never used it so..) anyway I don't think I even need TypeVar in this case, I just need to say the return object is of type Polygon ...

[leaden oak]

Yeah that would make more sense.

[soft matrix]

(mypy should actually complain on master)

[leaden oak]

If they were trying to link strings with the specific concrete type, then you'd have to use a bunch of overloads instead

[sacred spindle]

I have a base-class, the type annotation in this case wants to return that I'm returning something that inherited from that baseclass...

no issue specifying the return-type as being that base class tho

[leaden oak]

@overload
def makePolygon(shape: Literal["circle"]) -> Circle: ...
@overload
def makePolygon(shape: Literal["square"]) -> Square: ...

def makePolygon(shape: str) -> Polygon:  # or NoReturn?
    if shape == "circle":
        return Circle()
    elif shape == "square":
        return Square()
    else:
        raise NotImplementedError

[trim tangle]

I have a mini-tutorial in an... alpha version. Maybe it can provide some help
https://decorator-factory.github.io/typing-tips/tutorials/generics/

yeah I have seen typevars used incorrectly

[sacred spindle]

thank you!!!

[leaden oak]

You can do this if you need the better type inference, but honestly it might not be needed

[soft matrix]

personally id do ```py
@overload
def makePolygon(shape: Literal["circle"]) -> Circle: ...
@overload
def makePolygon(shape: Literal["square"]) -> Square: ...
@overload
def makePolygon(shape: str) -> Never: ...

[leaden oak]

Hmm, I thought mypy required you to have the concrete implementation's type be valid for all overload types (like a supertype or an union) because it refuses to infer the concrete types for you

[sacred spindle]

ooo forgot about "overload" ...may try that one ... is there something like Literal but tests for membership ...for example a Literal Union? ... LiteralUnion["circle","oval","ellipse"] -> Circle ?

[trim tangle]

Literal["circle","oval","ellipse"]

which is like Union[Literal["circle"], Literal["oval"], Literal["ellipse"]]

[sacred spindle]

🤦 that's what I get for typing before glancing at the docs

thanks

[leaden oak]

err, this is not relevant since I misread your example @soft matrix, I didn't realize the last was an overload too

[sick notch]

Up 😦

[leaden oak]

(salt rock lamp, I accidentally mentioned you, apologies for that!)

[rustic gull]

Trying to configure mypy is a pain the ass

Might as well drop it, ngl

[sacred spindle]

I need to get a lot more familiar w/ type annotations since type annotating pyqtgraph/pyqtgraph is likely in my not too distant future

[rare scarab]

I hope that gets a "type comprehension" syntax.

async def gather(*coros: *(Awaitable[T] for T in Ts)) -> Ts: ...

maybe Awaitable[T] type for T in Ts

[soft matrix]

thatd be nice

[trim tangle]

yep TypeScript has that (even more general perhaps) and it's awesome

[plain dock]

if i have some function that contains a return in an if inside a loop, and the algorithm of the function guarantees that that if will eventually succeed, is there a way to type hint that? Type checker (pyright) is complaining about the end-of-function-block implied return None it doesn't know can't be reached.

contrived example:

def foo(n: int) -> int:
    if not (isinstance(n, int) and n>0):
        raise TypeError('n must be positive int')
    for elem in range(n*100):
        if elem == n*2:
            return elem

[trim tangle]

put raise AssertionError at the end

[oblique urchin]

see also https://typing.readthedocs.io/en/latest/source/unreachable.html

[deep pendant]

Any idea how to inherit PlayerStatusUpdate from PlayerStatusData **and use total=False **?

[soft matrix]

You want to make the keys optional in a subclass?

[deep pendant]

Indeed

[soft matrix]

I don't think that's possible

[trim tangle]

yeah

it's not

because this is valid:

def foo(data: PlayerStatusData) -> None:
    print(data["duringBreak"])

in other words the class would violate LSP

[gaunt nebula]

Is there any tool which converts annotations like x | y to Union[x, y] or vice versa?

[soft matrix]

yes pyupgrade can go from union[x, y] -> x | y

[slender timber]

Union[x, y] can be upgraded to x | y but idk if it can be downgraded

I simply used regex replace to downgrade once

[rare scarab]

!e ```py
alias = int | str
print(alias)

[rough sluice]

@rare scarab :white_check_mark: Your 3.11 eval job has completed with return code 0.

int | str

[rare scarab]

hm...

I think there's a flake8 plugin for it

!pip flake8-new-union-types Not sure if it can be auto-fixed though

[rough sluice]

flake8-new-union-types v0.4.1

Flake8 plugin to enforce the new Union and Optional annotation syntax defined in PEP 604

[brazen jolt]

as gobot already suggested, pyupgrade can do that

you can then use the flake8 extension to avoid accidentally introducing old unions back into the codebase again

[trim tangle]

there's a flake8 extension for everything innit

[rare scarab]

if it has to do with source code enforcement, it can do it

[brazen jolt]

oh yeah, not to mention the all-in-one extensions like flake8-simplify or flake8-bugbear

[brisk hedge]

Should be fine, just target any unions inside annotation contexts

[runic tapir]

step one, done - mypy now allows @final on TypedDict on master.

now all that's left is figuring out how to type narrow a Union of final TypedDict's when "key" in or ["key"] or .get("key") are used 😅

[soft matrix]

havent you also got to narrow .items/.values?

[runic tapir]

for https://github.com/python/mypy/issues/7981 ? Don't think so

we're trying to narrow a union of non-overlapping final typeddicts to a specific typeddict based on which key is found

so, all the dict operations that get a value for a specific key, or check if a specific key exists

maybe not even d["key"], actually - it may actually just be "key" in d and d.get("key")

since you can't type narrow to "either it's this type or an exception is raised", and that's what d["key"] would do

[slender timber]

is there anyone here with an internal knowledge of _tkinter's typing?

[oblique urchin]

I probably reviewed most of it 🙂

[slender timber]

yea so

why is it so untyped yet

[oblique urchin]

there's a lot of it, it's hard to type because it's wrapping an untyped language

[slender timber]

like look at this

[oblique urchin]

_tkinter in particular is private

so feels low priority

[slender timber]

i mean its really a problem to even use tkinter comfortably with type checkers

[oblique urchin]

plus you are talking about it being untyped instead of submitting a PR to add types 🙂

[slender timber]

i would do a pr

but idk tkinter's internals

i try out the functions in IDLE to estimate their types

[slender timber]

if i import annotations from __future__, should I worry about using the newer GenericAlias syntax used by collections.abc types for Python versions older than 3.9?

For example, would this fail on Python <= 3.8 at runtime even though I imported annotations?
counts = collections.defaultdict[object, Iterator[int]](lambda: Count())

also why does pyright complain about this?

[oblique urchin]

from __future__ import annotations has no effect there because that's not an annotation

[slender timber]

then what is it

[oblique urchin]

a subscript

so that only works in 3.9+ where defaultdict defines __class_getitem__

[slender timber]

so there's no way to run this code in Python <= 3.8?

[oblique urchin]

no. you can write counts: defaultdict[object, Iterator[int]] = defaultdict(Count) instead

[slender timber]

:((

[sinful zinc]

in vs-code, is there an extension that will use type hints to prioritise compatible options and/or remove incompatible options from autocomplete suggestions?

[trim tangle]

Are you using Pylance?

could you explain a bit more about what you mean?

with an example perhaps

[sinful zinc]

yes, I have the ms python stuff installed.

say I have a property in some class,

class Root:

  @property
  def btype(self) -> SomeBaseClass:
    return self._btype

# and the setter

and some module model that has a bunch of classes including SomeBaseClass, derived classes from it, and unrelated classes.

When doing,

r = Root()
r.btype = model. # autocomplete options here

then the options are just everything from the model module, rather than things derived from SomeBaseClass

[trim tangle]

hmm I don't think that's possible

although that would be an interesting feature, I suppose

It does make sense to show everything, because you theoretically may want to do model.foo.bar + 42 and it will return SomeBaseClass

but prioritizing does sound like a good idea.

although... that does seem like an expensive tree search

Maybe you could open a discussion on https://github.com/microsoft/pylance-release ?

[slender timber]

@runtime_checkable
class IPlugin(Protocol):
    INTERNAL_NAME: ClassVar[str]

_PE_co = TypeVar("_PE_co", bound=AnyEvent, covariant=True)

class PluginBase(SingleEventModel, Generic[_PE_co]):
    def __init__(self, event: _PE_co, **kw: Any):
        super().__init__(event, **kw)

AnyPlugin = PluginBase[AnyEvent]
```I want `AnyPlugin` to bind to the addition of `IPlugin` and `PluginBase`, means `AnyPlugin` represents a class inheriting both `IPlugin` and `PluginBase`. I can't make `PluginBase` inherit from `IPlugin` without implemening it as well

Actually it is by design that a PluginBase subclass must implement IPlugin as well.

[slender timber]

Just notices typing_extensions dropped support for Python 3.6

https://github.com/python/typing_extensions/blob/main/CHANGELOG.md#release-420-april-17-2022
Should have made a major version change right?

[paper salmon]

!e @crystal wave
#discord-bots message

most argument annotations had a flaw off, arg: int = None, this is incorrect as arg's type can be an int or NoneType making it arg: int | None = None

afaik Optionals are implicit if None is the default as per the python docs https://docs.python.org/3.5/library/typing.html#typing.get_type_hints

import typing

def foo(x: int = None):
    int(x)  # None would result in TypeError

print(typing.get_type_hints(foo))``` although i dont think all linters/type checkers comply with this - i know pycharm's linter and mypy recognize the implied optional, though only mypy complains when the default isnt NoneType, and neither pycharm or pylint care about `int(x)` above)

[rough sluice]

@paper salmon :white_check_mark: Your 3.10 eval job has completed with return code 0.

{'x': typing.Optional[int]}

[paper salmon]

oh i just realized 3.11 changed it so it doesnt do that anymore

[brazen jolt]

pyright has a setting that you can toggle to respect this behavior

[soft matrix]

yeah cause its daft

[brazen jolt]

in fact, it was once the default behavior of pyright

I'm not sure about mypy, but it might have one too

[soft matrix]

mypy is changing it

but it has more of a stability policy than pyright or something

[brazen jolt]

yeah

[crystal wave]

Well in most pythons docs and peps they dont really specify it, but in others it may appear, either way i still feel like it should be specified or mentioned

[paper salmon]

hmm yeah it does seem a bit confusing

[crystal wave]

Interesting how linters dont care, they should, as you can avoid attribute and data errors and stuff along the lines

[crystal wave]

@paper salmon Most libs and stdlibs dont specify the None value as a default but maybe its due to the creation of the codebase and the version they were on as here https://docs.python.org/3/library/typing.html#typing.Optional it does show it

[cunning plover]

how to have typing for self of a abstract class, considering that its name is going to be changed

[trim tangle]

!d typing.Self

[rough sluice]

NEGATORY.

No documentation found for the requested symbol.

[trim tangle]

bruh

well... you use typing.Self or typing_extensions.Self

[cunning plover]

thanks. it exists

[cunning plover]

$ python3
Python 3.10.5 (main, Jun 11 2022, 16:53:24) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import typing_extensions
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'typing_extensions'

i have a feeling they don't wish to have Self for some reason xD

[trim tangle]

it's not standard, it's a backport

!pypi typing-extensions

[rough sluice]

typing-extensions v4.3.0

Backported and Experimental Type Hints for Python 3.7+

[sick notch]

what should i do to not have a warning??

warning is this

[soft matrix]

not use pycharm ;^)

[sick notch]

my bad

it's just what i did after the declaration that was wrong

oops

[sick notch]

isn't it the best IDE?

[soft matrix]

show a bit more of your code theres a small chance that pycharm isnt completely wrong

oh

[sick notch]

I mean I don't have a choiec haha i am obliged to use pycharm ^^ courtesy of my firm

[soft matrix]

its definitely not the best for typechecking

everything else is personal preference

[trim tangle]

sounds cursed tbh

what happens if you use vim or anything else?

[rare scarab]

mypy is the great equalizer

[sick notch]

vim?

idk what vim is haha

but only pycharm is installed on the computers at work

and we can't really install anything else without asking the IT department

[brazen jolt]

vim is a keyboard based editor focused on making you really fast

it gives you a very different experience from regular editors

you move around with hjkl instead of arrow keys, since it's closer to other things, and it works with different modes

[sick notch]

i mean i never move around with arrow keys haha

i just use my mouse

but i get the point

[brazen jolt]

in normal mode, you perform these movements, that's why using say j to go down doesn't type it, while in input mode it would actually type the letter

I'd recommend everyone to try it out for a while, since while it's probably not for most people, and it has a huge learning curve, you may really like it once you try it out, and it's worth experiencing something so different and unique

[rare scarab]

vscode can be installed as a user

[brazen jolt]

I think vscode can even be used from the browser with relative ease

I'm not sure how well would the terminal integration work though

[rare scarab]

benefit of vscode over pycharm: You don't need pro to do remote or container development

[soft matrix]

other benefit is that vsc has the best static typechecker

[rare scarab]

I'm sure there's a plugin to add pyright

[soft matrix]

there isnt in pycharm

[brazen jolt]

the python plugin does that actually

well, specifically pylance, but that's autoinstalled with python plugin

you just need to turn type checking on in settings

[rare scarab]

pyright is open source. pylance is not.

[brazen jolt]

I though pylance just used pyright internally along with some other stuff

or is it a fork?

[rare scarab]

pylance bundles pyright

[trim tangle]

not really it seems

pyright sometimes fetches updates from pylance, and vice versa?>.

it's confusing.

and pylance is closed source which is mildly cursed

[brazen jolt]

huh

[soft matrix]

i think people from the pylance team work on pyright inside of pylance and then they push the changes they made

[trim tangle]

anyway, I won't be working at microsoft any time soon

[brazen jolt]

glad I just use pyright alone as a language server in nvim without any of this closed sourced weirdness

[trim tangle]

pyright is so complicated inside that it might well be closed source 🤪

[brazen jolt]

you're not wrong

[rare scarab]

I'm sure the only reason the python extension is open source is because it's a fork.

[heady flicker]

Pycharm has 0 support for type checking. Mypy extension is nearly dead, lsp extension is dead. So no, if you want any typechecking -- mypy is a horrible choice.

[void panther]

while not that great, the builtin type checker is far from 0 support

[fierce ridge]

this doesn't make sense at all and mostly isn't true

[fierce ridge]

import logging
from collections.abc import Mapping, MutableMapping
from typing import Any, Generic, TypeVar


_A = TypeVar('_A')
_B = TypeVar('_B')
_L = TypeVar('_L', bound=logging.Logger | logging.LoggerAdapter[Any])


class SaferLoggerAdapter(logging.LoggerAdapter[_L], Generic[_L, _A, _B]):
    """Less-destructive version of ``LoggerAdapter``.

    The default :class:`logging.LoggerAdapter` implementation *replaces* the
    ``extra`` dict. This implementation instead *extends* it with new values.
    """
    extra: Mapping[str, Any]

    def process(self, msg: _A, kwargs: Mapping[str, Any]) -> tuple[_B, MutableMapping[str, Any]]:
        extra = kwargs.get('extra', {})
        extra = extra | self.extra
        return msg, extra

is there any way i can convince a type checker that _A == _B is acceptable?

e.g. https://pyright-playground.decorator-factory.su/?gzip=H4sIANwNHGMC_21T247aQAx9z1dY9IFLQz4AiUrQVlUl2j4s6ssKETMxYbqTmXQu0Kj03-tJCAsrkNCMHfvYPscjq9pYD8qUpdRlIjvzlzO6v7vGJXtrKhBGKRJeGu0y3Am4fP-ISuFOUQpfPdnu9g3rmuH4Enz0XO0n-h1IC-oQfRO9PdBCNyl8IU1WihRWMqKpFNZNTQsl0XXXn2iTJNkuYN6bo-F2MRwn2-WdaxldqzvXapjCzgRdzC_jZis-ycIZ7h2LAmuu_swdbcZcTSh0Dp5wT_YuYPQ4bbvaXAdhI4Xtgv_LzXiWAP8Gg8GKnJsW5LwNTOiR4EjWMbFg9pDnd2h5niVt2vpAUNAeg_Iwazua5Q_r55FQRRVpj1EtmFiqFQpyE_AHasHynP54i3kOhRQ-Y3Dp3qZJ7TxhARMOJV1wtvRwkv4Amk5wRBXIZf1E7dliznr1n3m8NKq66Qbg5qG2hvtwI0dqn0LlyllLzssJbekeZI5h-gF84L6et8u36_QatumYvfbAsneQWUl-NGx9LP7ff-PkGvgOvv9Yf57BJ6OHHoIjyM_zHA5keYF3JDC6eGIUPvCKN1BxcU8uctiylt5A9VyxfsIS88drjboAVkmWurX4_dRNz7RjWxs9NbujNMHF2Bu0oPf8IILmcjAojRcHHHCJtrLzHIu2ACV3Fi0_GdTIexBRvLkK3AFVxtJUmKoyWjXTF21Omkc7IBe17a6954lZaiWdv2h5y2F3niGqlbXGK32WfLA6SpjC5dN_-EYh30wEAAA%3D

wishful thinking?

import logging
from collections.abc import Mapping, MutableMapping
from typing import Any, Generic, TypeVar

_A = TypeVar('_A')
_B = TypeVar('_B')
_C = TypeVar('_C')
_D = TypeVar('_D')
_L = TypeVar('_L', bound=logging.Logger | logging.LoggerAdapter[Any])

class SaferLoggerAdapter(logging.LoggerAdapter[_L], Generic[_L, _A, _B, _C, _D]):
    """Less-destructive version of ``LoggerAdapter``.

    The default :class:`logging.LoggerAdapter` implementation *replaces* the
    ``extra`` dict. This implementation instead *extends* it with new values.
    """
    extra: Mapping[str, _C]

    def process(self, msg: _A, kwargs: Mapping[str, _D]) -> tuple[_B, MutableMapping[str, _C | _D]]:
        extra = kwargs.get('extra', {})
        extra = extra | self.extra
        return msg, extra

[hallow flint]

no, because we set python_requires, so nothing actually breaks. (thankfully so. if typing_extensions ever does do a major version change it's going to cause packaging nightmares for years because of upper bounds)

[hallow flint]

implicit optional is the current default with mypy, but --strict implies explicit optional. i plan on making explicit optional the default behaviour in the next release. fwiw PEP 484 changed in 2018. see https://peps.python.org/pep-0484/#union-types since you were curious for a reference

[heady flicker]

Haven't seen a warning from it even once. I guess it would give out warnings but not the "inference warnings". On a project with 500+ type errors in pyright, pycharm showed only ~30, all of which were more stylistic than type checking

[heady flicker]

If you think I'm wrong, then go ahead, take look. Make pyright work properly on pycharm, showing error highlights and stuff. But you won't be able to.

[slender timber]

I think it depends on what type of analysis mode you chose, basic or strict

[little hare]

okay so say i have a parameter which returns whatever is passed to it

this parameter is hinted with Union[ActionRow[ModalUIComponent], ActionRow[WrappedComponent]]

so yes, the function can return that, obviously, but i want to hint it as returning whatever was passed

i could use overloads for this

but a typevar or smth somehow with that would make the most sense

[heady flicker]

I have enabled every inspection type and pycharm still fails to show the most basic errors.

[slender timber]

Have you configured any pyright setting in your pyproject.toml?

[heady flicker]

You're talking about pyright errors. But pycharm is not even capable of showing pyright errors

[slender timber]

You do know that you can supress errors via pyproject.toml right?

[little hare]

welp that isn't cursed at all

[heady flicker]

Yes. Are you sure you understand what I'm talking about?

I am asserting that pycharm shows no useful typing errors.
You ask me about pyright configurations. But pycharm DOESN'T and CAN'T use pyright.

[slender timber]

Somewhere they might be suppressed

Because I do remember using pycharm once

[fierce ridge]

i used to use pycharm for a lot of things and yes, it does definitely catch and emit type errors

[slender timber]

And it reported a whole lot more errors than vscode ever did

Seriously tho, I don't know why pylance comes with pyright disabled

[little hare]

anyone know of a less cursed way to do this?

[slender timber]

It encourages poor coding practices

[slender timber]

Use generics

[little hare]

tldr the function returns self and i want to typehint as such but the class is also generic and we're already abusing that

[heady flicker]

Again, It does not support LSPs.

[little hare]

its already a generic https://github.com/disnakedev/disnake/blob/master/disnake/ui/action_row.py

[slender timber]

I don't have the slightest idea why you would type hint self argument

[heady flicker]

As I said in the other channel, either prove it or stop spreading misinformation, please.

[little hare]

function isn't valid for a specifically subscripted version of this class

[heady flicker]

You're talking about an entirely different thing.

[little hare]

(like, it'll work regardless of the typings, but the place where this class is used, doesn't let you use that function)

[fierce ridge]

that seems like a design smell to me

[slender timber]

Hey that's not possible. I am quite sure, pycharm did emit errors once I used it and they were quite strict ones like mypy level strict. I don't where the errors came from but they did

[little hare]

blame the discord api 🤡

no, seriously

[heady flicker]

Show me :)

[little hare]

action rows are the containers for all components

but modals only support text inputs, and messages only support buttons and selects, but eventually modals will support buttons and selects too so making multiple classes wouldn't have been a good idea

[heady flicker]

Everybody talks about how it once worked. The coolest thing is: they once worked for me too! But that was years ago. Now nothing useful comes out -- not on professional, not on community, not on my computer, not on my colleagues' computers.

[slender timber]

I deleted pycharm because they frustrated me with errors. Anyways imo vscode is better and incredibly fast, give it a shot?

[little hare]

we don't care if a user attempts to put a text input on a message, but we want to typehint that its not possible

[heady flicker]

I'm using vscode

[little hare]

i'm just trying to typehint these methods to return self

[slender timber]

But an IDE of that popularity dropping LSP support really makes no sense. Its like shooting themselves in the foot

[little hare]

but can't use Self because we changed what self is typehinted as, so I'm basically asking for the best way to typehint this

rn i have a typevar for each type of component

[heady flicker]

Exactly. That's why I say: it has 0 typehint support. It can show you that you lack typehints but that's pretty much it. That's not even in the same realm with pyre, mypy, or pyright.

[slender timber]

You don't need a different typevar unless it has different bounds or covariant etc. You can use the same typevar its meaning changes according to the scope its used in

[fierce ridge]

please can we move this debate to #editors-ides

[heady flicker]

No problem.

[little hare]

lemme commit the code i have

[fierce ridge]

other than overloading literally every method idk if it's possible to restrict a type to some subclasses but not others

[little hare]

https://github.com/DisnakeDev/disnake/compare/master...feat%2Factionrow-method-chaining

[little hare]

yeah :/

we use the generic typings to restrict the methods currently but we don't mind if they're used

we also have this at the end

[tranquil turtle]

TIL, thank you 🤯

[minor nimbus]

Is there a way in the current typing to solve this:

def process_value(value: int) -> None:
    ...

class Test:
    def __init__(self) -> None:
        self.var: int | None = None

    def ensure_var(self) -> None:
        if self.var is None:
            # do some processing here
            self.var = 5
        # note that self.var will definitely end up with 'int' as the type here

    def use_var(self) -> None:
        ensure_var()
        assert self.var is not None  # I want to get rid of this, or move it to ensure_var
        process_value(self.var)  # without the assert: int expected, got Optional[int]
        return self.var

I've looked into typing.TypeGuard, but it's quite clunky to use, requiring passing the self.var as an argument, which makes no sense in a class context. Not sure if it'd work here even.

wish there would be like a typing.Assert which I could then use as typing.Assert[self.var is not None] return type from ensure_var or smth like that

cos otherwise in every place where ensure_var() is called, I need the assert right after

[trim tangle]

Generally not. But depending on your situations, you might want to refactor the code if you have lots of these optional values

[minor nimbus]

Big OOF then

[soft matrix]

typing.TypeGuard on Self would be cool

[minor nimbus]

I do have some, however this example comes from an async code, where I kinda need to init the attributes with something, before the async code elsewhere sets proper values on them

[trim tangle]

You could make a helper method like ```py
def _cool_var(self) -> int:
if self.var is None:
raise AssertionError
return self.vae

[trim tangle]

Could you give a more realistic example then?

[minor nimbus]

I mean, the code I have is pretty much just like the example I provided, only async. The attributes start as ... | None type, with None assigned, and then later have the proper value assigned.

heck, if you really need an example, I can show you the session one

although I got around it quite nicely here

note there's nothing async inside here, I use this to get around the fact that aiohttp.ClientSession has to be initialized in an async context, since passing in loop is deprecated.

[trim tangle]

Generally I follow a pattern like this ```py
class FooClient:
def init(
self,
thing: Thing,
session: ClientSession,
other_conn: OtherConnection,
) -> None:
self._session = session
self._other_conn = other_conn
self._thing = thing

@classmethod
@contextlib.asynccontextmanager
async def connect(cls, thing: Thing, db_config: DbConfig) -> AsyncIterator[Self]:
    async with ClientSession(timeout=5) as cs:
        async with db_connect(db_config) as db:
            yield cls(thing, cs, db)

``` (classmethod could be replaced with a free function, or just some piece of glue code in async def main(), depending on the context`)

Also, how do you plan to ensure that the session is closed properly?

idiomatically that's done with context managers anyway

[minor nimbus]

the session lives through the entire lifetime of the application

and I have a shutdown async method that calls close on it

[trim tangle]

Well, if you have 10 resources it get kinda messy. And not all your resources will necessarily live throughout the whole lifetime.

[minor nimbus]

there's a couple of places I have these "ensure" functions in

session has to be ensured before a request is made

then I have an access token exchange, that also has to be ensured before any request that requires authorization is made

and so on

Hmm, but this gives me an idea

[trim tangle]

hm yeah if you have some workflow it gets tricky, kinda

[minor nimbus]

I managed to get around the issue here

by returning the attribute

I haven't even noticed that til now

Guess I could do it like this in other ensure functions

just return the attribute with the correct type

and use it as a local var

[trim tangle]

Not sure if it's helpful, but I have a setup like this:

class _RefreshableFooToken:
    def __init__(self, session: ClientSession, config: FooAccountConfig) -> None:
        ...
        self._token = _make_expired_token()

    async def fetch(self) -> _FooToken:
        now = datetime.now(timezone.utc)
        if self._token.expires_at <= now - timedelta(seconds=10):
            await self._refresh()
        return self._token

    async def _refresh(self) -> None:
        self._token = await _request_token(...)
        
        
 class Foo:
    def __init__(self, ...) -> None:
        self._token = _RefreshableFooToken(session, config, ...)
        ...

    @staticmethod
    @contextlib.asynccontextmanager
    async def connect(config: FooThingAccountConfig) -> AsyncIterator["Foo"]:
        async with ClientSession() as cs:
            yield Foo(cs, config, URL("https://www.example.com"))

    async def create_widget(self, duck_id: str, widget: Widget) -> None:
        token = await self._token.fetch()
        url = self._root_url / "hmm" / duck_id / "widgets"

        ...

[minor nimbus]

yeah, you have the same solution

[trim tangle]

that way the refreshable token object keeps track of when to do an actual request

You can also avoid the None by using the "null object" pattern. In my case I just have a hack like this

self._token = _GoogleDriveToken("", datetime.fromtimestamp(0, timezone.utc))
``` 🤪 
maybe I'll change it later

[minor nimbus]

def process_value(value: int) -> None:
    ...

class Test:
    def __init__(self) -> None:
        self.var: int | None = None

    def ensure_var(self) -> int:
        if self.var is None:
            # do some processing here
            self.var = 5
        return self.var  # return the var with the correct type

    def use_var(self) -> None:
        int_var = ensure_var()
        process_value(int_var)  # no error

it's this one

Alright, well, I've got it now

Cheers for the help

[slender timber]

!d typing.TypeGuard

[rough sluice]

typing.TypeGuard

typing.TypeGuard```
Special typing form used to annotate the return type of a user-defined type guard function. `TypeGuard` only accepts a single type argument. At runtime, functions marked this way should return a boolean.

`TypeGuard` aims to benefit *type narrowing* – a technique used by static type checkers to determine a more precise type of an expression within a program’s code flow. Usually type narrowing is done by analyzing conditional code flow and applying the narrowing to a block of code. The conditional expression here is sometimes referred to as a “type guard”:

[slender timber]

Thank you Python docs

I understood perfectly none of it 🙂

What would ideally be the use case of such a thing

[soft matrix]

def is_str_list(val: List[object]) -> TypeGuard[List[str]]:
    '''Determines whether all objects in the list are strings'''
    return all(isinstance(x, str) for x in val)

def func1(val: List[object]):
    if is_str_list(val):
        # Type of ``val`` is narrowed to ``List[str]``.
        print(" ".join(val))
    else:
        # Type of ``val`` remains as ``List[object]``.
        print("Not a list of strings!")```i think thats a good usecase

[slender timber]

So the function acts like a type narrower

Hmm

[brazen jolt]

yeah but it's not really helpful in this case

[slender timber]

Its actual return type is ommited tho

[brazen jolt]

the thing is you'd need to do something like if ensure_var(self.var), with ensure_var returning true

[brazen jolt]

tackling something like this is actually really annoying when dealing with type hinting, and it's best if you can somehow avoid this logic entirely

[slender timber]

This is one of the more abstract typing features imo

[brazen jolt]

the way many people do it is to just define the type as int with type ignore, like: ```py
def init(self):
self.var: int = None # type: ignore

[slender timber]

Or you can choose to not give it a value at all

Just annotate it with a type

[minor nimbus]

Yeah, I kinda wanted to avoid that, especially since there are cases where I have to reset the value back to it being None

not all cases, but some are like that

[slender timber]

And check it in locals, if it has a value, it will be present

[minor nimbus]

that's not a biggie though, as long as I can work around it with a function like that

it functions sort of like an async property, but with a function call associated to it

[slender timber]

There is no compulsion to define variables in the ctor only

[brazen jolt]

well, I'd generally say just use property directly

async does pose issues here though

in that case, you'll probably just want to make setter and getter functions

[slender timber]

This TypeGuard function would make a great use for an isnone or isnotnone function

[brazen jolt]

why not just use if x is None or if x is not None though?

[slender timber]

Yea I was just giving an example

[brazen jolt]

the example by gobot is actually a really nice one already

[slender timber]

But a custom function can redefine the meaning of a variable being None

[brazen jolt]

because it's not so easy to narrow the type from list[object] to list[str] for example

[slender timber]

Be a chad, just typing.cast it

[minor nimbus]

well, my idea was something along the lines of a typing.Assert that'd accept a single statement

that you could then use as a return type

[brazen jolt]

that's almost certainly not gonna happen

[minor nimbus]

self.var: int | None = None

def ensure_var(self) -> typing.Assert[self.var is not None]:
    if self.var is None:
        ...
        self.var = ...

I know, but it'd be cool to have here lol

there is technically a way to do this already

with typing.TypeGuard

but self.var would need to be passed as the first argument

[brazen jolt]

and you'd need to be in an if block

[minor nimbus]

which makes this really meh

[slender timber]

Huh

[minor nimbus]

That's not a thing

[slender timber]

Is this new

Oh

[minor nimbus]

no, it's not a thing at all

it's something I came up with as an idea

[brazen jolt]

yeah, they were just explaining that it'd be cool to have it

[slender timber]

Yea you mean like C++ static_assert

I wonder why is there no Python alternative yet

[minor nimbus]

it just makes it clunky to use the function, and then follow it with an assert statement

[brazen jolt]

I'd recommend following with a cast rather than an assert

[minor nimbus]

TypeGuard sounds like the solution, but it has a restriction of applying only to the first argument passed

[slender timber]

Well the reall thing that is needed are nullable operator syntaxes

[minor nimbus]

would it be possible to use a TypeGuard even

[brazen jolt]

yes but as I shown you'd need to be in an if block

[minor nimbus]

or does it really need to be used in an if statement

[brazen jolt]

the thing is typeguard is just for type narrowing

just like ```py
x: int | None = None

if x is None:
reveal_type(x) # x inferred as None here
else:
reveal_type(x) # x inferred as int here

reveal_type(x) # x inferred as int | None here

[trim tangle]

wasn't there some PEP for assertion functions?

[brazen jolt]

you could do ```py
if not my_type_guard(x):
raise Exception("This will never be reached")

here, x is narrowed

[trim tangle]

like a function that only returns if some type condition is met

[slender timber]

Hang on

[trim tangle]

that does exist in TypeScript, maybe I'm just misremembering

[slender timber]

There is assert_type

[brazen jolt]

is there?

[slender timber]

3.1

3.11

Check it in typing_exstension

[trim tangle]

That's a different thing

[slender timber]

There is assert_never as well

[slender timber]

And the Never Type

[minor nimbus]

Yeah, it seems to work only in an if statement:

eh

[brazen jolt]

obviously, again, this is because typeguard returns a bool actually

it's a function that's supposed to verify whether it's safe to narrow and what to narrow to

[minor nimbus]

oh

right

[brazen jolt]

that's why I showed this example ^^

[minor nimbus]

never really found a use for a TypeGuard in my code yet, so

[brazen jolt]

I've used it like twice

and I type-hint in pretty much all of my projects

[slender timber]

TIL TypeGuard

[brazen jolt]

typeguard is neat, but it doesn't solve this specific issue

and yeah, it really would be cool to have a better way to solve this

as it is a pretty common issue

[slender timber]

TypeGuard is for use with invariant generics ig

[minor nimbus]

I'm used to just using asserts when this happens

[slender timber]

Like List[T]

[minor nimbus]

with full knowledge that I'll never actually get an assertion error from them

and if I would, well

[brazen jolt]

!pep 647

[rough sluice]

**PEP 647 - User-Defined Type Guards**

Link

Status

Accepted

Python-Version

3.10

Created

07-Oct-2020

Type

Standards Track

[minor nimbus]

something would go terribly wrong

[brazen jolt]

the thing about asserts is, you don't always get an error

it actually depends on how the program is ran

assertion errors can be ignored with a flag to python interpreter

I generally don't like using asserts in production code

I prefer if x: raise

[slender timber]

Btw if I yield an element from my iter dunder method, what should be the return type? Iterator or Generator?

By default pyright assumes Generator

But generator got like 3 typevars

[minor nimbus]

I'm aware, but as I said, I'm putting them in just to get around the typing issues, not as a sanity check they're usually meant to be for

[brazen jolt]

Iterator is a subclass of generator