#type-hinting

1 messages · Page 63 of 1

trim tangle
#

ah

#

🥲

#

or tuple[str, int]

soft matrix
mortal fractal
#

sry @soft matrix I think I made that merge conflict for you in Self with my ClassVar change >.<

#

luckily it was trivial

soft matrix
#

its fine

mortal fractal
#

Is main blocked for a alpha release? I'm not familiar with dev cycle

soft matrix
#

its blocked cause buildbots are failing iirc

mortal fractal
#

o

soft matrix
mortal fractal
#

Ah I found it it's in python-dev

#

Thanks

#

Oh nice there's a PR clarifying GenericAlias and some other things

supple lily
#

i just spent like 12 hours trying to setup type hints + vscode + mypy, and its still not working totally. this is exhausting.
i feel like the juice was not worth the squeeze at all. i could've worked on features instead 😭

trim tangle
#

Pylance (which uses pyright) is a bit different from mypy, but IMO it is better polished, and is more intuitive. It is also used to provide types when you hover over something.

If for some reason you need mypy specifically, we can move to #editors-ides 🙂

supple lily
#

don't need mypy specifically, it just seemed like the most recommended after some quick googling. will take a break and try out pylance, thx

trim tangle
mortal fractal
#

whoa black changes failed my CI

leaden oak
#

we changed a lot this release haha 😄

oblique urchin
#

that's why you should pin it 🙂

mortal fractal
#

I prefer the failing to just annoy me into fixing

oblique urchin
#

if it's due to spaces around ** you can blame @leaden oak by the way

mortal fractal
mortal fractal
leaden oak
#

*runs away*

#

... hopefully y'all like it lemon_sweat

mortal fractal
#

I primarily use black so I don't have to have an opinion on formatting

leaden oak
#

I don't know if I should be proud or afraid 🤔

upbeat wadi
#

is it recommended for __slots__ to be annotated (as Tuple[str, ...])?

upbeat wadi
#

also, py class A: x: int should x: int be in an if TYPE_CHECKING: block?

hearty shell
#

I wouldn't bother, because you wont get much from it, __slots__ only defines the names to reserve, not the types

oblique urchin
oblique urchin
upbeat wadi
#

alright, thanks

upbeat wadi
# oblique urchin no

would that still be the same if there are a lot of annotations in the class meant for type checking? (performance wise)

oblique urchin
upbeat wadi
#

I don't think I need worry about it then, thanks again

tranquil turtle
# 
class A:
  __slots__ = {
    'a': int,
    'b': str,
    'c': dict[int, list[float]],
  }
hearty shell
#

I dont think that works

#

!e

class A:
  __slots__ = {
    'a': int,
    'b': str,
    'c': dict[int, list[float]],
  }

print(A.__annotations__)
print(A.a.__annotations__)
rough sluiceBOT
#

@hearty shell :x: Your eval job has completed with return code 1.

001 | {}
002 | Traceback (most recent call last):
003 |   File "<string>", line 9, in <module>
004 | AttributeError: 'member_descriptor' object has no attribute '__annotations__'
rustic gull
# 
from heapq import nlargest
from typing import *

def topKFrequent(self, nums: List[int], k: int) -> List[int]:
    count = Counter(nums)
    return nlargest(k, count.keys(), key=count.get)

# main.py:1600: error: Argument "key" to "nlargest" has incompatible type
# overloaded function; expected "Optional[Callable[[int], SupportsLessThan]]"
#             return nlargest(k, count.keys(), key=count.get)
#                                                  ^

count.get takes an int and returns an int, wouldn't that support the less than operator, what's wrong?

oblique urchin
hearty shell
oblique urchin
rustic gull
#

righttt, key=count.__getitem__ makes more sense

acoustic thicket
#

count.__getitem__

#

yea

bleak wind
#

Hey! How can I do something like this

def func(item) -> item.value:
    ...
trim tangle
bleak wind
#

an object which has the an instance attribute value

trim tangle
#

any object whatsoever?

#

not a specific class, like Item?

bleak wind
#

Yes, it's an instance of Item

trim tangle
#

And what type is the value of the item?

bleak wind
#

That depends on what the argument, Item was created with, for example Item(value=0)
here, value attribute will be int

boreal ingot
#

are you familiar with typing.Generic and typing.TypeVar ?

bleak wind
#

only TypeVar

boreal ingot
#

okay so you know how a TypeVar scope is specific to a function?
a Generic allows you to extend that scope to a hole class ```python
from typing import Generic, TypeVar

T = TypeVar("T")

class Item(Generic[T]):
def init(self, value: T) -> None:
self.value = value

def func(item: Item[T]) -> T:
return item.value```

bleak wind
#

Ooo I see

#

Thank you!

boreal ingot
#

np 😄

ruby ember
#

I don't understand what this message is saying:

Incompatible types in assignment (expression has type "Dict[str, Dict[Union[float, int], str]]", target has type "Dict[Union[float, int], str]")  [assignment]mypy

I have this from something such as :

_dict[x] = computed_dict

where _dict[x] = Dict[str, Dict[Union[float, int], str]]

and computed_dict is similar to {'something' : {2 : 'a', 4 : 'b'}}.

It doesn't make sense to me as the _dict has a union for the key to the subdict, so why does it care?

boreal ingot
ruby ember
# 
 Dict[str, Dict[Union[float, int], str]] -> {'something' : { 1 : 'a' }}
boreal ingot
#

but _dict[x] = ... means you are trying to overwrite the subdict

# 
_dict = computed_dict
``` should work
ruby ember
#

oh sorry - what i'm doing is:

_dict['something'] = {2 : 'a'}
#

got those mixed up, but that's what's happening - and i have that error

boreal ingot
#

in that case, hmmm

ruby ember
#

i tried casting and moving on with my life but that didn't work either

boreal ingot
#

this works when I test it ```py
from typing import *

_dict: Dict[str, Dict[Union[float, int], str]] = {}
_dict['something'] = {2 : 'a'}``` are you doing anything differently?

ruby ember
#

going to try and create a mwe - just tried in playground and it didnt' seem to work out 🤔

#

ok i can get it to work by casting to Dict[Union[float, int], str]

#

might just live with that as i don't really understand why it's happening lol

#

@boreal ingot actually i guess this represents it (https://mypy-play.net/?mypy=latest&python=3.10&gist=0d054b6a0c5b7108585c2cd35ae7f4bb):


from typing import Dict, Union 

TypeAlias = Dict[str, Dict[Union[int, float] , str]]

x : TypeAlias = {}
y : Dict[int, str] = {2 : 'this'}

x['something'] = y

So here I'm getting the previous error, but I've explicitly stated x to be more permissive type-wise than y is, and am assigning y to x, so intuitively (for me) i'd have thought the type x would take precedence here

mypy Playground

The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.

boreal ingot
# ruby ember <@!366331361583169537> actually i guess this represents it (https://mypy-play.ne...

!e oh that makes sense, and it has to do with the fact that dicts are mutable, take this example ```py
from typing import Dict, Union
TypeAlias = Dict[str, Dict[Union[int, float] , str]]
x : TypeAlias = {}
y : Dict[int, str] = {2 : 'this'}
x['something'] = y

all fine so far, but if we do

x["something"][2.3] = 4

print("x:", x) # still fine

BUT look at this

print("y:", y) # now y is no longer Dict[int, str], OH NO!```

rough sluiceBOT
#

@boreal ingot :white_check_mark: Your eval job has completed with return code 0.

001 | x: {'something': {2: 'this', 2.3: 4}}
002 | y: {2: 'this', 2.3: 4}
leaden fog
#

I really like generic aliases

ruby ember
boreal ingot
ruby ember
#

just to make it conform to x, makes sense

boreal ingot
ruby ember
#

any reason that's preferable to cast( )?
right - makes sense 🙂

boreal ingot
ruby ember
#

cool - just wasn't sure if it was more idiomatic or more personal, all good tho

dusky pond
boreal ingot
rough sluiceBOT
#

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

dusky pond
terse sky
#

since they're read only

#

so if the original code, you want to be able to pass something as an argument to a function and it's not working when you expect, annotating the function with Mapping instead of Dict might help

oblique urchin
terse sky
#

ah yeah, if it's the key type then no luck

#

my mistake

#

you want to be really careful with float as a key to a Dict though, anyhow

cold osprey
#

hello, anyone can clarify what's up with typing.Generator being "deprecated" since 3.9? Not sure i understand what should be used as a return type of my yielding functions instead of Generator, as mypy still reports error when i try to replace it with List[yieldType]

oblique urchin
#

nothing else changes

cold osprey
#

ooh, right, i was fixated on typing module and didnt realize other parts of "std" can be typed 🙃

void panther
cold osprey
#

yes, simplifies it a bit, i just didnt understand deprecation notice, thus collections.abc.Iterator is my thing...

#

thanks Jell & Numerlor

hidden abyss
#

Hi all, often you have some m: Mapping[KT, VT] and you need to call m.get(obj[, default]) with some obj that may not be of type KT. This is the whole reason you're using m.get(obj) and not m[obj] in the first place -- the very next thing you're going to do is to check if you got back None (or whatever default value you passed). But currently, mypy gives a type error for this: error: No overload variant of "get" of "Mapping" matches argument type "...". It looks like this is because Mapping.get(..) currently only accepts __key: KT .) What is the right way to handle this? It seems like typeshed's type annotations here are thwarting this very common idiom. The same goes for MutableMapping.pop(obj, default) too.

rough sluiceBOT
#

stdlib/typing.pyi line 456

def get(self, __key: _KT, __default: _VT_co | _T) -> _VT_co | _T: ...```
rustic gull
#

Hello guys, I've been once again distracted by python and instead of actually working on my project i decided to implement a useless rust feature for more than 3 hours but now I have two major issues which keep me a way from using/running my code how it is expected to be.

So, Result should have the Ok and Err params type annotated to be Ok/Err while also allowing type annotation so when Ok gets created with the value of a string then on doing ok.value it gets shown as a string but when I pass it into the Result with res = Result(Ok(True, "my value"), Err(False, None)) i should see that res.Ok.value is a string but i dont.

My code:

@dataclass()
class Ok(Generic[_TOK]):
    status: bool = field(default=False) # on true this got returned
    value: _TOK = field(default=None)

@dataclass()
class Err(Generic[_TERR]):
    status: bool = field(default=False) # on true this got returned
    value: _TERR = field(default=None)

_T = TypeVar("_T")
_TOK = TypeVar("_TOK", Ok, covariant=True)
_TERR = TypeVar("_TERR", bound="Err")

@dataclass()
class Result(Generic[_T, _TOK, _TERR]):
    Ok: _TOK = field(default=Ok(False, None))
    Err: _TERR = field(default=Err(False, None))

    ...

I know that this is a very advanced python problem but please give it a try. Thank you!

sturdy plinth
#

Hello, I hope you are well, can you help me with this, can you do me a favor, please

in this
This is why we must choose to make the best possible code. Therefore:

Research good programming practices in Python.

Create a summary document, where these good practices are consolidated.

Create a software program that can analyze the Python code of other programs and identify if best practices are being followed.

A summary should be identified and presented at the end indicating which good practices were identified and which were not.

It should give a suggestion on how to apply the good practice.

I already have the document of good practice and bad practice, but I don't know how to do it so that I can identify the practice

I already know how to make him read it line by line

still monolith
rough sluiceBOT
#
**PEP 8 - Style Guide for Python Code**

Link

Status

Active

Created

05-Jul-2001

Type

Process

still monolith
#

Some 3rd-party compilers (and linters) can already identify that

hidden abyss
oblique urchin
rustic gull
hidden abyss
oblique urchin
#

In an ideal world we'd have some check that the key type overlaps, so you could use a partially overlapping type, but still get an error if you do d.get(1) on a dict with str keys

hidden abyss
#

@oblique urchin, I think that there are a lot of legitimate uses of d.get(obj) where you have no idea what type obj is, and the code is still correctly handling the case where obj is missing. Do you think that all uses of d.get(obj) should have to use some kind of look-before-you-leap type narrowing to avoid a mypy error, even though it's common and often more idiomatic to check the result after the call? (I'd agree with this for d[obj] , but not d.get(obj) which should explicitly support the case where you don't know obj's type, and same goes for d.pop(obj, None) )

sturdy plinth
oblique urchin
hidden abyss
# oblique urchin it's not common in code I have seen 🙂 But yes, if that's common, that's a good ...

Interesting, I've seen this all the time (and can give examples if you like), but good to know you're open to reconsidering! (Put another way, I agree it can help catch type errors, but it can also flag perfectly idiomatic code that is already handling all cases correctly, which becomes less clear when rewritten to appease mypy. Would definitely be a victory for typing in Python when it can avoid causing cases like this.)

ruby ember
#

They're floats atm as that's what was returned by the file format, there's no reason for then to be floats really tho

oblique urchin
ruby ember
oblique urchin
ruby ember
acoustic thicket
#

You can run into precision issues with something as simple as 0.1+0.2

terse sky
#

pretty much

#

I forget the exact examples but you can have thinks like, hash at 1.0, and then try to retrieve at 0.7 + 0.3 and it fails

oblique urchin
#

but yeah, it's not reliable in general

acoustic thicket
#

Are there any common workarounds for hashing floats? Rounding to some decimal places comes to mind but that still has sharp edges like 0.00049999 and 0.0005

terse sky
#

simplest workaround which is obviously a pain in python is to use a BST instead

#

and then if you wanted you could use approximate equality, or even access all keys within epsilon of your access, check if there's only one, etc

#

but it really depends what you're doing

oblique urchin
acoustic thicket
#

Dont have any specific use case in mind heh

terse sky
#

you could imagine a situation where you wanted to form a "set" of measurements, to count how many of something you thought there was. Suppose you get readings from multiple sources, and they are floats, and two "identical" readings mean that it's the same object and shouldn't be double counted

#

but in a situation like that, you'd obviously want to have some kind of tolerance

#

so a more practical thing to do is to sort it and then do some kind of group_by operation

#

or if this group was being modified continually, a BST

#

but either way you need an appraoch that is robust to using tolerances, which sorting and BST's are, but hash structures are not

acoustic thicket
terse sky
#

The problem is that rounding runs into the edge cases discussed above

#

But it that's good enough then sure

blazing nest
#

You could just multiply it by 10**X to get X decimals and then turn the float into an integer.

mortal fractal
#

almost, only a couple more failing buildbots?

mortal fractal
#

So part of the discussion on ClassVar/Final in dataclasses was basically that the PEP should be amended to allow both of these in one annotation in dataclasses specifically. It's easy to implement this at runtime by adding the allow_special_forms flag to ClassVar and Final, but at the cost of allowing this everywhere at runtime and also allowing stuff like ClassVar[ClassVar[int]] (at runtime only)

clever bridge
#

If I have the following type alias
T = TypeVar("T")
Missing = Union[_MISSING, T]

Why does the type of a variable annotated with Missing[Type[Enum]] end up being _MISSING | Enum?

oblique urchin
clever bridge
#

Yes

#

pyright/pylance/VScode

oblique urchin
#

I think that's a bug. I noticed pyright has some occasional odd behavior around type[] in type aliases

clever bridge
#

Ack

#

The really odd thing is that hovering over the variable has VSCode show the right type, then when I use reveal_type or pyright shows type checking errors it's _MISSING | Enum

# 
Cannot assign member "choices" for type "Option"
  Expression of type "Type[Enum]" cannot be assigned to member "choices" of class "Option"
    Type "Type[Enum]" cannot be assigned to type "Missing[Type[Enum]]"
      "Type[EnumMeta]" is incompatible with "Type[_MISSING]"
      "Type[EnumMeta]" is incompatible with "Type[Enum]"PylancereportGeneralTypeIssues

if TYPE_CHECKING:
    self.choices = cast(Type[Enum], self.choices)

Can't even cast it dead

hearty shell
#

Yeah it is a bug with pyright, the same works on mypy

clever bridge
#

The casting works when choices is Missing[EnumMeta]

hearty shell
#

Does the same issue happen if you just use the EnumMeta directly?

clever bridge
#

JINX

hearty shell
#

I mean without casting

#

x)

clever bridge
# 
Could not bind method "__iter__" because "EnumMeta" is not assignable to parameter "self"
  Type "EnumMeta" cannot be assigned to type "Type[_T@__iter__]"PylancereportGeneralTypeIssues
#

It complains about a different error

hearty shell
#

Humm

# 
from typing import TypeVar, Union, NewType, Type
from enum import EnumMeta

_MISSING = NewType("_MISSING", object)

T = TypeVar("T")
Missing = Union[_MISSING, T]

def factory() -> EnumMeta: ...

a: Missing[EnumMeta] = factory()

reveal_type(a)
#

here it does reveal as being EnumMeta

clever bridge
#

_MISSING is an Enum is part of the problem

acoustic thicket
clever bridge
#

That's what PEP 484 recommends for singletons after I spent two hours trying my own metaclasses and things

hearty shell
#

so is _MISSING an Enum or enum instance

clever bridge
#

_MISSING is the enum, MISSING is an enum instance

# 
T = TypeVar("T")


class _MISSING(Enum):
    MISSING = "MISSING"

    def __bool__(self) -> Literal[False]:
        return False

    def __eq__(self, other: Any) -> Literal[False]:
        return False

    def __repr__(self) -> str:
        return "MISSING"


MISSING = _MISSING.MISSING
Missing = Union[_MISSING, T]
oblique urchin
clever bridge
#

It's purpose is for when a value is missing it's MISSING

#

The __eq__ is just there, I'm debating whether to keep it or not since the goal is for it to be used in an is check

hearty shell
#

I never used one but dont people use sentinels in those instances?

clever bridge
#

That is exactly what I'm trying to do, just in a type safe manner

#

MISSING: Any causes me way to many headaches when I forget to check if the value is MISSING

#

The sentinels PEP isn't around yet, so I'm trying my best to follow the guidelines for type checking singletons

hearty shell
#

Humm, so with the Missing why would you need to pass it Type

#

Like what is the usage

clever bridge
#

The variable is meant to be an Enum class, not an instance of Enum

#

It's for defining a set of choices, so I need Enum.__iter__ functionality

hearty shell
#

Ohhh, I got what you are trying to do now

clever bridge
#

Yay!

#

The issue is stupid pyright is turning my type into an instance

#

And my if self.choices is not MISSING: check is saying yeet to the type checking as well

#

Typehinting self.choices as Missing[EnumMeta] makes it pass the check correctly, then pyright proceeds to complain that it's incompatible with _T@__iter__, which is just wrong

#

An if TYPE_CHECKING: block to cast self.choices to Type[Enum] after my MISSING check makes it work again though

hearty shell
#

Why are you using the class as the singleton though?

#

and not the instances

#

the instances are singletons

clever bridge
#

I tried to use a metaclass and a class and instances and all kinds of stuff to get it to work, it just won't deconstruct the Union correctly is the problem

#

The way PEP 484 makes it for deconstructing a Union with singletons is with an Enum, and it works exactly the way I want it to except in this one case where the other part of the Union is also an Enum

#

Either way, I need to run to bed now

hearty shell
#

What am I asking is that from the example you showed, it seems you are doing

class _MISSING(Enum):
    MISSING = "MISSING"

class SOMETHING(Enum):
    CHOICW = ...
clever bridge
#

Yes, _MISSING is a primarily internal value for when something is just not present (missing) and SOMETHING is a user-provided Enum

#

(I'll be back tomorrow, thank you for the help so far!)

hearty shell
#

aight, and I see now, I was wondering why would couldnt put both enums together

raw spear
#

what does future annotations do?

#

from __future__ import annotations

blazing nest
acoustic thicket
hearty shell
trim tangle
#

It's really disappointing to me that this hack is going to be in Python

hearty shell
#

What different does it make to the interpreter?

#

Were they not already stored as strings in the __annotations__ dict?

#

They were not nvm

trim tangle
hearty shell
#

Humm are there workarounds though? I had heard that the reason for the delay from 3.10 to 3.11 was pydantic

trim tangle
#

There is a workaround with typing.get_type_hints which runs eval on these annotations, giving the context of the class they are defined in. And it does seem to work in most cases.

hearty shell
#

I see, just finished reading 649, so basically better or equal in almost every instance except when your code relies on circular imports

#

Given that the other pep is already accepted, why is this was as a draft? They wouldn't change behaviour on 3.11 just to then go back on some next version right?

pastel egret
#

The runtime consequences were brought up after the fact, and after some discussion the Steering Council basically deferred making it default behaviour for 3.10 and then 3.11, so there's time to try and figure out a solution that works for everyone.

mortal fractal
#

cpython is unblocked I think?

#

🎊

solid light
#

If I have a load of TypedDict classes (e.g. AccountJson), how can I typehint that a tuple is of the NamedTuple equivalent without having to manually redo all the classes as a NamedTuple?

#

I found a way to automatically generate the NamedTuple variants, but since it uses globals().__setitem__ they're not recognised when I try to import them

trim tangle
#

can you show an example?

solid light
# 
from typing import NamedTuple, TypedDict


class AccountJson(TypedDict):
    id: int
    name: str
    email: str
    phone_no: str
    permissions: int


class AccountTuple(NamedTuple):
    id: int
    name: str
    email: str
    phone_no: str
    permissions: int
```like this, but I want to automatically generate the second one since it's the same. 
The script I came up with is:
```py
[
    globals().__setitem__(
        tuple_class_name := class_name.replace('Json', 'Tuple'),
        NamedTuple(tuple_class_name, list(class_obj.__annotations__.items()))
    )
    for class_name, class_obj in globals().copy().items()
    if class_name.endswith('Json')
]
```but as I said this causes issues with importing
trim tangle
#

Why do you need both?

#

or rather, why do you want to autogenerate them?

#

If you autogenerate them, you definitely won't get any type hinting support

solid light
# trim tangle Why do you need both?

I use both. Fetches to my database will return the NamedTuple version, and then I convert it to the TypedDict version in certain areas for easier use

#

I suppose I can just convert straight away and then I don't need the NamedTuple versions though

trim tangle
#

I don't see how it's less convenient than TypedDict

solid light
#

Hmm

#

I just realised my reason is mute

#

I was using TypedDict since I can access by name instead of index

#

But NamedTuple literally does that too

#

I guess there's still the fact that the API has to return a dict though, which is where I'd use TypedDict I guess?

trim tangle
solid light
#

It already is

#

I think

#

If I'm understanding correctly

#

The API returns stuff from the database, so return values match the structure of the database

trim tangle
#

do you know what "coupling" is?

solid light
#

Nope

trim tangle
#

The database structure is an implementation detail of how you store the data. You might want to change it if you want to change the database system you're using, or change the database structure (e.g. denormalize some data), or split the data across two different databases, or introduce a cache, or do something as trivial as rename a column.

#

But the public API is not supposed to change with the database structure, or even to match it. The API should make sense to the user of your service

solid light
#

The data returned by the API is the same structure as the database

trim tangle
#

Why does the API exist then?

solid light
#

Because I need to allow others to access the database

trim tangle
#

Can you explain what the project is supposed to do?

solid light
#

The overall project purpose?

trim tangle
#

yeah

solid light
#

It's a sports center

#

So p.e. I have an endpoint in my API for viewing all of your bookings

#

And the format it returns is the same structure as the database, just as a json (dict) instead of a tuple

#

It's basically```py
def get_bookings(account_id: int) -> BookingJson:
data = database.execute("SELECT * FROM bookings WHERE account_id=%s", (account_id,))
return {
"id": data[0],
"facility_id": data[1],
...
}

trim tangle
#

!e
If you really want to return the dict representing a tuple,

import collections
Foo = collections.namedtuple("Foo", "a b c")
foo = Foo(a=1, b=2, c=[3, 4, 5])
print(foo._asdict())
rough sluiceBOT
#

@trim tangle :white_check_mark: Your eval job has completed with return code 0.

{'a': 1, 'b': 2, 'c': [3, 4, 5]}
trim tangle
#

I assume this is not a real system that you'll have to change and expand, so it's fine

solid light
#

Hmm

solid light
#

I'll have to think about whether this does what I'm after

hasty hull
hearty shell
#

It got postponed again x)

solid light
trim tangle
#

no

#

if you're just returning this from an API, where do you return the dict?

solid light
#

The API returns the dict

trim tangle
# 
def some_http_handler(request):
    foo = database.find_foo(request["id"])
    if foo is None:
        return 404, "Not Found"
    return foo._asdict()
solid light
#

Ah right

#

So

#

Hmm

#

My foo isn't a NamedTuple, so this won't work

#

But

trim tangle
solid light
#

Yeah, exactly

trim tangle
#

like, database.find_foo() can return a NamedTuple

solid light
#

I'm not sure how well this will work

trim tangle
#

what DBMS are you using?

solid light
#

PostgreSQL & psycopg2 as the driver

#

Psycopg2 will return list[tuple] or just tuple

#

Hmm

#

This doesn't really work, so I can't really find how to make database.find_foo() return a NamedTuple

#

Like it's currently```py
def get_bookings_of_user(account_id: int) -> list[BookingJson]: # my API endpoint function
bookings = database.execute_sql("SELECT * FROM bookings WHERE account_id=%s", (account_id,), fetchall=True) # will be a list[tuple]
return [booking_to_json(tup) for tup in bookings]

trim tangle
solid light
#

Hmm

#

It says that's not valid though smfh

#

Or are you saying to make it return tuple instead of dict?

#

I need get_bookings_of_user to return the BookingJson TypedDict

trim tangle
#

why?

solid light
#

Because that's how I use it

trim tangle
#

why do you need named tuples then??

solid light
#

I'm honestly lost at this point

#

Lemme think

trim tangle
#

🙂

#

I am lost in life generally, hence I'm procrastinating here

solid light
#

lol

#

I think I don't need a tuple thing

#

So, new question instead

#

Do TypedDicts not have a constructor?

#

I've tried doing MyTypedDict(*tup) and it gives an error saying it doesn't take arguments

#

Should I instead be using a dataclass or something?

hearty shell
#

pip install attrs

#

and don't look back

hearty shell
#

because you can only start it via kwargs

terse sky
#

Why not a dataclass

#

Can be constructed from both positional and keyword args

#

I don't see much use for named tuples these days tbh

solid light
terse sky
#

What exactly does "dict like" mean?

hearty shell
#

Yes, it has a asdict method

#

actually sorry

#

it is not a method

terse sky
#

Yes, they can be converted to and from dicta

hearty shell
#

it is a function in the module

terse sky
#

But it doesn't have the actual API of a dict

#

But I doubt you want that

hearty shell
terse sky
#

i like having slots in principle but never actually needed it. kw_only on the other hand

#

not having that is painful. can't wait to upgrade.

#

I do regret choosing dataclasses overall (over attrs)

solid light
#

Like a .todict() or whatever

hearty shell
#

!docs dataclasses.asdict

rough sluiceBOT
#
dataclasses.asdict 

dataclasses.asdict(obj, *, dict_factory=dict)```
Converts the dataclass `obj` to a dict (by using the factory function `dict_factory`). Each dataclass is converted to a dict of its fields, as `name: value` pairs. dataclasses, dicts, lists, and tuples are recursed into. Other objects are copied with [`copy.deepcopy()`](https://docs.python.org/3/library/copy.html#copy.deepcopy "copy.deepcopy").

Example of using [`asdict()`](https://docs.python.org/3/library/dataclasses.html#dataclasses.asdict "dataclasses.asdict") on nested dataclasses...
solid light
#

If I have a class AccountDataClass, with an instance called account, and I do account.asdict() will I still get the auto-suggest for the keys?

#

Or does it lose that?

terse sky
#

yeah you lose that

solid light
#

In fact, thinking about it, I think I don't really need it to keep it. I can just typehint that it's a AccountDataClass or whatever

terse sky
#

dont need to keep what?

#

if the fields have different types then you basically want to avoid using it as a dict, generally speaking

#

you'll get auto completion for the fields by just typing .

#

and you don't have to quote anything

solid light
# 
@dataclass
class AccountDataClass:
    id: int
    name: str


def print_name(account: AccountDataClass):
    print(account.name)

account = AccountDataClass(1, 'Bob')
print_name(account.asdict()))```would this work?
terse sky
#

did you mean print_name there

hearty shell
#

you have to do dataclasses.asdict(account) I think

terse sky
#

and not print(name(...

solid light
#

Yeah, thanks @terse sky

terse sky
#

you don't need asdict at all

#

just print_name(account)

solid light
#

I have an API which will return this, and it needs to return a dict cause that's what the schema says it returns

terse sky
#

yes, that's fine, but usually what you want to do when you have dicts (and other json-like things) that have schemas is use a dataclass for them throughout your code

#

and do to/from dict conversions at the boundaries

solid light
#

Hmm

terse sky
#

you want to work with a raw dict as little as possible

solid light
#

Yeah

#

Agree

terse sky
#

but yeah it's common that you eventually serialize your dataclass back to json for example and return that

#

I'd also consider looking at pydantic early on if you're doing this kind of stuff

#

asdict is nice but it doesn't work recursively

#

so if you start having schemas that are nested it won't get the job done on its own

#

pydantic lets you define schemas, and they can be nested and it will still provide automatic to/from dict/json

solid light
#

So if I'm understanding this correctly, I want to changepy class AccountJson(TypedDict): id: int # id of the account name: str # account holder's name email: str # account holder's email phone_no: str # account holder's phone number permissions: int # account holder's permissions valueto be a dataclass?

terse sky
#

yeah

hearty shell
#

attrs asdict will work recursively though

solid light
#

And then I use .asdict() and .fromdict() as needed?

terse sky
solid light
#

What do you mean by "recursively" by the way? The nested thing?

terse sky
hearty shell
#

attrs ftw x)

terse sky
# 
@dataclass
class Foo:
    id: int

@dataclass
class Bar:
    foo: Foo
#

In this example, if you have a Bar and you asdict() it, the dictionary will simply contain a Foo

#

instead what you probably want is for the asdict() call to also get applied to the Foo member

#

so you end up with something that can actually go straight into json (i.e. just dict, list, str, numbers, None)

solid light
#

Ah right, I see, yes. I don't have any nested types so it's not actually an issue for me, but that's good to know, thanks

terse sky
#

Yeah, if you do have nesting later it can be annoying to change is the only thing. But if you're confident you won't you can just stick with dataclass

#

The main purpose of TypedDict is if you already have existing code, that passes around raw dictionaries everywhere.
It's not the ideal way to write it, but it works, and you don't want to mess with it, you just want to add type annotations to help you a bit, e.g. with minor modifications/additions

#

If you're writing code from scratch it's better to ignore TypedDict and just passes around dataclasses and do conversions at the last second when you need them

solid light
#

Apparently fromdict isn't a thing. Do I instead do Account(**{...})?

unknown.png
terse sky
#

yeah

#

well, no

#

There's no point doing **{}

#

you can just use keyword arguments

#

looks nicer and the IDE will flag mistakes

solid light
#

Or I suppose I can just do Account(*account)? Since it allows positional args?

terse sky
#

Yes, that was the next thing I was going to say

solid light
#

Right

terse sky
#

obviously that's a little more fragile

solid light
terse sky
#

Correct. Life is full of trade-offs 🙂

solid light
#

Right, thank you 👍

#

I don't need validation here, so I'll go for *account

terse sky
#

If you use pydantic, and probably there are ways with attrs too, it will automatically validate that the type is correct

solid light
#

I'd like to avoid going too deep

terse sky
#

yeah this is fine to start with

solid light
#

I don't really need validation anyway since I know 100% it will be valid since it's just what's returned from the database, which doesn't change

#

I suppose if I change the schema and forget it could be helpful, but not gonna worry about that

#

Thanks for the help 👍

# 
class DictConverter:
    def as_dict(self: dataclass) -> dict:
        if is_dataclass(self):
            return asdict(self)


@dataclass
class Account(DictConverter):
    id: int  # id of the account
    name: str  # account holder's name
    email: str  # account holder's email
    phone_no: str  # account holder's phone number
    permissions: int  # account holder's permissions value```I'm getting a warning on the `return asdict(self)` saying it has to be done on dataclasses. How do I get around that?
#

I'm essentially trying to make it so that I can do .as_dict() for each of my dataclasses (e.g. Account(*account_tup).as_dict())

terse sky
#

that would be account.as_dict()

#

not Account.as_dict()

#

what's wrong with as_dict(account)

hearty shell
#

I think they are doing that though

#

Account(*account_tup).as_dict()

terse sky
#

I know, I was just clarifying

#

that they want an instance method, not a class method

hearty shell
#

Yeah it should be working

solid light
#

I edited my message to correct that

terse sky
#

yeah, but why?

solid light
#

To save me having to import asdict in all the files I use dataclasses

hearty shell
#

It should be working, not sure how you care calling it

terse sky
#

eh tbh that's not a good reason. you're better off not adding this complexity for something like that.

solid light
terse sky
#

you can just ignore it

#

pycharm's linter isn't perfect

#

more important to see if mypy complains

#

(or something comparable to mypy)

solid light
hearty shell
#

ahhh

terse sky
#

yeah, i mean pycharm's linter is a guide

#

but this base class is kind of a headache, you can already see it violates its contract for non-datalcass derived classes

solid light
#

Hmm

#

I guess I should leave htat part then

hearty shell
#

Is dataclass a real type?

trim tangle
#

no

hearty shell
#

that contract is also wrong since you would have to return an optional

terse sky
#

just use the free function, tbh

#

it avoids all this

hearty shell
#

Yeah

trim tangle
#

yeah

terse sky
#

you have to individually import a lot of things, in basically every programming language

#

if you write asdict(account) and you ahven't imported it yet, pycharm will offer to automatically import asdict

solid light
#

Yeah, true

#

In fact, since I from datatypes import * I can just add it to my __all__, like this

unknown.png
terse sky
#

yeah, you can do that

solid light
# 
def _account_to_json(account: Union[list, tuple]) -> Account:
    """
    Convert account data into a dictionary.

    :param account:     account data
    :return:            account data as a dictionary
    """
    return asdict(Account(*account))```this seems to work
hearty shell
#

I use to also mind my imports, then I discovered isort and now I just put whatever import anywhere and pretend they aren't there

solid light
#

Not sure whether I should instead have -> dict

#

PyCharm doesn't complain though

terse sky
#

yeah, you're returning a dict

#

not an Account

#

also, for the annotation if you're just doing * you can just annotate it as Iterable

#

rather than list/tuple

# 
def _account_to_json(account: Iterable[Any]) -> dict[str, Any]:
    ...
solid light
#

I don't like using Iterable for some reason

#

So avoid it where possible

#

Have update to be a dict though

#

Now I'm more confused

#

I think I do actually want to be using TypedDict

#

Since I want to know what the keys of the dict are

hearty shell
#

Stop overthinking xD

solid light
#

The whole point of me adding this typing, is that I know what the keys are

#

I can't get that with dataclasses(?)

terse sky
#

yes, you can?

#

You just return the dataclass

solid light
#

It has to return a dict though

terse sky
#

if it has to return a dict, then it's because somebody else is calling it

#

maybe a step back would help

#

why does it have to return a dict?

solid light
solid light
hearty shell
#

Yeah but in that case, you dont need the types

terse sky
#

Okay, then this dict is being serialized over the network

#

so why do the types matter

hearty shell
#

it is no longer your concern

solid light
#

I don't understand what either of you are saying there

#

I want the types to make my programming easier

terse sky
#

when you say "response"

solid light
#

I've coded an API, and a program which uses that API. I want to know the types so that I can make development of the program easier

terse sky
#

it sounds like this is going over the network

solid light
terse sky
#

Okay. then you don't need the types once it goes over the network?

#

you're doing this conversion at the very last moment

#

(you should be)

solid light
#

Hmm

terse sky
#

somewhere you have a function that actually sends a dict as a response yeah

hearty shell
#

Ohh I think they are making a an api and are themselves using it

terse sky
#

def respond(data: Dict[str, Any])

hearty shell
#

But in that case you know it is correct

terse sky
#

yes, but they still aren't calling that function correctly, they're still sending a request

#

and receiving a response

solid light
terse sky
#

if you're using the API yourself, then what you can do is on the client side, have a function that receives the data, and turns it back into an account

#

Like, there's only two possibilities here: the client code can see the API code, or it cannot.
in the first case, you can just reuse the dataclass on the client side and immediately re-create it. And now you're using a dataclass again, which is much better than a typed dict.
In the second case, the client code is not benefitting from the type hints of the API code anyway.

#

but usually if you're writing code just for yourself, and the code is shared, there's not that much reason to convert things to json and back in the first place

#

you could just pickle stuff

solid light
# 
# The code for endpoint
def get_account(account_identifier: str) -> Response:
    """
    Returns the data of provided account.
    ...
    """
    account = _convert_to_account(account_identifier)
    if account[0] == -1:  # account not found
        abort(404, account[1])

    return make_response(asdict(Account(*account)), 200)

# Send request to the endpoint
def get_account(account_identifier: Union[str, int], *, auth_token: str) -> Account:
    resp: requests.Response = requests.get(
        f"{BASE_URL}/accounts/{account_identifier}",
        headers={
            "Authorization": f"Bearer {auth_token}",
            "Content-Type": "application/json"
        }
    )

    if resp.status_code == 200:
        return Account(**resp.json())```does this seem right?
#

I think that's what you were saying to do?

#

@terse sky

terse sky
#

These two functions seem to have identical names

#

But yes this seems like the idea

#

Really you want convert_to_account to already return an Account

#

Or an optional Account

solid light
#

They're in two different files, so that's why they have the same name

solid light
#

I'll make a note and change that

#

Then I just do```py
account = _convert_to_account(account_identifier)
if account is None:
abort(404, 'Account not Found')

return make_response(asdict(account))```

#

It means I lose some precision as to why it failed to convert (unknown email / username / id) but that's not a big deal, and if I really want I can make _convert_to_account return like Union[Account, Error] where error has a .status_code and .reason or something

#

Or even make _convert_to_account abort (I think I can do that)

solid light
terse sky
#

I probably wouldn't ever make things abort

#

no reason to do that

#

you can throw

#

and if it's not caught that will result in an abort

#

but more likely you want to at least catch at top level and log an error before exiting

solid light
#

I do catch all the errors when calling the API

#

So I think it's fine?

terse sky
#

yeah, that's fine, just saying you should throw and not abort

#

maybe that's what you meant

solid light
#

I am doing abort

#

Idk what the difference is

#

I'm using Flask's abort()

trim tangle
#

flask.abort(status, *args, **kwargs)
Raises an HTTPException for the given status code or WSGI application

#

well, it is throwing an exception

terse sky
#

awful naming by flask

trim tangle
#

i agree

#

sort of

terse sky
# 
os.abort()
Generate a SIGABRT signal to the current process. On Unix, the default behavior is to produce a core dump; on Windows, the process immediately returns an exit code of 3. Be aware that calling this function will not call the Python signal handler registered for SIGABRT with signal.signal().
#

this is what people who know about computers expect from a function named abort

trim tangle
#

I'm not sure I like the whole idea of "returning HTTP codes with exceptions"

#

from my (limited, granted) experience, it makes it tempting to put too much business logic into HTTP handles

terse sky
#

I can't comment as to whether the function should behave that way or not, it's not my wheelhouse.
But according to the docs, all the function is doing is throwing.

#

So it should be called raise or throw, or something similar

trim tangle
#

I would just raise HTTPException(your_code) and not add an extra abstaction

terse sky
#

abort is a super duper loaded term which to many people is going to mean specifically not throwing an exception, but simply ending everything immediately

solid light
#

I don't really like the name either

trim tangle
# 
abort(Response('Hello World'))

Excuse me... wtf

solid light
#

But it's what flask uses

terse sky
#

yeah I understand

#

anyway "calling abort" (i.e. throwing) is fine

#

or it could be fine, I don't reallyknow 🙂 but in that case you can return an Account rather than Optional[Account]

trim tangle
solid light
#

Nice lol

terse sky
#

well, lots of things are <something> goto's

#

returns are gotos

#

etc

#

all exceptions are gotos

trim tangle
#

well

terse sky
#

it's an easy way to completely exit the current request I suppose, even if you're nested

#

and then the framework I suppose catches that exception?

#

would be my guess

trim tangle
#

yeah, and sends a response

terse sky
#

Yeah. I don't think there's anything wrong with that function, it's arguably a little less error prone than telling peopl.e what exception type to throw

#

if they need to exit the current request

#

just the name is troublesome

#

if it was at least called abort_request,or better end_request_with_error

#

etc

twin marten
#

should a function that doesn't return anything be typed as -> None ?

hearty shell
#

Yes

#

Unless it is guaranteed to raise an exception, in which case if you wanna be fancy you can annotate it as typing.NoReturn

mortal fractal
#

oh now that pytype implements Final without crashing it's easier to check it for bugs on more codebases

mortal fractal
#

I think there's more I'll look later

#

most of the other errors are hard to find since pytype doesn't support | syntax yet outside of .pyi files

#

I think I'll wait for that before looking again

mortal fractal
#

@oblique urchin gratz on reveal_type

#

that was always going to be the least controversial, I think

#

assert_never() is the most interesting "new" addition, and reveal_locals() is the next least controversial probably

#

well, I don't think assert_never is controversial, just less established

oblique urchin
mortal fractal
#

I see

oblique urchin
#

personally I'm most excited about assert_type(), that actually unlocks new behavior

mortal fractal
#

what new behavior are you interested in? the thing that came to mind immediately is type checkers unifying their unit tests

oblique urchin
#

yes, typeshed unit tests

mortal fractal
#

not that they should literally share unit tests, but just everybody now has an obvious way to write them

oblique urchin
#

assert_never() can be implemented in user code and everybody implemented reveal_type() already

mortal fractal
#

has assert_type feedback has been positive?

oblique urchin
mortal fractal
#

ah yes you mentioned this 😄

#

Guido committed my change to ClassVar (it was an incredibly trivial change, after all)

#

except for one interesting interaction with a unit test

#

have you come across the python-dev thread about PR "spam"?

oblique urchin
#

yes

mortal fractal
#

I don't know what to make of that thread; weirdness all around

#

it seems non-explicitly directed at typing changes, but as far as I can tell nobody is really doing anything wrong

oblique urchin
#

there are indeed some people who just come and approve random PRs without saying anything

#

but the OP was clearly talking about Nikita's changes, which seemed quite unfair to him

#

So not great for that poster to just assume all of these changes were being made for GitHub points or something

mortal fractal
#

that sounds like something I would have done in middle school too if in the right position (approve random PRs for GH stats), and then be embarrassed about it 8 years later

mortal fractal
#

it seems like pytype often implements features separately for stub files and normal files

#

I wonder what about their codebase makes that necessary

oblique urchin
#

yes, they have a wholly separate parser for stub files

#

not sure why. Perhaps because originally they were for Python 2?

#

and stub file syntax was always Python 3

mortal fractal
#

Maybe

#

I use it sometimes with sets

#

Rarely

#

I don't recall seeing it in code much

#

Even for generic classes that aren't bultin

mortal fractal
#

I'm not so sure about the Strict name for conveying what the difference is, but I also don't have a better suggestion

blazing nest
solid light
acoustic thicket
#

just int

pastel egret
#

Exceptions aren't part of the typing system in general.

#

NoReturn is for the special case of a function that always raises an exception or otherwise quits - because then the type checker can understand that functions like sys.exit operate like a return or raise.

boreal ingot
#

I have been working on a pytest confest.py (not made it a plugin yet) that runs pyright on files ending in types.py (so like test_foo_types.py), still a few quirks to work out, but happy with the result so far

unknown.png
oblique urchin
blazing nest
#

I've only read the typing PEP in sections, isn't reveal_type() the only one stated there?

oblique urchin
#

reveal_type() isn't mentioned in PEP 484, but a bunch of other PEPs use it in examples without explanation

hasty hull
#

Just a warning its DX is not very good at the moment though lol

dire bobcat
# 
class Field:
    """
    Represents a database field.

    Parameters
    ----------

    table: Table
        The parent table class that
        the field is inside.
    """

    table: Table

class Table:
    """
    Represents a database table.
    
    Parameters 
    ----------
    name: str
        The name of the table.
    id: int
        The id of the table.
    fields: List[Field]
        The fields within the table.
    """

    name: str
    id: int
    fields: List[Field]
#

how do I typehint table: Table within the Field class?

#

bearing in mind ive defined Table afterwards..

oblique urchin
dire bobcat
#

ill use the latter

#

i dont like using it

#

it fucks with code i hear

hearty shell
#

You mean the former?

dire bobcat
#

im saying im using from __future__ import annotations but i dont like using it

hearty shell
#

Oh, lol

#

alright x)

dire bobcat
#

im doing ```py
bot.db = Database(
database_server=938485141616070696 # discord server to use for the database
)

#

where ```py
class Database:
"""
Represents a database connection
This class is used to interact with the Discord Database server

Parameters
----------
database_server: `int`
    The discord server to read, write and edit data from.

    .. versionadded:: 0.1
"""

def __init__(self, *, database_server: int):
    self._database_server = database_server
    self._tables = []
#

but bot.db isn't getting typehinted

#

like when i use methods with it

#

my ide isnt recognising it as a class of Database

#

how do I typehint it to Database?

#

I cant do bot.db: Database for some reason

oblique urchin
#

yes, type checkers won't recognize it if you add attributes outside the class definition

dire bobcat
#

how do i make it recognize it?

hearty shell
#

You could subclass discord.Bot or commands.Bot

#

and then add there

dire bobcat
#

ok

terse sky
#

I just upgraded my numpy and I'm seeing some type errors, some of which are surprising

#

it no longer considers ndarray to be a List[float] which probably makes sense

#

but a call to np.any is now complaining that it returns type bool_, not bool?

#

seems like just casting/constructing to bool from bool_ works. Kind of an odd situation but I assume numpy has a reason

trim tangle
blazing nest
dire bobcat
#

interesting

blazing nest
#

As if you wrapped all of them in quotes

oblique urchin
terse sky
#

And it was passing

trim tangle
#

how

#

why

oblique urchin
#

probably it was just Any

#

huh all github repos 500 for me

#

or I'd try to find the issue I alluded to above 🙂

solid light
twin marten
#

using VS Code with Python and Pylance extensions, I would expect the editor to show an error when trying to deconstruct the call to the function

unknown.png
#

how can I enable that?

soft matrix
#

the return type is "incompatible" with the actual implementation

#

if you wanted an error here this should be -> typing.NoReturn/Never

twin marten
#

still no error

unknown.png
hearty shell
#

yeah it should error

soft matrix
#

that should error

hearty shell
#

You can't unpack a tuple of one

#

into 3

soft matrix
#

you need to enable basic type checking

twin marten
#

exactly, do I need some pip package? VS Code settings?

soft matrix
#

its in extension preferences

#

you should enable it by default

fierce ridge
#

itd be interesting if we had a mypy bot here 👀

#

like !eval but !mypy

twin marten
hearty shell
#

That would indeed be great

soft matrix
#

Yep that's it

#

Does it now show errors?

twin marten
#

but this is cursed haha

unknown.png
#

better

unknown.png
soft matrix
#

Yeah sometimes error messages get very big

oblique urchin
#

also, fyi Tuple[int] means strictly a one-element tuple. It's very rarely a useful type

#

Tuple[int, ...] means a tuple of any number of ints

soft matrix
#

In your case you want tuple[int, int, int]

hearty shell
twin marten
hearty shell
#

👀

soft matrix
#

Jelle how does pyanalyze work with if TYPE_CHECKING?

#

Does it set it to True?

oblique urchin
#

at runtime TYPE_CHECKING is False so that's what it sees

#

it's something I need to support better

soft matrix
#

Does it not support cyclical imports then?

oblique urchin
#

as much as Python supports them

#

basically it imports everything, then checks every module independently

#

and just looks at the runtime type/function objects from other modules as needed

#

the main area where that causes problems is attributes, because you can't tell from a class definition what attributes actually exist

#

so for that it keeps some cross-module information around

#

and defers errors about non-existent attributes until the whole program has been checked

soft matrix
#

Right

#

Thanks for the explanation

cold osprey
#

hello, i am observing strange issue in some old code i am trying to annoatte, but fail to identify the cause... i have tried to isolate short example, but that one works no problem:

# writeutil.py
ItemType = TypeVar('ItemType')

def write_strings(writer: Callable[[str], Any]) -> Generator[None, ItemType, None]:
    while True:
        item: ItemType = yield
        if isinstance(item, str):
            writer(item + "\n")

from writeutil import write_strings

def some_usage(output: TextIO) -> None:
    sink: Generator[None, str, None] = write_strings(output.write)
    # other stuff to do...
    sink.send("ahoy")

Above seems to work fine, i get no error in mypy...
But in my code where there is a bit more complex module structure, i get error for line:
sink: Generator[None, str, None] = write_strings(output.write)
stating:
Incomatible types in assignment (expression has type"Generator[None, ItemType, None]", variable has type "Generator[None, str, None]")
Do i have some invalid annotation or need to annotate function call somehow?

soft matrix
#

ItemType has no meaning in that function

cold osprey
#

sure i understand that, but why is it not unpacked/interpreted before importing write_strings... or do i need to import it to make it work?

#

i mean above code excerpt does not report mypy error, but due to some possibly other code/relations in my codebase error is reported, which is a bit confusing to me

soft matrix
#

why are you making this generic in the first place?

cold osprey
#

maybe i didnt get some concept right, am not sure how to specifiy when using the "consumer/generator", what type of variable it should provide

#

it's some legacy code (different author) -> write_strings is supposed to be (in my understanding), generic sink that logs str only from any received items

soft matrix
#

you should use just str instead of a ItemType

#

and then it will work

cold osprey
#

but wont it prevent other items to be sent to that "generator"?

soft matrix
#

no but thats why you use a type checker, it will tell you if you send something to this that you shouldnt

cold osprey
#

i mean legacy code is dumping/sending all the different types of items into that fn, that/s why i assumed it has to stay "typevar"

#

but now i realize i can use "Any" instead of typevar and functionality will stay basically same 🤦

soft matrix
#

you shouldnt use Any, that defeats the point of adding type checking to this function

cold osprey
#

i realize that, but the functionality of that write_strings was "prescribed" as "accepts all types of items, does internally something to strings only"

soft matrix
#

why though?

#

thats some very strange api

cold osprey
#

that's good question no one can possibly answer at this point 🙃 it's rather old chaotic codebase 😦

#

to clarify - i am annotating some small/mid size app created in the past to make it a bit easier to orient-in and possibly allow code cleanup at later stage

#

one difference in target codebase is, that the write_strings is wrapped in some functools.wraps and coroutine (not the python async, but manually annotated fn), which adds possibly some extra fluff 😦

#

thanks for directing questions though, worst case i stay with Any instead of ItemType = Typevar as a workaround until code is reworked to more reasonable state

solid light
#

If I have a list of types, is there a way to typehint "one of this list"? Something like```py
API_TYPES = [Account, Booking, Facility, Login, Sport, Token]
API_TYPE = Union[*API_TYPES]

trim tangle
#

Why do you want this?

solid light
#

Gimme a sec

#

Basically I have a parameter which will be either ClassType or 'ClassType[]'

#

And I need to be able to typehint the parameter as Union[ValidClassTypes, str]

# 
API_TYPES = {class_.__name__: class_ for class_ in (Account, Booking, Facility, Login, Sport, Token)}
API_TYPE = Union[Account, Booking, Facility, Login, Sport, Token]

def perform_test_not_expecting_error(
    test_name: str,
    function: Callable,
    *args,
    expected_response_type: Optional[Union[API_TYPE, str]] = None,
    **kwargs
) -> None:

    try:
        result = function(*args, **kwargs)
    except Exception as e:
        raise UnexpectedError(test_name, type(e), e.args[0])
    else:
        if expected_response_type is not None:  # expected_response_type was provided

            if isinstance(expected_response_type, str):
                if expected_response_type[-2:] != '[]':
                    raise InvalidResponseType(test_name, expected_response_type)

                expected_type = API_TYPES.get(expected_response_type[:-2])
                if not expected_type:
                    raise InvalidResponseType(test_name, expected_response_type[:-2])

                # We expect `result` to be list[expected_type]
                # First make sure that result is a list:
                if not isinstance(result, list):
                    raise InvalidResponse(test_name, expected_response_type, result)

                # And now ensure each item is of type `expected_type`
                for item in result:
                    if not isinstance(item, expected_type):
                        raise InvalidResponse(test_name, expected_type.__name__, item)

            else:
                if not isinstance(result, expected_response_type):
                    raise InvalidResponse(test_name, expected_response_type.__name__, result)

            print(
                f"Test {test_name!r} passed! No unexpected errors occurred "
                f"and got the expected response type of ({expected_response_type})."
            )
```This is the actual code. Essentially ensuring that a function returns the expected type
#

This should work, was just wondering if there was a nicer way

solid light
#

Actually it doesn't seem to work. PyCharm gives this warning:

unknown.png
#

No clue what's going on

grave fjord
#

why do most of the asyncio lock primatives always return True

#

I supposed to feel a bit like threading.Lock?

trim tangle
#

Heyyy

#

!pep 681

rough sluiceBOT
#
**PEP 681 - Data Class Transforms**

Link

Status

Draft

Python-Version

3.11

Created

02-Dec-2021

Type

Standards Track

trim tangle
#

Not sure how I feel about this.

#

It definitely sounds like a bandaid

#

Weirdly specific, that's a better word maybe

soft matrix
#

I like it but a better more general purpose solution would be very cool

terse sky
#

idk, it may feel weirdly specific until you realize that what dataclasses are basically doing is bringing what would largely be built-in class functionality to python

#

If you want to statically describe your class, and you also have a typical init function, you're already repeating each field four times, which is horrible

#

This would actually solve the exact issue I've had in the past where I want to write my own decorators that do relatively simple things before delegating back to dataclass/attrs decorators (like change defaults, add some other simple functionality, etc)

#

you can't really do this right now because your decorator won't work properly with the typechecker

grave fjord
grave fjord
soft matrix
#

Yeah they'd be nice

trim tangle
#

I would imagine some kind of procedural language on types. Like in Zig

#

But that would be quite a lot of work to implement in all the different type checkers

void panther
#

Does pyright see function as a method here?

class Test:
    def __init__(self, function):
        self.function: Callable[[int], int] = function

  file.py:6:14 - error: Cannot assign member "function" for type "Test"
    Expression of type "() -> int" cannot be assigned to member "function" of class "Test"
      Type "() -> int" cannot be assigned to type "(int) -> int"
        Function accepts too many positional parameters; expected 0 but received 1 (reportGeneralTypeIssues)
#

or where is it getting from when the param is not typed?

hearty shell
#

Weird that this is the issue

#

although if you move it to a to function signature the error stops

#

which is where it should be anyway

#

I honestly can't parse Pyright erros, why cant they be like mypys xD

cold osprey
#

where can i identify/find type annotation for argparse.ArgumentParser.parse_args() method? googling does not give any obvious hints/reasonable links for my understanding, and it feels like quite widespread lib imho to have this already addressed
My legacy code has nsargs (output of parse_args()) pased around, and i need to annotate it properly - Any feels a bit too generic

hearty shell
#

Maybe you can hunt it down but there doesn't seem to be any typing for that

rough sluiceBOT
#

Lib/argparse.py line 1824

def parse_args(self, args=None, namespace=None):```
hearty shell
#

Maybe with the help with typeshed

#

which also doesnt seems to document it from just looking at it

#

Actually nvm it is right there, how did I miss it lol

# 
    # The type-ignores in these overloads should be temporary.  See:
    # https://github.com/python/typeshed/pull/2643#issuecomment-442280277
    @overload
    def parse_args(self, args: Sequence[str] | None = ...) -> Namespace: ...
    @overload
    def parse_args(self, args: Sequence[str] | None, namespace: None) -> Namespace: ...  # type: ignore[misc]
    @overload
    def parse_args(self, args: Sequence[str] | None, namespace: _N) -> _N: ...
    @overload
    def parse_args(self, *, namespace: None) -> Namespace: ...  # type: ignore[misc]
    @overload
    def parse_args(self, *, namespace: _N) -> _N: ...
#

@cold osprey

cold osprey
oblique urchin
#

That's not something that the standard type system can do in this case

grave fjord
cold osprey
cold osprey
grave fjord
cold osprey
#

neat! 🙂 i started on about 500-ish errors in strict mode, and am down below 10 currently, fun few days it's been 🙃

acoustic thicket
#

nice

blazing nest
#

What's the status of MyPy ParamSpec? I want to use it but this project needs to support MyPy 😔

hearty shell
#

They said they would bring basic ParamSpec support this month

#

idk what is meant by basic though

oblique urchin
#

The latest release supports it for functions I believe, but no Concatenate yet

hearty shell
blazing nest
#

Oooh, that's fine! I just need ParamSpec

#

I have a decorator that applies return values :3

blazing nest
#

Is it possible to write typings for a module using ModuleType somehow maybe? It's for Cython. There's a file called Cython/Shadow.py which is imported under cython. I can probably type a Protocol and then be able to use from cython import my_module.

In reality I can patch sys.modules to be able to use from cython.my_module import ... but is there a way to tell the static type checker about this?

soft matrix
#

Implementing ModuleType might not even be necessary as typing.re/io never did that and I think they were usable as modules

oblique urchin
soft matrix
#

Oh well maybe it doesn't work then

blazing nest
#

🤔 yeah that's a shame

blazing nest
#

I assume it's complaining about the positional-only character

oblique urchin
#

that means it doesn't have a parser that can understand pos-only syntax

blazing nest
#

Ah, hmm. That puts me in a very difficult situation.

oblique urchin
#

to fix it you could make typed-ast support positional-only syntax 😄

#

Sebastian Rittau tried that years ago but couldn't get it to work

blazing nest
oblique urchin
#

and now we've basically given up on typed-ast and mypy uses the builtin parser

blazing nest
#

I see

soft matrix
#

Why do you need positional only anyway?

#

Just prefix it with a __ and it should achieve the same thing

grave fjord
oblique urchin
grave fjord
#

No the superclass takes a self, lock: Lock

#

This is a self, *, loop: AEL

oblique urchin
#

the comment you're pointing at is a class with no superclass?

grave fjord
#

class Lock(_ContextManagerMixin):

#

You need to expand the line for the fragment to link :/

oblique urchin
#

oh right, mypy doesn't care about LSP violations for __init__

#

because they're too common in practice

grave fjord
#

Not sure what to do about the loop kwarg though

#

It's still there in 3.10 - it just NoReturns, so @overload?

oblique urchin
#

like it always throws an error if you supply it?

#

then I'd just omit it from the stub, so type checkers can tell you not to pass it

grave fjord
#

Hmm, might be a change that's insufficiently esoteric that mypy_primer might complain

oblique urchin
#

seems like we'd generate new true positives

acoustic thicket
#

what should be the typehint for other here

class A:
  def __lt__(self, other) -> bool:
    if not isinstance(other, A):
      return NotImplemented
    return ...

other: object?

#

also i dont need to account for NotImplemented in the return type annotation, do I

soft matrix
#

other: object is correct

#

And bool is fine yes

acoustic thicket
#

aight

mortal fractal
#

do pyright strict next, then pyre non-strict 😎

tranquil turtle
cunning plover
# tranquil turtle What should be returned if we dont know how to compare objects? False, True or N...

how about


class A:
  def __lt__(self, other) -> bool:
    if not isinstance(other, A):
      raise NotImplementedError("describe the error")
    return True

a = A()
b = 5

print(a<b)

Traceback (most recent call last):
  File "/home/naa/repos/exp/test.py", line 14, in <module>
    print(a<b)
  File "/home/naa/repos/exp/test.py", line 8, in __lt__
    raise NotImplementedError("describe the error")
__main__.NotImplementedError: describe the error
#

;b

#

lets it be raised as a NotImplementedError

#

NotImplementedError is already existing built-in available exception pithink

#

nobody forces to consider raised errors in type hints 😁

soft matrix
#

Returning NotImplemented is better I think

#

You get a TypeError

acoustic thicket
#

NotImplementedError is for a different purpose, when a base class needs to have its subclasses implement a method

class BaseClass:
    def method_that_has_to_be_overriden(self):
        raise NotImplementedError

class Child:
    def method_that_has_to_be_overriden(self):
        # implementation
acoustic thicket
cunning plover
#

it sounds funny to return NotImplemented only in order to have hidden TypeError

soft matrix
#

It's shorter and the message is pretty obvious

cunning plover
#

the behavior is obscure in comparison to raising exception directly

blazing nest
cunning plover
#

looked with confusion at appeared from nowhere __rand__ and __and__

#

completely expected behavior, completely expected, thought Darkwind

#

i thought we just had overloaded < operator

blazing nest
#

Yeah I know we're talking about __lt__, where the behaviour would be __gt__ I believe: ```python

x < y

result = x.lt(y)
if result is NotImplemented:
result = y.gt(x)

if result is NotImplemented:
raise TypeError(...)

cunning plover
#

okay, I get it where appear rand and and

blazing nest
#

!e ```python
class Lefthand:
def add(self, other):
return 1

class NotImplementedHand:
def add(self, other):
return NotImplemented
def radd(self, other):
return NotImplemented

class Righthand:
def radd(self, other):
return 2

print(Lefthand() + Righthand())
print(NotImplementedHand() + Righthand())
print(Lefthand() + NotImplementedHand())
print(Righthand() + Lefthand())

rough sluiceBOT
#

@blazing nest :x: Your eval job has completed with return code 1.

001 | 1
002 | 2
003 | 1
004 | Traceback (most recent call last):
005 |   File "<string>", line 18, in <module>
006 | TypeError: unsupported operand type(s) for +: 'Righthand' and 'Lefthand'
blazing nest
#

__and__ is for & and originally the first example I could come up with, but it's probably easier to visualize __add__

mortal fractal
#

yes this is why you return notimplemented

rustic gull
#

Can a bytearray be passed where AnyStr is expected?

hearty shell
#

Apparently yes

#

Ah there, I found the relevant docs

#

As a shorthand for this type, bytes can be used to annotate arguments of any of the types mentioned above.

#

so bytes is just promoted to typing.ByteString which

#

This type represents the types bytes, bytearray, and memoryview of byte sequences.

#

This is rather counter intuitive, but it seems that because almost no one uses bytearray, it is made an exception in the type system for convenience

#

@rustic gull

#

I wonder how many of these "conveniences" are there

vale glen
#

Is there a good way to set a method on a class to be a union type?

class SomeClass:
    def example(self) -> list[int]:
        return [1, 2, 3]


class SomeChildClass(SomeClass):
    example = [1, 2, 3]

In this example I want example to be Callable | list
Declaring it on the child class with a type removes warnings (eg. example: list = [1, 2, 3]) but wondering if there is a way to define this on the parent class?

One workaround we had was to do this:

class SomeClass:
    def default_example(self) -> list[int]:
         return [1, 2, 3]

    example: Callable | list = default_example
hearty shell
#

What is wrong with the workaround? Although isnt that a runtime error?

#

because default_example isnt defined yet

#

Also the workaround isnt really that, doing example: list = [1, 2, 3] on the subclass should still lead to warnings

vale glen
#

Sorry, yeah, edited

#

No warnings with that workaround using pylance

vale glen
hearty shell
#

Well, a better way would be to see why you are needing to change an method into an attribute when you are subclassing

vale glen
#

Our base class defines the default implementation but the child can override it with a hardcoded list, if it likes

The class is used to build pages from SQLAlchemy models
The method on the base class will dig into a model and get a list of columns
A child class might want to manually set a subset of columns to use

#

We could make the child use a method and just return the hardcoded list, but would be nice if we could type hint that it's a union type and use a list

brisk hedge
#

Unfortunately method definitions introduce a new type declaration, i.e. shadow the old example: list | Callable
Therefore you need to explicitly assign it as an attribute if you want a "method-or-literal" like so

#

And there's really no way around that

vale glen
#

Thanks for the info, suppose we'll stick with the workaround of setting it to a default explicitly 😄

hearty shell
#

Yeah actually I am very confused now

# 
class SomeClass:
    def default_example(self) -> list[int]:
         return [1, 2, 3]
    
    example: Callable[[Any], list[int]] | list[int] = default_example

class SomeChildClassA(SomeClass):
    example = lambda self: [1,2,3]

class SomeChildClassB(SomeClass):
    example: list[int] = [1, 2, 3]

class SomeChildClassC(SomeClass):
    def default_example(self) -> list[int]:
         return [1, 2, 3]
    example = default_example

class SomeChildClassD(SomeClass):
    def example(self) -> list[int]:
         return [1, 2, 3]
# 
main.py:23: error: Signature of "example" incompatible with supertype "SomeClass"
Found 1 error in 1 file (checked 1 source file)
#

Yeah I dont know what is happening here x)

#

Also if I change the Any to a TypeVar bound to SomeClass then I also get an error on SomeChildClassC

# 
main.py:20: error: Incompatible types in assignment (expression has type "Callable[[SomeChildClassC], List[int]]", base class "SomeClass" defined the type as "Union[Callable[[T], List[int]], List[int]]")
main.py:23: error: Signature of "example" incompatible with supertype "SomeClass"
brisk hedge
#

Try either ... or [Self] (if your type checkers supports it)

hearty shell
brisk hedge
#

Callables are contravariant on their arguments, so (ChildD) -> list[int] isn't compatible with (Any) -> list[int]

hearty shell
#

I can try ... but mypy doesnt support Self yet, maybe on main

hearty shell
#

Wait the other way around right?

#

(ChildD) -> list[int] isn't compatible with (bounded T) -> list[int]

#

Because SomeChildClassC did work with Any

#

But I had completely forgotten about the contravariant part

#

Using ... gives the same as using Any, and yeah Self wasnt merged in yet

brisk hedge
#

Self is indeed the behavior you're looking for

#

You can probably use a bound typevar as a replacement but I'm not 100% sure how that interacts with contravariant attributes and subclassing

hearty shell
#

I see, although if I just use a generic it should work no? The thing I find weird here is mostly the inconstancy in erros

# 
class SomeClass:
    def default_example(self) -> list[int]:
         return [1, 2, 3]
    
    example: Callable | list[int] = default_example

class SomeChildClassA(SomeClass):
    example = lambda self: [1,2,3]

class SomeChildClassB(SomeClass):
    example: list[int] = [1, 2, 3]

class SomeChildClassC(SomeClass):
    def default_example(self) -> list[int]:
         return [1, 2, 3]
    example = default_example

class SomeChildClassD(SomeClass):
    def example(self) -> list[int]:
         return [1, 2, 3]
#

SomeChildClassA is perfectly fine with it, I guess it is because lambdas arent typed checked the same way

#

SomeChildClassB is fine

#

SomeChildClassC is fine, but SomeChildClassD is not fine

brisk hedge
#

Yeah odd

tranquil turtle
hearty shell
tranquil turtle
#

my bad

hearty shell
#

Their issue is solved, all that remains is mypy trickery

#

x)

tranquil turtle
#

we can use metaclass to annotate and implement class-property

tranquil turtle
hearty shell
#

?

hearty shell
whole panther
#

any reason why mypy raises invalid syntax on this line

unknown.png unknown.png
tranquil turtle
#

it complains about type comment

#

# type: here.should.be.valid.python.syntax - you have syntax error here

oblique urchin
#

I think the issue is you have both an annotation and a type comment

#

you only need one

whole panther
oblique urchin
whole panther
tranquil turtle
#

because you have errors in code or your annotation isnt valid

whole panther
#

alr ima try a workaround

hearty shell
#

why are you annotating it as Type?

#

fp: Union[str, Type[os.PathLike], Type[io.BufferedIOBase]]

#

@whole panther

whole panther
#

work around was just to set it as io.BufferedIOBase and use pre on the validator

hearty shell
#

Ah alright

rustic gull
whole panther
#

is there a way of using Generic TypeVars in a Union?

soft matrix
#

Generic TypeVars aren't a thing lemon_pensive

#

can you give an example of what you mean?

mortal fractal
#

HKTs eventually; I think there's still other higher priority/lower hanging fruit in python typing though

hearty shell
whole panther
#

this is what mypy wants

#

which is fine

#

till u stick it in a union

brisk hedge
#

Generic on its own does nothing

#

What are you trying

soft matrix
#

and shouldn't be allowed

#

neither should a type var with only one constraint

whole panther
whole panther
# brisk hedge What are you trying

so theres a typevar which represents an str which has its own formatting and what not.
So doing Union[..., T] is fine, but MyPy asks for Generic[T] or Protocol[T] and im abit stuck here

soft matrix
#

can you show the entire code?

#

!paste

rough sluiceBOT
#

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

whole panther
#

MyPy

acord\payloads.py:90: error: Type variable "acord.models.Snowflake" is unbound
acord\payloads.py:90: note: (Hint: Use "Generic[Snowflake]" or "Protocol[Snowflake]" base class to bind "Snowflake" inside a class)
acord\payloads.py:90: note: (Hint: Use "Snowflake" in function signature to bind "Snowflake" inside a function)

Code

SN = TypeVar("SN", str, int)

class Message(pydantic.BaseModel, Hashable):
    message_reference: Union[dict, SN]
oblique urchin
#

I don't know what that type is supposed to mean. Can you explain what you want message_reference to hold?

whole panther
#

to reply to a message you can use its ID or a message ref object which is a dict

#

you also receive the reference as a dict

#

SN stands for snowflake which is the ID

hearty shell
#

But how are you actually using the code?

whole panther
trim tangle
#

It's not clear to me either

hearty shell
#

Well, at some point you are passing something to message or using in some way

whole panther
hearty shell
#

At that point is probably where the error originates

trim tangle
whole panther
# 
return Message(**(request.json()))
trim tangle
whole panther
trim tangle
#

What is an SN object?

whole panther
#

the typevar

trim tangle
#

What is it? a string? a number? a list?

hearty shell
#

It is the type var, but I dont get why no just use the Snowflake then

trim tangle
#

Why not this, for example?

class Message(pydantic.BaseModel, Hashable):
    message_reference: Union[dict, str, int]
whole panther
whole panther
hearty shell
#

? I thought it was a Snowflake, now I am more confused

trim tangle
# whole panther guess that works

If you want a type alias for a repeating pattern, you can do this:

Snowflake = Union[int, str]

class Message(pydantic.BaseModel, Hashable):
    message_reference: Union[dict, Snowflake]
#

Unrelated question:
are there any resources/blogs/tutorials that focus on typing? apart from the docs of mypy, pyright and pyre

mortal fractal
#

@oblique urchin I'm going to look into that callable() check to see what removing it breaks (if anything) and if there's alternatives; it was the cause of that dataclasses bug and will probably continue to be a surprise/source of bugs to anyone trying to implement their own typeforms

whole panther
oblique urchin
oblique urchin
hearty shell
#

Or did you just define a Union on that module?

mortal fractal
hearty shell
#

called Snowflake

whole panther
mortal fractal
#

(it's not in the PEP)

whole panther
#

called Snowflake

trim tangle
# oblique urchin we're trying to put something together at typing.readthedocs.io, but not much pr...

I was thinking of making some kind of micro-blog where I post some easy to understand articles (preferably without type theory nonsense). Also including some common pitfalls and misunderstandings (like using NoReturn when the function simply doesn't have a return, or using a typevar when you need a type alias, or using Any when you probably wanted object, etc.).
I wonder if that would be useful to at least one person

mortal fractal
#

idk if pyre is the best beginner resource because it has some very strict views on a few things

oblique urchin
trim tangle
oblique urchin
whole panther
trim tangle
#

I'll try to start something today, I think

#

I am in dire need of a pet project

hearty shell
#

Ahhh, yeah in that case you should be able to just use it as a TypeVar, if you need to proliferate the Type forward

brisk hedge
#

So to subclass

whole panther
#

ah

trim tangle
#

well, in your case you don't need type vars

whole panther
#

ye forget that

trim tangle
#

Why not wrap this raw value in a proper object? I can imagine it's a pain in the ass to deal with a value that's either an int or a string-encoding-an-int

#

like ```py
@final
class Snowflake:
def init(self, source: Union[int, str]) -> None:
...

def timestamp(self) -> datetime:
    ...
hearty shell
#

You could also then make so that the conversion is in the model itself

#

Or idk, do you have to do that for every field that uses Snowflake? Again never used pydantic

trim tangle
# trim tangle I was thinking of making some kind of micro-blog where I post some easy to under...

This is my plan for the near future:

- `TypeVar` series:
    - What `TypeVar`s are for (copy&paste from dev.to)
    - Using standard library generics
    - Variance
    - Making your own generics
    [- More examples of generics]
    [- Overengineered/advanced/esoteric uses of generics]
- `NewType` vs `TypeVar` vs type aliases
- "Best practices" series?
    - Use `object` instead of `Any`
    - Use read-only collection types
    - "Type smells"?
    [- When to _not_ use type hints]
- Interfaces: ABCs and Protocols
``` any additions/comments?
#

it would also be cool if some native speaker dedicated a bit of time to read the mess that I write lol

oblique urchin
trim tangle
#

ah

#

yeah

oblique urchin
#

variance in general

#

especially as applied to protocols and callables

hearty shell
#

ParamSpec and Concatenate and just annotating decorators in general might be something else

#

it is very confusing

trim tangle
#

I have seen some bizzare misuses of typing in some real code, so I think I'm slightly more equipped to talk about stuff than a year ago lemon_pleased

#

One issue is, though: how do you embed code snippets with errors in some standard way?

#

Maybe that's a good use case to add to pyright-playground

#

like, export a thing you can embed that will show errors

#

@oblique urchin Completely random but: how to pronounce your name?

oblique urchin
trim tangle
#

ah