#type-hinting

1 messages ยท Page 69 of 1

grave fjord
#

ah of course

#

it wouldn't be python if there weren't two wrong ways to do it

covert dagger
#

and math one obviously can't handle complex or anything else either

trim tangle
#

map and zip technically are not functions, but god damn

unknown.png
buoyant swift
#

they're called built-in functions by the docs though ๐Ÿ˜”

hearty shell
#

This would be solved by the new TupleVarTypes right?

covert dagger
#

itertools has fun stuff like that too

trim tangle
covert dagger
covert dagger
terse sky
#

if it's any consolation python is in pretty good company

#

properly supporting variadics is a very rare feature

#

dig deep enough into Rust, Scala, many other languages with "nice" type systems (maybe even Haskell, not sure) and you will see things very similar to that

#

except of course, since these are statically typed languages, you don't just have the type annotations repeated for N different arities, but an implementation as well

trim tangle
#

compared to the very small advantage that not even all users of asyncio.gather will benefit from or even notice

trim tangle
oblique urchin
trim tangle
#

I guess someone could do this ```py
things = await asyncio.gather(...)
things.append(42)

#

the documentation also states that it returns a list, not just any iterable

boreal ingot
#

asyncio.sleep not defining what the default type of result is, so using it without passing 2 args, the return type is unknown

grave fjord
#

gather is getting replaced by broken task groups

#

So yay for that I guess

boreal ingot
#

Type of "asyncio.sleep(1)" is "Coroutine[Any, Any, _T@sleep]" while Type of "asyncio.sleep(1, None)" is "Coroutine[Any, Any, None]"

trim tangle
#

what's broken about them?

#

(haven't looked deeply into them)

grave fjord
soft matrix
#

The strangest part about the whole uncancel thing is that it's meant to be removed at some point

#

So why's it not private?

brazen jolt
#

Is it possible to specify what type a typevar should be from within a function body?

def foo(x: Mapping[K, V] | Iterable[K]) -> Mapping[V, K]:
    if isinstance(x, Iterable):
        cast(V, int)  # Doesn't work
        return {i: v for i, v in enumerate(x)}
    else:
        return {v: k for k, v in x.items()}
``` I know I could use overloads, but I actually need this for a generic class where I will need `V` later, so I need to to be set
trim tangle
# brazen jolt

cast only returns a new value, it doesn't mutate the type

brazen jolt
#

ah, yeah, forgot V =

#

still though, it wouldn't work

trim tangle
#

You can do x = cast(Mapping[K, V], x)

brazen jolt
#

but it won't set V

trim tangle
#

It won't

brazen jolt
#

I need to set V since again, this is actually a constructor for a generic class which uses V later

trim tangle
#

Besides, you know that every mapping is iterable, right?

brazen jolt
#

is it though, I was wondering that, but issubclass(list, Mapping) returned false

trim tangle
#

Because an iterable is not necessariy a mapping, you checked the other way around

brazen jolt
#

ig type-wise it may be though

trim tangle
#

The else branch will never fire, that's what I meant

#

Because every mapping is necessarily an iterable

brazen jolt
#

ah

#

it could be swapped then

trim tangle
#

Also, I suppose you meant Iterable[tuple[K, V]]?

brazen jolt
#

that's the way it's actually in the code

brazen jolt
trim tangle
#

Where is it going to take the Vs then?

#

Like, if I pass [1,2,3]

brazen jolt
#

it can just be a list like: ["a", "b", "c"] which will mean K will be string and V should be int (I'm using the indices)

trim tangle
#

Ah

brazen jolt
#

I just don't know how to inform the typing system that V should become int here

trim tangle
#

That's a really strange function, anyway

brazen jolt
#

it is

#

it's not mine..

trim tangle
brazen jolt
#

how?

trim tangle
#

typing.overload

brazen jolt
#

wouldn't that just hard-code it as int

#

but not set V to it

# 
@overload
def foo(x: Iterable[K]) -> Mapping[int, K]:
#

this is true

#

but it doesn't set V

#

I then need to do something like ```py
def encode(key: K) -> V:
...

#

which would be a problem

trim tangle
brazen jolt
#

then K isn't set

trim tangle
#

In the implementation, just type: ignore stuff

brazen jolt
#

hm

#

could i make a protocol instead?

trim tangle
brazen jolt
#

something that would be generic over K and V and cover both list and mapping like this?

#

since I still need to take both K and V in the implementation, I can then just overload the specific things

trim tangle
brazen jolt
# 
@overload
def foo(x: Iterable[V]) -> Mapping[int, V]:
    ...

@overload
def foo(x: Mapping[K, V]) -> Mapping[V, K]:
    ...

# Could Subscriptable be made somehow?
def foo(x: Subscriptable[K, V]) -> Mapping[V, K]:
    ...
brazen jolt
#

it'd be really neat if something like Subscriptable protocol could be implemented somehow, sharing the things common between iterable and mapping and clearly having both K and V which would be obtained from the passed type then

trim tangle
#

You can't type something as "iterable but necessarily not a mapping"

#

That's the root issue here

brazen jolt
#

you mean every Iterable[T] is just Mapping[int, T] by default and that would pass type-wise?

#

I don't really understand

trim tangle
#

e.g. a dict mapping str to str is an iterable of str

brazen jolt
#

well yeah, but that doesn't help me

trim tangle
#

Why do you need this function though?

#

Can you show how it's used maybe?

#

Maybe you want Sequence instead of Iterable? That should theoretically solve it

brazen jolt
#

well, it's a class that's supposed to store both the original mapping and a reversed one, for O(1) lookups both ways. But when a list instead of a dict is passed into __init__ it should instead use the positions as the values and the elements in the list as keys (with a reverse mapping of that generated later)

So for example if ["a", "b", "c"] would be passed into the constructor, the class should store self.x = {"a": 0, "b": 1, "c": 2} and also self.rev_x = {0: "a", ...}

brazen jolt
trim tangle
#

Can you just disallow passing in a non-mapping?

brazen jolt
#

I can't, it's not my code and for backwards compatibility, the owner didn't like that..

#

that's why I'm trying to type-hint it somehow but I've no idea how

trim tangle
#

Make one overload with Sequence and another with Mapping

brazen jolt
# 
from typing import Mapping, Iterable, TypeVar, Generic, cast, overload

K = TypeVar("K")
V = TypeVar("V")

class Example(Generic[K, V]):
    x: Mapping[K, V]
    rev_x: Mapping[V, K]
        
    def __init__(self, x: Mapping[K, V] | Iterable[K]) -> None:
        if isinstance(x, Mapping):
            self.x = x
        else:
            # With an iterable/sequence, we use the indices as values
            # and the elements as keys.
            # That's why I wanted to manually set V to int, since that's
            # the type of the values of this dict now
            self.x = {v: i for i, v in enumerate(x)}
            
        self.rev_x = {v: k for k, v in self.x.items()}

    def rev_lookup(self, key: V) -> K:
        return self.rev_x[key]

    def lookup(self, key: K) -> V:
        return self.x[key]
#

this is a very simplified example of the actual class

pastel egret
#

Would a self-type specification work in the type checkers you care about?

brazen jolt
#

yeah, I can use self type

trim tangle
# 
@overload
def __init__(self: 'Example[int, V]', x: Sequence[V]): ...
@overload
def __init__(self: 'Example[K, V]', x: Mapping[K, V]): ...
brazen jolt
#

oh

#

that's interesting

#

could I do type_extensions.Self[int, V] instead?

brazen jolt
soft matrix
#

Maybe when HKT is a thing

brazen jolt
#

what's HKT?

soft matrix
#

Higher kinded type vars

pastel egret
#

Itโ€™s a concept where youโ€™d be able to do like T[U, V] and then pass in a generic as the T typevar.

#

Not implemented in any type checkers, itโ€™d probably be quite difficult.

brazen jolt
#

ah, is that something that's already being worked on?

soft matrix
#

No

#

Lol

#

No one has a pep for it

trim tangle
soft matrix
#

So no one's gonna work on it

soft matrix
#

Like any subclass should be able to meet Example

trim tangle
#

I think classes should be final by default. If you want a class to be subclassable, you need to make an explicit decision

#

And then design your class so that it works with subclassing as expected

#

Subclassing is rarely "the best tool", so it shouldn't be a major design driver by default

devout barn
#

why is it that we have to make a enum to use a Literal[SENTINEL]
what I mean is :
This is invalid :

MISSING = object()

def foo(__t: AnyStr | Literal[MISSING] = MISSING, /):
    if __t is MISSING:
        return print("No value but that's okay!")
    return print(f"Got value: {__t}")

This is valid :

class Sentinels(Enum):
    MISSING = object()


def foo(__t: AnyStr | Literal[Sentinels.MISSING] = Sentinels.MISSING, /):
    if __t is Sentinels.MISSING:
        return print("No value but that's okay!")
    return print(f"Got value: {__t}")

Interestingly enough, doing

class Sentinels(Enum):
    MISSING = object()
MISSING = Sentinels.MISSING

Allows us to do Literal[MISSING], but why?

This is the warning I get from pyright when I don't use enums :

Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value
trim tangle
#

Only strings, bytes, numbers, enums

#

There has been some discussion/PEPs on sentinels, don't remember how it went exactly

devout barn
twilit badge
#

is there any good way to handle cases like these

class MyConnectionClass:
   def __init__(self):
       self.stream = Optional[AsyncTCPStream] = None

   async def connect(self) -> None:
       _, writer = await asyncio.open_connection(self.host, self.port)
       self.stream = AsyncTCPStream(writer)

   @staticmethod
   def with_stream(func: Callable[P, R]) -> Callable[P, R]:
       """Make sure that decorated method has 'stream' attribute that's not None.

       Only works for instance methods, behavior for class methods or static methods
       isn't defined!
       """
       @wraps(func)
       def wrapper(*a: P.args, **kw: P.kwargs) -> R:
           stream = getattr(a[0], "stream", None)
           if stream is None:
               raise ValueError("Can't run this function without initialized stream (forgot to run connect function?)")
           return func(*a, **kw)
       return wrapper

   @with_stream
   async def my_func(self):
       # Accessing self.stream here, even though I know it exists because of the decorator
       # doesn't work since it thinkgs it can still be None
#

I really don't want to have to use type ignores everywhere

soft matrix
#

I'd just type hint as AsyncTCPStream and remove the optional

#

If you always know that it should be valid

oblique urchin
trim tangle
#

Death by a thousand bikes ducky_bike

fallen fulcrum
#

how the tables have turned when the sentinels are the ones terminated

#

lol

fierce ridge
#

is it possible to "refine" a type var in a subclass of a generic?

# 
from myapp.orm import BaseModel

_Model = TypeVar('_Model', bound=BaseModel)

class HandlerMeta(Generic[_Model]):
    model: None | type[_Model] = None

class BaseHandler(Generic[None | _Model]):
    _meta: ClassVar[HandlerMeta[None | _Model]] = HandlerMeta(None)

class ThingHandler(Generic[_Model):
    _meta: ClassVar[HandlerMeta[_Model]] = HandlerMeta(BaseModel)
#

this None | _Model stuff doesn't work, but i don't know how else to explain what i'm trying to do

#

basically i don't want to / can't put None in the type var _Model, because in the ThingHandler subclass i want to restrict _Model to not be None

#

it also gets weird because i have a type[_Model] in there

#

i actually don't even know how to do this without the restriction

#

๐Ÿค”

_Model = TypeVar('_Model', bound=BaseModel)
_ModelType = TypeVar('_ModelType', bound=type[_Model])
_OptionalModel = TypeVar('_OptionalModel', None, BaseModel)
_OptionalModelType = TypeVar('_OptionalModelType', None, _ModelType)
soft matrix
#

you cant have only one constraint to a type var

fierce ridge
#

i just realized ๐Ÿ™‚

#

i feel like we discussed this in this channel before

#

i remember you didn't used to be able to do bound=Union[A, B]

#

it looks like you can't define type vars in terms of other type vars anyway

soft matrix
#

wdym?

fierce ridge
# 
_Model = TypeVar('_Model', bound=BaseModel)
_ModelType = TypeVar('_ModelType', bound=type[_Model])
_OptionalModel = TypeVar('_OptionalModel', None, BaseModel)
_OptionalModelType = TypeVar('_OptionalModelType', None, _ModelType)

main.py:11: error: Type variable "__main__._Model" is unbound
main.py:11: note: (Hint: Use "Generic[_Model]" or "Protocol[_Model]" base class to bind "_Model" inside a class)
main.py:11: note: (Hint: Use "_Model" in function signature to bind "_Model" inside a function)
main.py:13: error: Type variable "__main__._ModelType" is unbound
main.py:13: note: (Hint: Use "Generic[_ModelType]" or "Protocol[_ModelType]" base class to bind "_ModelType" inside a class)
main.py:13: note: (Hint: Use "_ModelType" in function signature to bind "_ModelType" inside a function)
Found 2 errors in 1 file (checked 1 source file)
#

its kind of hard to describe what i am trying to do, not sure if that helps

soft matrix
#

you might be looking for a type alias

fierce ridge
#

i'm not sure... here is a sketch of how my code is laid out

import attrs

# hypothetical
from myapp.models.base import BaseModel


@attrs.define
class HandlerMeta:
    model: type[BaseModel] | None = None
    requires_auth: bool = True
    ...  # more stuff here

class RequestHandler:
    _meta: HandlerMeta = HandlerMeta(None)
    ... # more stuff here

class ResourceHandler(RequestHandler):
    _meta: HandlerMeta = HandlerMeta(BaseModel)
    ... # more stuff here
#

so this RequestHandler is parameterized by "BaseModel | None"

#

but HandlerMeta is parameterized by "type[BaseModel] | None"

#

i'm not sure how to set up the generic types to enforce the constraint that the parameter is either "always None" or "some specific subclass of BaseModel", but with the added complexity that it's actually a type[BaseModel] in one case

#

it's also funky because ResourceHandler and BaseModel are both technically abstract classes, but that's a secondary concern

#

so it seemed pretty straightforward, but the type[BaseModel] is tripping me up

#

because type[_A] isn't valid if _A can be None... i think

soft matrix
# 
from dataclasses import dataclass
from typing import TypeVar, Generic

class BaseModel:
    foo: str


_Model = TypeVar('_Model', BaseModel, None)


@dataclass
class HandlerMeta(Generic[_Model]):
    model: type[_Model] | None = None
    requires_auth: bool = True
    ...  # more stuff here

class RequestHandler:
    _meta = HandlerMeta(type(None))
    ... # more stuff here


reveal_type(RequestHandler._meta)

class ResourceHandler(RequestHandler):
    _meta = HandlerMeta(BaseModel)


reveal_type(ResourceHandler._meta)
```this seems almost there
#

the first reveal_type is Unknown though

#

i think you do need constraints here though

fierce ridge
# 
main.py:26: error: ClassVar cannot contain type variables

didn't even know about this restriction

soft matrix
#

if you use type(None) you get the correct behaviour

#

but idk how to avoid needing that

fierce ridge
#

that's not bad tbh

#

i could try that, although wouldn't i need a literal NoneType then?

#

this seems like it should be possible

rotund flax
#

!d typing.TypeVar

rough sluiceBOT
#
typing.TypeVar 

class typing.TypeVar```
Type variable.

Usage:

```py
T = TypeVar('T')  # Can be anything
A = TypeVar('A', str, bytes)  # Must be str or bytes
```  Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function definitions. See [`Generic`](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") for more information on generic types. Generic functions work as follows...
oblique urchin
#

(I mean the restriction isn't a good idea. Generic ClassVars do make sense sometimes)

fierce ridge
#

i can try with pyright

oblique urchin
fierce ridge
#

what do you mean by that?

oblique urchin
#

class X(Generic[T]): attr: ClassVar[T]

#

^ this basically can't work, because there's only one X.attr

soft matrix
oblique urchin
#

But it does make sense if you have a subclass of X that does class Child(X[int]):

fierce ridge
#

yeah, i guess my case is like this

class Silly(Generic[T]):
    meta: ClassVar[Meta[type[T]]]

    def get_item(self) -> T: ...
oblique urchin
#

PEP 526 doesn't really discuss why they made that choice though

fierce ridge
#

seems like a pretty natural thing to want to do in the context of an ORM

#

is the github issue on the mypy repo? or typing?

oblique urchin
#

lol I always forget where I read these things. let me find it back. I think it was Tin Tvrtkovic

fierce ridge
#

Note that a ClassVar parameter cannot include any type variables, regardless of the level of nesting: ClassVar[T] and ClassVar[List[Set[T]]] are both invalid if T is a type variable.
weird restriction imo

#

i see the explanation, but it's the kind of thing where unless there's a technical / type-inference problem here, i don't see why you wouldn't let people do it

terse sky
#

it should be fine if the class itself is generic on T

fierce ridge
terse sky
#

yeah

#

pretty.... silly of me to not see that ๐Ÿ˜‰

#

but yeah, having things like ClassVar and Final as type constructors (or wahtever terminology you want to use) is pretty damn weird

noble pilot
#

Hey all. Messing with generics and have an issue where my test variable below infers to an Any type instead of the class. Assuming I'm doing it wrong. Basic example:

# 
T = TypeVar("T")
S = TypeVar("S")

class BaseFoo(Generic[T, S], ABC):
    foo: Any
    def __init__(self, item: T):
        self.item = item
        self.foo = 1

    @abstractmethod
    def test(self) -> S:
        ...

class Foo(BaseFoo[...]):

    def __init__(self, item: T):
        super(Foo, self).__init__(item)
        
    def test(self) -> S:
        return str(self.item)

test = Foo[int, str](1)
fierce ridge
#

as far as i know, Foo[int, str] is just Foo at runtime

#

is that actually Foo(BaseFoo[...])? is that ... even allowed?

#

basically removing the parameterization in the child class, not sure if that's what you had in mind

noble pilot
#

So that does work, but I was hoping I could keep the generics in as a poor man's mocking for testing

fierce ridge
#

well you've clearly constrained S here at least, to be str

#

but you can leave T free

# 
from abc import ABC, abstractmethod
from typing import Any, Generic, TypeVar

T = TypeVar("T")
S = TypeVar("S")

class BaseFoo(Generic[T, S], ABC):
    item: T
    value: Any

    def __init__(self, item: T):
        self.item = item
        self.value = 1

    @abstractmethod
    def test(self) -> S:
        ...

class Foo(BaseFoo[T, str]):
    def test(self) -> str:
        return str(self.item)

foo1: Foo[int]
foo2 = Foo(2)
foo1 = foo2
noble pilot
#

oh I see what you mean, because I've casted it to a type

#

but that wouldn't make Foo infer to as Any right?

fierce ridge
#

well str isn't casting as such

#

but yes it always returns a str

#

and you annotated test(self) -> S

#

which means (at least to my eye) that S must unify with str

noble pilot
#

that is true. Wasn't meant to illustrate that, so assume a 2nd param in the constructor
weird issue is when I instanciate the class pycharm infers it as Any type instead of the class

fierce ridge
#

https://mypy-play.net/?mypy=latest&python=3.10&gist=68eaaedba9a1bf422fe3388cdb9be02d

from abc import ABC, abstractmethod
from typing import Any, Generic, TypeVar

T = TypeVar("T")
S = TypeVar("S")

class BaseFoo(Generic[T, S], ABC):
    item: T
    value: Any

    def __init__(self, item: T):
        self.item = item
        self.value = 1

    @abstractmethod
    def test(self) -> S:
        ...

class Foo(BaseFoo[T, str]):
    def test(self) -> str:
        return str(self.item)

foo1: Foo[int]
foo2 = Foo(2)
foo1 = foo2
reveal_type(foo1)
reveal_type(foo2)
reveal_type(foo1.item)
reveal_type(foo2.item)

main.py:26: note: Revealed type is "__main__.Foo[builtins.int]"
main.py:27: note: Revealed type is "__main__.Foo[builtins.int*]"
main.py:28: note: Revealed type is "builtins.int*"
main.py:29: note: Revealed type is "builtins.int*"
mypy Playground

The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.

#

not sure what the asterisk means but

noble pilot
#

hmm mypy complaining about the ...

fierce ridge
#

what would you expect ... to do in a type parameter?

noble pilot
#

I thought I read that it copied the base class

#

heh

fierce ridge
#

ah, no

#

would be a nice feature if you could "pull down" the type parameters that way!

#

good feature request for mypy i suppose

noble pilot
#

anyway I wasn't using mypy, just in general for autocompletion

#

lame. Not sure if I'm doing something wrong or PyCharm limitation

fierce ridge
#

because Foo[int, str] isn't a valid way to instantiate a Foo and it doesn't know what to do, so it falls back to Any

#

Foo[int, str] is only valid for type annotations, not at runtime

noble pilot
#

oh

fierce ridge
#

python might even give you an error if you try that

noble pilot
#

wait, so am I thinking too much like C# here. I thought python supported that

fierce ridge
#

yeah, don't forget that the type hinting stuff is all pretty much "bolted-on" and has no effect on python runtime, although 3rd party libraries can introspect on and use the annotations at runtime if they wish to

noble pilot
#

argh. I wanted to be able to mock out some objects easily

#

(new to testing and seemed like a good idea)

fierce ridge
#

well what do you actually want to do here?

noble pilot
#

My overall design sucks anyway for this class. Trying to abstract out my observer pattern for keeping a cache up to date

fierce ridge
#

it sounds like you just want some kind of dependency injection

#

don't forget that python is not a pure OO language, and a lot of OO design patterns often aren't strictly necessary

#

or don't need to be formulated in such strict terms

noble pilot
#

whats our favorite pastebin site these days?

fierce ridge
#

!paste

noble pilot
#

I'll show you what I was trying

rough sluiceBOT
#

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

fierce ridge
#

we have our own

noble pilot
#

My goal was to try to remove as much coupling as possible but this is proving dumb heh

fierce ridge
#

what do you mean by coupling?

#

coupling between what?

#

this seems possibly over-abstracted tbh

noble pilot
#

discord objects or asyncpg specificalyl

#

yeah.... it is.. I was trying to make it so I could do unit tests with fake objects easier

#

and the entire item: Any really sucks too

fierce ridge
#

well... what are converters and fetchers

#

why is item: Any and not item: Item_T?

noble pilot
#

I figured a converter would be some object that I could take a primitive type and turn it into something (like a discord object)

fierce ridge
#

ah, fair enough

noble pilot
#

item: Any is a huge code smell. Some type of container. But my containers are different. I can't assume just a flat dict. For some, I might have Dict[int, Item] while others are Dict[int, List[Item]]

fierce ridge
#

๐Ÿค” self.item: Dict[int, List[Item_T]] = {}

#

yeahhhh that's not good

#

but i see why you did it

#

does item need to be accessible from the outside at all?

#

just define it as needed in subclasses

noble pilot
#

yeah item is the main accessor for it

fierce ridge
#

then the type of the container is a parameter for this cache thing!

noble pilot
#

the idea is in a cog, I could self.cache = MetarChannelCacheObject(blah, blah)
and self.cache.item[guild_id]

fierce ridge
#

this is parameterized by the entity type, the item type, and the item container type

#

weird imo, but that seems to be the way you need/want it

noble pilot
#

It was mainly for testing. I can just subclass with the types directly

fierce ridge
#

or is item always a Mapping[GuildId, Sequence[Item_T]]?

noble pilot
#

its not. Mapping[GuildId, Item_T] or Mapping[GuildId, List[Item_T]]

fierce ridge
#

what's the use case for having it be one or the other?

#

as opposed to specifically one

noble pilot
#

one is a scalar while the other is a collection

fierce ridge
#

ahhh ok

#

so you have one "thing", but that "thing" might either be a single thing or a collection of things

#

i wonder if this works in the parent class:

Mapping[GuildId, Sequence[Item_T] | Item_T]
#

then in subclasses you can specify either one that you need

#

you can also just write super(), it auto-magically inserts super(MetarChannelCache, self)

#

explicitly passing args to super() is only necessary if you are messing with the MRO (which you probably don't want to do)

noble pilot
#

ah ok

#

what do you think about the approach? Right now that listener logic is in my cog and its just distracting looking (and very similar in each cog)

fierce ridge
#

this is getting more into #software-architecture, but it's hard to say without seeing how it looks in the actual usage

#

it looks like you just need to invoke _notify at various moments, using some kind of constructed payload

noble pilot
#

This is pseudo code but something like

class Metar(AVWX, commands.Cog):
    def __init__(self, bot: AmeliaBot):
        super(Metar, self).__init__()
        self.bot = bot
        self.cfg: t.Dict[int, MetarDB] = {}
        self.metar_channels: = MetarChannelsCache(self.some_converter, self.some_fetcher)
        self.bot.pg.register_listener(self.metar_channels._notify)

    
        async def get_metar_channels(guild_id: int):
            return self.metar_channels.item[guild_id]

        async def add_metar_channels(channel):
            self.bot.pg.add_channel(channel)
#

At first I was going to just do the register_listener inside the cache object directly, but then it would need to know about my pg class which is a custom service class I use to write to my database

fierce ridge
#

this doesn't look bad to me

#

2 lines of boilerplate per class?

#

one to create the cache, the other to register it

noble pilot
#

yeah or however many things I want to cache for that cog. I figured keep the Cache class to a single item or purpose

fierce ridge
#

of course the listener method being "private" is weird

#

PgListenerCache.notify seems fine, not _notify

noble pilot
#

agreed

#

at first I was doing it inside that class... refactored it out but never renamed

fierce ridge
#

which could be several things per cog?

noble pilot
#

yeah, so this cog would have two. A config table for the guild and one that shows a 1:M relationship of the guild to TextChannels

fierce ridge
#

why two? i just see metar channels

noble pilot
#

I was just starting with the metar channels for now

#

But I do reference another table that is the parent that metar channels relates to

# 
CREATE TABLE MetarConfig
(
    id SERIAL NOT NULL PRIMARY KEY,
    created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
    guild_id BIGINT NOT NULL UNIQUE,
    restrict_channel BOOLEAN NOT NULL DEFAULT TRUE,
    delete_interval INT NOT NULL DEFAULT 5,


    CONSTRAINT fk_server_guild_id_metar_config_guild_id
        FOREIGN KEY (guild_id)
            REFERENCES Server(guild_id)
            ON DELETE CASCADE
);

CREATE UNIQUE INDEX metar_config_id_pkey on MetarConfig (id);
CREATE INDEX metar_config_guildid_key on MetarConfig (guild_id);

CREATE TRIGGER set_timestamp
        BEFORE UPDATE ON MetarConfig
        FOR EACH ROW
            EXECUTE PROCEDURE set_updated_at();

CREATE TRIGGER notify_metar_config_event
    AFTER INSERT OR UPDATE OR DELETE ON MetarConfig
        FOR EACH ROW EXECUTE PROCEDURE notify_event();

CREATE TABLE MetarChannel
(
    id SERIAL NOT NULL PRIMARY KEY,
    created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP,
    guild_id BIGINT NOT NULL,
    channel_id BIGINT NOT NULL,

    CONSTRAINT fk_metar_channel_server_id
        FOREIGN KEY (guild_id)
            REFERENCES Server(guild_id)
            ON DELETE CASCADE
);
CREATE UNIQUE INDEX metar_config_channel_id_pkey on MetarChannel (id);
CREATE INDEX metar_config_channel_guildid_key on MetarChannel (guild_id);

CREATE TRIGGER set_timestamp
        BEFORE UPDATE ON MetarChannel
        FOR EACH ROW
            EXECUTE PROCEDURE set_updated_at();

CREATE TRIGGER notify_metar_channels_event
    AFTER INSERT OR UPDATE OR DELETE ON MetarChannel
        FOR EACH ROW EXECUTE PROCEDURE notify_event();
#

full schema just for context

fierce ridge
#

so it still sounds not-that-bad

#

even if you had 5 individual caches on each cog, i don't know that i'd want to abstract this any further

noble pilot
#

design is fun, but frustrating haha

fierce ridge
#

also i don't think annotating assignments to self.foo work as you'd expect

noble pilot
#

I'm hoping ultimately that my cog in discord.py will be cleaner and contain command handlers rather than all this listen/notify logic

#

I'm going to have to abandon the generic approach I think and just learn traditional mocking for testing

#

I was reading though its not ideal to mock objects you don't particularly own

fierce ridge
#

i don't think you have to abandon it!

#

and i think it's the other way around: you don't want to mock things that you do own if you can avoid it

#

mocking is a backup strategy in case you can't set up actual test resources

#

e.g. some database connection or filesystem setup that you can't easily replicate

#

it's better to spin up a real postgres server than to mock out the postgres library, for example

#

what is the expected type of fetcher btw?

#

it seems like something that is possibly an implementation detail of each class, as opposed to something that belongs in its interface

noble pilot
#

the idea of fetcher is so that the cache class didn't see my pg service class

#

it would be just in the cog that would populate the cache

fierce ridge
#

hmmm.. but the cache is clearly coupled to postgres as it is

noble pilot
#

PgActions yeah which is an enum. I can just replace it as strings. But you are right the implementation itself assumes postgres and a very specific notify format

fierce ridge
#

i'm just looking at the implementation of MetarChannelCache and seeing how fetcher is used, i'm wondering if there is any consistency at all across PgListenerCache implementations in how it's used

#

that is: what is the expected interface for fetcher?

#

i see that you didn't annotate it beyond Callable, which is why i ask

noble pilot
#

that would depend on item
So returning either Item_T or List[Item_T]

fierce ridge
#

i see

noble pilot
#

my pg service class is a mess. picture a lot of methods like fetch_metar_config() and fetch_metar_channels()

#

not the most ideal

#

heh

fierce ridge
#

this is the kind of thing that imo should be easy to do but is starting to get into complicated territory with python. not sure what e.g. C# would do

#

yeah, well one thing at a time

noble pilot
#

I decided to make it without an ORM for giggles

#

and I found out it wasn't for giggles and that I very much hated life

fierce ridge
#

does Item_T have some common base parent class?

#

or can it really be anything in there?

noble pilot
#

It could be anything, but in reality they all have a FK reference to a "Server" table (which I should have called Guild in retrospect)

fierce ridge
#

but it's not like it all has some BaseItem parent class, right?

noble pilot
#

no. Not sure if thats a mistake or not but it doesn't right now

fierce ridge
#

fair enough

#

that's a matter of taste, i think

#

personally i like doing that, it keeps things grouped together even if just for organizational purposes

#

so wait, a fetcher returns an item or an entity?

#

are fetcher and converter inverses in some sense?

noble pilot
#

its essentially some type of entity/DTO

#

the converter I was just brainstorming. But the idea was "ok this class doesn't know about discord objects, and I don't want it to, but it would be nice to have cache items that were discord objects rather than primitives I would have to do lookups on"

fierce ridge
#

ah yeah, i've done things like that

#

i tend to think of my applications in "data layers"

#

so there is the database layer, the application layer, and the discord layer -- for example

noble pilot
#

yeah heh

#

I'm trying to treat discord very much like a frontend view

fierce ridge
#

and there is a flow through the application, database -> application and then application -> discord, at the boundaries

#

the small overhead of creating new class instances is worth it for the type safety and organizational / self-documenting benefit

noble pilot
#

basically because a lot of the code I'm planning to use on a webserver as well for the rest api interface that will lead into a dashboard

fierce ridge
#

that's a good approach

#

i've lately taken to using (abusing?) type aliases for things like this

noble pilot
#

I really like a lot of DDD concepts around naming and controlling the domain

#

I think I don't like DDD in general though haahha

fierce ridge
#

i haven't formally studied ddd but i have heard and seen some things that i like

# 
DatabaseFetcher: TypeAlias = Callable[[DatabaseRecord], MyAppThing]
DiscordConverter: TypeAlias = Callable[[MyAppThing], DiscordPayload]

that kind of thing

#

even if DiscordPayload: TypeAlias = Dict[str, Any]

noble pilot
#

yeah I like that

fierce ridge
#

that way you don't have to encode so much information in the variable names themselves too imo

noble pilot
#

and its descriptive

fierce ridge
#

so you can call it fetcher, but if you check the type you at least can see what kind of "fetcher" it is

#

and yes i love that aspect

noble pilot
#

well thanks for reaffirming I'm not completely off my rocker here. heh

fierce ridge
#

no not at all

noble pilot
#

I'm a hobbyist programmer so not the norm day-job kind of guy

#

design intrigues me but my kryptonite is I hate diagraming and just start diving into coding

fierce ridge
#

programming is basically my second career, i only learned it so i could do data science, but then spent more time on the programming than on the data stuff after a while ๐Ÿ˜› so now i am a software engineer, at least for the time being

#

i don't think there's a lot of benefit in "formal" diagramming, but i do like sketching things on a notepad

#

just arrows and blobs, not any formal uml stuff

noble pilot
#

ok going to go back to some hacking

#

or watching Picard

#

heh

fierce ridge
#

hah enjoy

noble pilot
#

Thank you so much for the help

fierce ridge
#

๐Ÿ‘

hearty shell
#

Would there be any way of making a "real" type annotation to differentiate between type.__call__(int) and type(int)?

#

I can't even figure out what the emulated python code would be for that x)

fierce ridge
hearty shell
#

No I mean actually annotating it, making a stub for those two calls

#

Like in typeshed it is just def __call__(self, *args: Any, **kwds: Any) -> Any: ...

pastel egret
#

No, because there's no real difference between x.__call__(...) and x(...), the latter does the former.

hearty shell
#

but is that for type.__call__ or type(), is it even possible to differentitate

hearty shell
pastel egret
#

Like all magic methods it skips the instance lookup, descriptors etc.

hearty shell
#

Okay maybe this was a bit of an X Y problem, give me a sec

pastel egret
#

Ahh. I think in this specific case, there's an exception in the interpreter perhaps? type is very special.

hearty shell
#

Basically I am playing with making a small type checker as a toy project, i am just at the step of doing resolving builtin type annotations like int, but I didnt want to just do class A: ... A() is of type A, I wanted to do the whole lookup, like Metaclass.__call__(int) -> int.__new__() -> int() is of type int

pastel egret
#

Okay it's that you need to do type.__call__(type, 45) - that does type(45).

#

Since type is both the metaclass, and an instance of itself (IE a class), regular binding rules break.

hearty shell
#

I think that would be the other case no?

#

I am trying to just do int()

pastel egret
#

Yeah that's the 1 arg case.

#

type.__call__(int, "12") calls __new__, it's an unbound method.

hearty shell
#

Ohh, I see

hearty shell
pastel egret
#

It is yeah. You'll definitely need special cases here I think.

#

Actually this is going to make sense. Any attribute on type is going to look in the "instance" dict first, and always succeed - since the "class" dict is the exact same one, it's never going to get to checking that. So no __get__ is called.

hearty shell
#

That's true, although that would be one up the chain, when I do int() the "instance" in that case is int and the class is type no?

#

the type.__call__ would just be what the type checker would look for, I havent actually thought about what I would do if it had to type check for type.__call__ itself

#

I will make a note of that if I ever get to checking that ^^

#

Thanks for clearing up the type class mess ๐Ÿ˜†

errant silo
#

why is this typehint, not working?

class MyClass(Generic[T]):
    def __init__(self, value: Sequence[T]): ...
    def get(self) -> T: ...
acoustic thicket
#

whats the error

errant silo
#

nvm sorry, didn't save the file lemon_sweat

summer root
#

hey typing fans. have I misconfigured my vscode? I'm happy it has type checking in the 2nd case here, but confused why it doesn't have it in the first:

unknown.png
soft matrix
#

Probably cause it's only type checking the second cause it has annotations?

summer root
#

ah you know what? the default language server wasn't good enough. just installed pylance and

unknown.png
soft matrix
#

Or you need to save

rustic gull
summer root
#

yes. working now with vscode and this set of extensions

unknown.png
rustic gull
#

Thanks..looks good

grave fjord
#

Why is a cyclic dataclass ok but a cyclic TypedDict forbidden?

trim tangle
#

maybe a mypy implementation limitation/design limitation

grave fjord
#

Yeah of course it does xD

rustic gull
#

i know its not necessary but you typehint self as self: ClassName?

trim tangle
#

but, well, you don't need to type it

rustic gull
#

yeah ik

rustic gull
upbeat wadi
#

typing_extensions.Self, typing.Self will be a thing in 3.11

rustic gull
#

yeah ik that

#

!pep 673

rough sluiceBOT
#
**PEP 673 - Self Type**

Link

Status

Accepted

Python-Version

3.11

Created

10-Nov-2021

Type

Standards Track

upbeat wadi
#

Im not sure what you're asking then

rustic gull
#

i just wanted to know how it was annotated correctly

trim tangle
#

the correct way is to not annotate self

rustic gull
#

okie

trim tangle
#

now, sometimes you do need to annotate self

#

when you want to constrain a generic, e.g.:

# 
class Foo(Generic[A]):
    @overload
    def __init__(self: 'Foo[None]') -> None: ...

    @overload
    def __init__(self: 'Foo[B]', value: B) -> None: ...

    def __init__(self, value=None) -> None:
        self._value = value
twin field
#

Does anybody here have a solution for type hinting a "Variadic Cycle": that is *args which behaves as a set of related fields that are processed as a batch, but not all the same type:


import itertools

def foo(*args: Repeat[int, str]) -> str:
    return "".join([a*b for a,b in itertools.pairwise(args)])


print(foo(2, "a")) # VALID
print(foo(2, "a", 3, "b")) # VALID
print(foo(2, 3, "a")) # INVALID

(Repeat is not something I know to exist, but would be useful for type hinting existing methods, I think)

terse sky
#

I think you'd probably just want to transform that to a List[Tuple[int, str]] at an earlier stage

twin field
#

This is a toy example, but what I'm trying to work on is long standing code that the signature really does need to stay the same for

oblique urchin
terse sky
#

that's a pretty big ask

#

You could do it in C++ but it would be hideous

#

and probably in D, and less hideous. Very few languages you could do this in.

hasty hull
#

Is there an easy way to show the difference between multiple overloads?

#

I've got to use a function that takes a lot of arguments making it difficult to figure out what the actual difference is

terse sky
#

if you have existing code, what I'd suggest is refactoring it so that you have outer function

hasty hull
#

Yeah it's not my code unfortunately

#

pandas.read_csv

terse sky
#

with a signature like this:

def reorgnize(*args: Union[int, str]) -> List[Tuple[int, str]]:
    ...
#

I think that's probably the best you can do

twin field
#

A signature of Union does get part of the way, but does not fully encapsulate the cyclic nature of the argument relationship

terse sky
#

Actually, that's not true

#

If you are willing to limit the number of args passed in this way

#

....

#

you see where this is going

twin field
#

Think plt.plot(x1, y1, fmt1, x2, y2, fmt2)

terse sky
# 
@overload
def reorganize(arg1: int, arg2: str) -> List[Tuple[int, str]]:
   ...

@overload
def reorganize(arg1: int, arg2: str, arg3: int, arg4: str) -> List[Tuple[int, str]]:
   ...
...
#

yeah, I mean APIs designed by quants are hard ๐Ÿ™‚

#

Sure, but if you control the code that calls plt, then you can wrap it in an argument that takes a List[Tuple[int, str]]

#

I assume that you are not annotating plt.plot after all

#

if you really absolutely cannot change the way the code works at runtime, and you absolutely want to have the best possible static type annotation

#

then your only option that I can see is to write a python script that generates @overload reorganize up to some limit

#

and implement reorganize itself (which is a one liner)

twin field
#

That idea occured to me, but was hoping to avoid such things

terse sky
#

Yeah, I mean code like this is everywhere

twin field
#

Does anyone know if PEP-646 helps here?

hearty shell
#

I just tried to see if that was possible with Pyright and I think I broke it x)

terse sky
#

This came up a few days ago

#

Rust, Scala, etc

hearty shell
terse sky
#

it's called variadic generics so I would expect that it would help

#

but you'd still need to do some non-trivial metaprogramming on top of having variadics

#

I'd be really surprised if any static type checker supports that

oblique urchin
hearty shell
#

Yeah I think it is not possible, you would need to be able to unpack two tuples in the expression which doesnt seem to be allowed

#

I tried this *args: Unpack[tuple[Unpack[tuple[str, ...]], Unpack[tuple[int, ...]]]]
If you somehow get it to work do tell xD

trim tangle
#

typing aside, I think that's a pretty strange API ๐Ÿ‘€

#

I would expect taking a list of tuples or something.

pastel egret
#

It's because matplotlib.pyplot borrows a lot of its api design from MATLAB, where this sort of repeated argument pairs are common.

trim tangle
#

well

#

borrowing an API from another language often ends up a mistake ๐Ÿ™‚

trim tangle
#

stuff like: "this function behaves like so because the C function with the same name does this as well!"

brisk heart
#

how do I annotate an abstract attribute that returns a callable which may or may not require self?

# 
class Parent(abc.ABC):
    # make this abstract:
    getter: typing.Callable[[int], str]

class AttrChild(Parent):
    def __init__(self, getter: typing.Callable[[int], str]):
        self.getter

class MethodChild(Parent):
    def getter(self, value: int) -> str:
        return str(value)
#

mypy doesn't like attributes annotated as callables unless they're optional so that's also a problem

#

I use pyright but wanna keep some mypy compatibility

trim tangle
brisk heart
#

in my code I already use a typealias

#

will a protocol change the behavior?

trim tangle
#

yes

brisk heart
#

funky

brisk heart
trim tangle
#

๐Ÿคท much more people use mypy in their CI builds than pyright

#

so if you're making a library, it makes sense to check that it plays ok with mypy

brisk heart
#

I use both mypy and pyright but only pyright with strict

brisk heart
brisk heart
#

hope it doesn't break everything

dire bobcat
# 
 /home/Caeden/Github/discii/discii/embed.py:92:35 - error: Argument of type "str | None" cannot be assigned to parameter "name" of type "str" in function "set_author"
    Type "str | None" cannot be assigned to type "str"
      Type "None" cannot be assigned to type "str" (reportGeneralTypeIssues)
#

fields: Optional[List[Dict[str, Optional[Union[str, bool]]]]]

hearty shell
#

Just check if the field is None or not before assigning

#

Unless you deal with that case inside the set_author method, in that case your annotation should be str | None and not just str

#

or Optional[str]

acoustic thicket
#

imo it doesnt make sense for functions to accept Optional[T] if theyre only going to be using T
like, functions that are just

def f(x: T | None):
    if x is None:
        return None
    ...
trim tangle
#

yeah, it is a bit strange

#

to be fair, Python makes it pretty hard to apply a function to an optional value only if it is not None

#

None if x is None else f(x) ๐Ÿ˜”

#

the names are usually longer, and it might be in a bigger expression.

#

flat-mapping is even worse

#

as you need := or just creating a bunch of variables you didn't have before

acoustic thicket
dire bobcat
#

okay so this is going to require some backstory

# 
fields: Optional[List[Dict[str, Optional[Union[str, bool]]]]] = payload.get(
            "fields"
        )
#

and then im calling embed.add_field(name=fields.get("name")

# 
def add_field(
        self,
        *,
        name: str = None,
        value: str = None,
        inline: bool = False,
    ) -> None:
# 
 /home/Caeden/Github/discii/discii/embed.py:99:26 - error: Argument of type "str | bool | None" cannot be assigned to parameter "name" of type "str | None" in function "add_field"
    Type "str | bool | None" cannot be assigned to type "str | None"
      Type "bool" cannot be assigned to type "str | None"
        "bool" is incompatible with "str"
        Type cannot be assigned to type "None" (reportGeneralTypeIssues)
hearty shell
dire bobcat
acoustic thicket
#

the type of fields looks incredibly complicated

hearty shell
#

but you give None as a default value

dire bobcat
dire bobcat
acoustic thicket
#

if its an Optional[list[...]] how does it have a .get

hearty shell
# 
def add_field(
        self,
        *,
        name: Optional[str] = None,
        value: Optional[str] = None,
        inline: bool = False,
    ) -> None:

def add_field(
        self,
        *,
        name: str = "",
        value: str = "",
        inline: bool = False,
    ) -> None:
#

Either of these

dire bobcat
acoustic thicket
#

lists dont have .get either

dire bobcat
#

it was in a list comp

dire bobcat
hearty shell
#

Yeah that should also be an error, slightly confused where fields goes from being a list to being a dictionary

acoustic thicket
#

i think you actually want a TypedDict, not a dict[str, str | bool | None]

dire bobcat
acoustic thicket
#

calling .get("name") on dict[str, str | bool | None] gives you a str | bool | None, and the function expects just str
so youd need more if-elses to get the str | bool | None down to just str

dire bobcat
#

i need to add checks?

#

my list comp wouldnt work

hearty shell
#

Could you show more code?

dire bobcat
#

type: ignore

dire bobcat
acoustic thicket
# dire bobcat a what

if you have a known set of string keys, and each key is associated with a value of a known type, you can use a typeddict

class Schema(TypedDict):
    name: str
    other_key: bool
    ...
dire bobcat
hearty shell
#

Where is this list comprehension? There is stuff happening outside of what you provided

acoustic thicket
#

fields would then have to be made a fields: Optional[list[SchemaName]], and field["name"] would be just str

dire bobcat
#

when i do fields: Optional[List[EmbedField]]

acoustic thicket
dire bobcat
hearty shell
#

fields.get or field.get? Where do you take the list items out to be able to use get?

hearty shell
dire bobcat
#

no

dire bobcat
#

my pre-commit is still horrid

hearty shell
trim tangle
hearty shell
#

Is there a way to tell mypy/pyright that a function still follows the semantics of another function? I am not sure how either of them check for the "dataclassnes" of a dataclass

from functools import partial
from dataclasses import dataclass, field


define = dataclass(slots=True)

@define
class Context:
    name: str
    table = field(default_factory=dict)
    
    
Context("sdfsdf")

main.py:13: error: Too many arguments for "Context"
oblique urchin
#

and with pyright you can use dataclass_transform in this case

hearty shell
#

Actually it is a bit weird...

from dataclasses import dataclass, field

from typing import Any, Literal, NewType, Optional, Callable, TYPE_CHECKING

if TYPE_CHECKING:
    define = dataclass
else:
    define = dataclass(slots=True)

@define
class Context:
    name: str
    table: dict[str, int] = field(default_factory=dict)
    
    
Context("foo")

This works fine on pyright, but mypy seems to check for the name "dataclass"

#

But if you from dataclasses import dataclass as define and then type ignore it works for mypy?

#

which means that for both of them to work in a module you have to do

from dataclasses import dataclass as define, field
from typing import TYPE_CHECKING

if not TYPE_CHECKING:
    define = define(slots=True) # type: ignore

@define
class Context:
    name: str
    table: dict[str, int] = field(default_factory=dict)
    
    
Context("foo")
#

or just not rename it

hearty shell
#

Is there a way to get an invariant version of Type?

from typing import TypeVar, Type

T = TypeVar('T', covariant=False)

InvType = Type[T]

class BaseOne: ...
class BaseTwo: ...

class SubOne(BaseOne): ...
class SubTwo(BaseTwo): ...

def baz(a: InvType[BaseOne] | InvType[BaseTwo]): ...

baz(BaseOne)
baz(BaseTwo)

baz(SubOne)
baz(SubTwo)
#

so that the last two options fail?

#

Also on that note, general question, why is class SubOne considered a subtype of class BaseOne?

soft matrix
#

you should probably just be using

#

!d typing.final

rough sluiceBOT
#
typing.final 

@typing.final```
A decorator to indicate to type checkers that the decorated method cannot be overridden, and the decorated class cannot be subclassed. For example...
soft matrix
#

all your problems will disappear

hearty shell
#

my question is mostly around the type stuff itself

hearty shell
hearty shell
acoustic thicket
trim tangle
#

Since e.g. Optional[None] is kind of strange

acoustic thicket
#

close enough

mortal moss
#

!d def

#

!d print

#

!d print

rough sluiceBOT
#
print 

print(*objects, sep=' ', end='\n', file=sys.stdout, flush=False)```
Print *objects* to the text stream *file*, separated by *sep* and followed by *end*. *sep*, *end*, *file*, and *flush*, if present, must be given as keyword arguments.

All non-keyword arguments are converted to strings like [`str()`](https://docs.python.org/3/library/stdtypes.html#str "str") does and written to the stream, separated by *sep* and followed by *end*. Both *sep* and *end* must be strings; they can also be `None`, which means to use the default values. If no *objects* are given, [`print()`](https://docs.python.org/3/library/functions.html#print "print") will just write *end*.

The *file* argument must be an object with a `write(string)` method; if it is not present or `None`, [`sys.stdout`](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") will be used. Since printed arguments are converted to text strings, [`print()`](https://docs.python.org/3/library/functions.html#print "print") cannot be used with binary mode file objects. For these, use `file.write(...)` instead.
Similar names: 2to3fixer.print
grave fjord
#

it's weird print recommends file.write and not io.TextIOWrapper(file)

void panther
#

does it care about anything other than the write method and flush if it's ran with it?

grave fjord
#

That would be a funny @override

unreal plaza
#

How do I reference any Tkinter widget? They all derive from a Misc class at some point

rustic gull
#

Do the companies that actually use type checkers turn off asserts when they're running code in production, like is that a normal practice? I find I sometimes have to assert something when it's actually impossible to help the type checker, but I also know that slows things down.

trim tangle
#

If you have hundreds/thousands of these asserts to hack around types, your design might be problematic

#

moreover, it would be very strange to check some mission-critical invariant only in development and then just don't bother with it in production

#

if an assert fails, it means that we've broken something important and we should not continue

#

"Hmm... at this point in launch_missiles the admin key must not be None but it is. Whatever, let's proceed with the transaction!"

boreal ingot
#

in that situation is using a structure that can be bypassed with a command line flag good security ๐Ÿ™ƒ

trim tangle
#

I guess that is true

boreal ingot
#

"this missile launch code needs to run at peek efficiency! let me turn off assert"

trim tangle
#

you shouldn't rely on asserts for verification, perhaps

#

still, if your performance bottleneck is type assertions, you have bigger problems

warped lagoon
#

any appropriate type annotation for an is_iterable function?

def is_iterable(item):
    try:
        iter(item)
        return True

    except TypeError:
        return False

    # return isinstance(item, Iterable)  # shorthand```
I don't want it to lose concrete types (i.e. get erased to `Any`)
of course I could keep on using `isinstance` but it gets repetitive
trim tangle
warped lagoon
#

well, what should the type of item be?

trim tangle
#

object?

warped lagoon
#

still, you'd lose the T needed in Iterable[T]

trim tangle
#

where would you get the T from?

brisk hedge
#

A bit of a generic question:
Suppose I have a function foo that accepts type[A | B | ...] | X | Y | ... as input, i.e. "one of these types or one of these objects", and returns type[A | B | ... | X | Y | ...], i.e. "the normalized types of the inputs", how could I express using generic types that the input & output are related? (i.e. type[A] -> type[A], and X -> type[X]?

warped lagoon
#

say you have ```python
item = [13, 25, 42]

if is_iterable(item):
reveal_type(item) # Iterable[int]

else:
... # TypeGuard does not support negations so```

trim tangle
#

in that case you don't need is_iterable, you already know it's iterable

#

If you have some arbitrary object, if iter succeeds on it, it can be an iterable of anything at all

#

Like, Foo[Bar] could be an Iterable[Bar], but it also could be Iterable[tuple[int, Bar]] or Iterable[Baz]

warped lagoon
#

actually, fair
I never really use it for type-hinting, rather for type-checking to ensure an iterable is provided (which is already expected)
so I can simply take an Iterable (in type hints) and then raise an error if anything

trim tangle
brisk hedge
trim tangle
#

yeah screw typing tbh

warped lagoon
#

huh, you're passing in many things?

brisk hedge
#

uh, here's the gist
foo(*args: type[A | ...] | X | ...) is a decorator that wraps a callable (Union[Unpack[*args converted to types]]) -> SomeT
so unsurprisingly you can't type it

#

unless we had type-level functions :>

trim tangle
#

yeah we can't have nice things

#

you could make a ladder of overloads, like map and zip and asyncio.gather do

brisk hedge
#

in concrete terms it's

@foo(int, bytes, None, Ellipsis)
def thing(x: int | bytes | EllipsisType | NoneType): ...
# yes I know None is allowed as an annotation but
#

anyways, overloads are going to be pretty unwieldy for this because args can be any subtype of type[A | ...] | X | ...

#

Unless I write them all out by hand :>

#

cursed stub file time?

#

I'll probably just give up and loosen some of the generic relationships

#

let the users figure it out from docs

fierce ridge
#

dare i ask what this is for

#

you can probably do this in idris 2 ๐Ÿ˜›

trim tangle
#

yeah talk to your supervisor about potentially rewriting the python app in Idris 2 /s

terse sky
#

Interesting. I'm trying to think whether this is possible in C++.

#

You'd need to be able to take an argument that's either a type, or a value, if I understand correctly.

#

I don't immediately see how to do that

trim tangle
#

@brisk hedge

@foo(int) | bytes | None | Ellipsis
#

then you can just overload a single | operator (or a method like aber)

terse sky
#

Actually, you pretty much can do it in C++

#

You can always lift a value into a type

# 
template <auto T> struct lifted_value {};
trim tangle
terse sky
#

and then you could have a metafunction like

template <class T>
sturct normalize { using type = T; };

template <auto T>
struct normalize<lifted_value<T>> { using type = decltype(T); };
summer horizon
#

What is the type hinting args for numpy.ndarray?

rain sorrel
summer horizon
#

Oof okay

covert dagger
summer horizon
#

Oh thanks

#

Never knew that existed

acoustic thicket
#

variadic generics when ๐Ÿฅบ

trim tangle
#

PEP 646 was accepted

acoustic thicket
#

mm

#

3.11 when ๐Ÿฅบ*

trim tangle
acoustic thicket
#

๐Ÿ˜ฉ

oblique urchin
#

what's wrong with typing-extensions?

trim tangle
#

I wish more things were like typing_extensions. Like an external dataclasses with kw_only that can be used in 3.7

#

I guess attrs exists. But that's a gorilla for the banana

#

not quite a drop-in replacement I suppose

covert dagger
#

it can be almost a drop in replacement I think, it just has slightly worse names for all the features, so have to rename them while importing

#

probably also needs dataclass_transform for proper typing support

rain sorrel
covert dagger
#

there is a mypy plugin for some dtypes, but NDArray can be imported directly and used without the plugin, pyright accepts it too

#

though looks like np.ndarray also has __class_getitem__ now

#

but it has more cumbersome way to express the same thing as NDArray

terse sky
#

will be nice for typing extensions to be merged

#

I was really surprised that I had to use it to start with, to query really basic things about types

oblique urchin
terse sky
#

that's good

#

looks like all I'm using is this

from typing_inspect import is_generic_type, is_union_type, get_args, get_origin
#

I have a recursive function that looks like this

def dataclass_from_dict(return_type: Type[T], data) -> T:
oblique urchin
#

that's a different library

#

typing-inspect vs typing-extensions

terse sky
#

ah damn you're right, I misread

oblique urchin
#

get_origin and get_args are in the stdlib now though (as of 3.9 I think)

terse sky
#

that's good

#

hopefully is_generic_type and is_union_type will be in eventually

oblique urchin
#

there hasn't been much movement in that direction. I think most of those things you can get with just isinstance checks now

#

I'd be open to adding more introspection helpers to the stdlib though if they prove useful

terse sky
#

i thought that many isinstance checks for these things didn't work

#

like, they used to, but then they stopped working

#

as a result of a change in the implementation

#

maybe now they are working again? idk

oblique urchin
#

PEP 655 (Required/NotRequired) and 675 (LiteralString) have been accepted!

buoyant swift
#

๐Ÿ‘€๐ŸŽ‰

glass crystal
#

How do you do type hint for __new__ and __init__? Specifically I have:

        """Create a new ClientWrapper."""
        # Try to get the name of the client
        try:
            name = socket.recv(2**12).decode().strip()
        except Exception:
            return None

        # Create the object
        rtn = super(ClientWrapper, cls).__new__(cls)
        rtn.name = name

        # Return the client if the name was received
        return rtn``` And I want to add a return type for `__new__`, as well as type hint `cls`. Similarly, I want to do the same with `__init__` and type hint `self` in my methods.
trim tangle
#

You should not annotate cls or self

fierce ridge
#

!pep 665

rough sluiceBOT
#
**PEP 665 - A file format to list Python dependencies for reproducibility of an application**

Link

Status

Rejected

Created

29-Jul-2021

Type

Standards Track

fierce ridge
#

__new__ returning None seems very questionable

fierce ridge
rough sluiceBOT
#
**PEP 675 - Arbitrary Literal String Type**

Link

Status

Draft

Python-Version

3.11

Created

30-Nov-2021

Type

Standards Track

trim tangle
#

Yeah, I would just make a classmethod or a free function

glass crystal
trim tangle
glass crystal
trim tangle
#

There is not that much point to use type hints if you're not using a tool that checks them, like mypy or pyright.

#

You should also look into editor integration. (what is your editor?)

glass crystal
#

Missing type annotation for self in methodflake8(ANN101

glass crystal
rustic gull
#

I think it's pretty helpful in staying organized and remembering what you're looking at, not getting confused about what is supposed to happen. Just at the scatter brained level i think type hinting is pretty useful, although maybe not strict type hinting.

oblique urchin
glass crystal
#

I believe so

oblique urchin
#

Instead use a type checker (mypy or pyright) and turn on strict settings that tell you about missing annotations

glass crystal
#

It was recommended in the game jam dev-requirements here a while back

trim tangle
# glass crystal vscode

Then:

  1. Install the Pylance extension
  2. Go to Settings -> search for "type checking" -> set to basic from off
#

now you should get completion, error reporting, jump-to-definition etc. based on type hints

glass crystal
#

Right now I have flake8, flake8-bandit, flake8-docstrings installed. Are any of those ones I shouldnt use?

#

And flake8-annotations

trim tangle
#

what do you want to use them for?

#

i.e. what is your goal?

glass crystal
#

Honestly no idea

#

Really I just wanted to start being more professional with my code

trim tangle
#

In some projects I've seen stuff that maybe makes sense as an informal note, but doesn't play well with type checkers, like (int, str).

rustic gull
#

I find strict type linting to be extremely annoying. I feel like I'm clearing red lines to clear and cause more red lines to appear and disappear in a never ending spiral towards where -- I forget because im trying to remove red lines.

I feel that way mostly in Javascript, TypeScript.
I understand the value of it... But it can be so annoying.

trim tangle
#

Yeah, I'm still not quite sure whether typing is a waste of time or not.

#

At least in Python, that is. I like TypeScript quite a lot.

rustic gull
#

I think going over my code and putting type hints into it has led to me discovering problems i was not seeing plenty of times. I like being able to not do it if i dont need it.

#

But sometimes... you might need it

#

In JS, you need something to hold the monstrosity of disaster together i think

#

I would like TS more if i did it more

glass crystal
#

What about just base flake8 for linting extra whitespace or stuff like that?

rustic gull
oblique urchin
glass crystal
#

It seems like standardizing stuff like lines between methods or checking that you dont have random whitespace characters might be good

oblique urchin
#

(disclosure: I'm a maintainer)

glass crystal
#

What does it do?

rustic gull
#

it basically is an opinionated formatter of your code so you dont have to think about what the right way to format it is.

oblique urchin
#

it formats your code for you, so you run black your_file.py and it fixes a bunch of formatting

rustic gull
#

as a result everyone does it the same

#

i dont use black religiously but i gotta say its wonderful when you get a file where the person who wrote it was just... on crack

trim tangle
rustic gull
#

I should do that sometime, i have a few things that im not sure if they are good or not.

glass crystal
#

If I did do something not up to standard

rustic gull
#

Im pretty consistent about format convention, its more that i sometimes question my design implementations.... what problems they introduce and how to improve them.

trim tangle
glass crystal
#

Just want to make sure I am doing that all right

#

Because I am completely self taught in python

#

So Ive never had any top down authority teach / enforce formatting rules

rustic gull
#

the world of programmers will judge you sufficiently enough if you share enough code lol.

trim tangle
#

There's more to programming than just formatting, though.

rustic gull
#

yeah way more.

glass crystal
#

Yeah I know that for sure

#

Ive just gotten to the point where I feel like I know enough that I can start focusing on the more minor details

#

Like formatting

rustic gull
#

I think formatting is important at a level where you have put some effort into consistency and convention. Sometimes there are multiple ways of formatting clearly based on what you have to express in the code and how much room it takes, etc.

#

But it is important to be consistent with the rest of your code

#

I'm gonna try install mypy into pycharm and see what i think

#

the official one is no good?

unknown.png
hearty shell
#

Last time I tried they were both awful

#

I ended up just using the cli

rustic gull
#

mm

hearty shell
#

Maybe they are better now, but they were both slow and inconstant, I would ask it to check directory or check file, it would give no indication if it was doing anything and sometimes it would think there were no issues but only because it had forgotten to clean the cash

soft matrix
#

Neither are good

#

Type checking inside of pycharm is an awful experience

rustic gull
#

thanks for the warning. is it something you need to just remember to run before doing the runtime, like as if you had a compile step but its just a checker?

soft matrix
#

You should either switch to something with better support for it or just run the cli yourself as Mathias mentioned

rustic gull
#

right thanks

#

to be honest, i think it will be less irritating if its not interrupting my coding to underline things.

#

just give it to me all at once and leave me alone otherwise sounds good

devout barn
#

is hinting with Any and object the same thing?

hearty shell
#

Nope, object is just the base class of all classes, and it implements very few methods, so if you expect and object for a given function, there is very little you can do with it since object doest implement "anything"

#

Any on the other hand allows everything

# 
a: Any

a.sdfjl
a.floor()
a.lower()
a.foo()
#

as far as the type checker is concerned

#

Any could implement any of those, so it allows it

devout barn
#

Oh that was very clear, thank you for the explanation!

rustic gull
#

def f() -> Union[list[dict[tuple[int, str], set[int]]], tuple[str, list[str]]]: when type hinting goes wrong.

oblique urchin
#

nah that's when your design goes wrong

#

the type hints are just the messenger

rustic gull
#

lol that is from the mypy docs

oblique urchin
#

touche

rustic gull
#

to explain that you could alias that if you wanted

rustic gull
#

I just went over the entire mypy docs and im pretty excited about this. Ive been doing type hinting for a while without running a checker to stay organized when im doing something that's confusing me a little to get right..

#

But i think this will step it up a level without irritating me the way a ts linter would.

terse sky
#

I mean types can get very complicated, that is life
Though, keep in mind that having anonymous rather than named union types, as python does

#

exacerbates this

#

If you wrote tuple[list[dict[tuple[int, str], set[int]...., that would also be bad but the solution there is much simpler

# 
@dataclass
class Something
    first_thing: list[dict[tuple[int, str], set[int]]]
    second_thing: tuple[str, list[str]]]
#

or something like that

#

and obviously, you can apply that solution again, to any of those nested tuples

#

that and a couple aliases and it'll clean up pretty quick

#

but unfortunately in python, the Tuple -> Dataclass transformation doesn't really have an analogue for Union

trim tangle
#

Discriminated union when ๐Ÿ‘‰ ๐Ÿ‘ˆ

soft matrix
#

whats that?

trim tangle
#

aka sum type aka Rust enum aka variant record aka tagged union aka algebraic data types...

soft matrix
#

oh rust enums

#

yeah theyd be very nice

rustic gull
#

how are rust enums like?

buoyant swift
#

they're cool

terse sky
#

notably their "fields" are named

#

the same way that a dataclass' fields are named, which is what makes it different from a Tuple

rustic gull
#

right

#

and some other things like uh... the comparison implementation and... i think there are other ones

#

mutability?

terse sky
#

another headache with Union is that nothign is really carrying around the "tag" to tell you which thing is being stored in it. that has a couple of implications:
a) you can't have two different "possibilities" of the same type
b) More importantly, you have a real problem when you have two different static types in a python Union, with the same runtime type

rustic gull
#

I dont use dataclasses and namedtuples very much tbh.

terse sky
#

you should

rustic gull
#

possibly so

terse sky
#

It's an incredibly easy way to make your code easier to read, benefit from the IDE, stronger typing, better auto completion

#

avoid many trivial errors

trim tangle
# rustic gull how are rust enums like?

A "Rust enum" or more commonly "algebraic data type" lets you pack a fixed number of variants into a single type. For example:

enum Instruction {
  Goto(u16),
  Add {src: Register, dest: Register),
  Sub {src: Register, dest: Register},
  Halt
}

Then you can match on this value:

fn execute_step(&mut cpu: CPUState, instruction: Instruction) {
  match instruction {
    Halt => cpu.halt_and_catch_fire(),
    Add {src, dest} => cpu.set(dest, cpu.get(dest) + cpu.get(src)),
    Sub {src, dest} => cpu.set(dest, cpu.get(dest) - cpu.get(src)),
    Goto(pos) => cpu.set(Register::InstructionPointer, pos),
  }
}
rustic gull
terse sky
#

yeah, it depends on context. In your example above, as soon as things are getting nasty and nested it's usually a sign that it's time for a dataclass. or just for general clarity.

#

In Kotlin to give an example, they actually sort of had tuples, and removed them once they were confident in their data classes

#

their argument was basically that while tuple can sometimes feel more concise, dataclasses basically always result in more readable code, and that's more important

#

You could pretty easily make an argument that there's no real reason to use tuple outside of genreic code (e.g. zip)

rustic gull
#

isn't an important part of the tuple the immutability tho? I suppose you could make an immutable dataclass tho.

terse sky
#

yeah, frozen=True

soft matrix
#

or just use a NamedTuple

terse sky
#

i wouldn't personally say that it's an important part

#

that ship sailed a long time ago in python

brisk heart
rustic gull
#

I dont really care for named tuples. I think it was a needed functionality that now has better collections to do it. for me anyway.

terse sky
#

Yeah, there's almost no reason to used named tuples now that I'm aware of

#

aside from certain kinds of performance concerns

soft matrix
#

yeah i think thats literally it

terse sky
#

they have some API that some people might like but I'd argue is pretty bug prone

#

they have iteration and destructuring. But if you're already working with named fields, destructuring doesn't really make any sense.

#

You're more likely to mix up the destructuring order and cause a bug that way

rustic gull
terse sky
#

and iteration doesn't really make sense in a more strongly typed context

brisk heart
#

I use namedtuples in my lib actually. For a single method that would've returned an object with just two integer attributes

terse sky
#

You don't need to create named tuples in that awkward way, it has a syntax that' smuch more similar to dataclasses

brisk heart
#

Unpacking is the only reasons I'd ever use named tuples though

terse sky
#

Right, so in that example, my issue is that you're encouraging users to use a bug prone API.

foo, bar = func_call()
#

when the actual fields can be ordered bar, foo

#

whereas if you do x = func_call(), x.bar and x.foo will always be what the user expects

#

another advantage of keeping tighter control of the API is that you can add additional fields without breaking API, though that may not be relevant

brisk heart
#

I feel like I'm forcing the user to write longer code like that

hearty shell
#

Isn't that arbitrary though? Tuple unpacking is used with and expectation that the user knows the order that the elements are coming in, same thing with pattern matching without names

terse sky
#

it's very marginally longer, and it's a small price to pay to know that something's correct

terse sky
#

That seems relatively objective

rustic gull
#

I think some things a tuple is great for, like.... an ordered pair of coordinates. I dont need to name those.

terse sky
#

Yeah, that's a decent counter-example. The thing is that even there, you suffer a lot of the downsides of tuples as generic types

#

for example, if you want to add behavior to your coordinates like addition, you can't really do it on tuple (or rather, even if you can, it's a bad idea)

hearty shell
#

Well, yes sorry my bad, objectively one is more explicit then the other, I just mean that the standard for something to be used or not based on that is arbitrary

#

I just mean that if you are unpacking a tuple, you ought to know the order it is coming in

#

But you should also allow users to use the explicit api

terse sky
#

that's the thing though, the order is almost always arbitrary

#

so the user is basically just reading the docs to verify the order, and then looking back at their own code to double check it

#

it's always risking a trivial error that the other approach just avoids completely for free

rustic gull
#

well maybe that is an occasion where there is a better way to organize it but i think sometimes things are simple and the ordered pair / more doesnt need added behavior.

terse sky
#

to look at a simple example, python and C++ for example, when iterating a dict, tend to yield pairs

#

that's because they're older languages ๐Ÿ™‚

#

new languages don't do that

#

they yield some kind of struct that has named fields/properties/etc, key and value or something like that

rustic gull
#

right.

terse sky
#

although in many cases destructuring is still allowed, I'll admit

hearty shell
#

are you saying that for key, value in dict.items() is a bad api?

rustic gull
#

i get what you're saying

#

even passing parameters into functions is often better when you can just use kwargs and not positional, especially if there are lot of options

terse sky
#

I have mixed feelings about that one. But dict.items() having the type, Iterable[Tuple[K, V]] is definitely bad

#

A strictly better approach would be for it to have some kind of dataclass type, and support destructuring on dataclasses. But that's part I have mixed feelings about.

rustic gull
#

well, it does iterate using a tuple...

terse sky
#

right, because python does not allow destructuring on dataclasses.
In the context of python, maybe a named tuple would be the best approach

rustic gull
#

i think with something as simple as "key" and "value" is not that hard to confuse which is which.

terse sky
#

but it's also a bit different, a dictionary is generic, and I did say at the start that generic code is where you're most likely to enjoy using tuples/destructuring

rustic gull
#

i agree with that

terse sky
#

for key, value in dict.items(); is very different with mean, std_dev = compute_population_charaacteristics()

rustic gull
#

mm.

hearty shell
#

Yeah I do agree with that

#

Can you pattern match on namedtuples?

terse sky
#

good question. I have yet to use pattern matching.

soft matrix
#

yes

#

you can match on tuples why would you be able to on namedtuples?

hearty shell
#

!e

from typing import NamedTuple

class Foo(NamedTuple):
    mean: float
    std_dev: float

for something in [Foo(mean=1.23, std_dev=0.2)]:
    match something:
        case Foo(std_dev=std_dev, mean=mean):
            print(mean, std_dev)
rough sluiceBOT
#

@hearty shell :white_check_mark: Your eval job has completed with return code 0.

1.23 0.2
hearty shell
#

Interesting

hearty shell
rustic gull
#

wait... did python get a switch case?

#

im behind! haha

hearty shell
#

From what I heard (I have never used "switch"), pythons pattern matching is supposedly way more expressive then switch stms

#

and yeah they were added last version

rustic gull
#

i see. I use it in Javascript when it's easier to express than conditional branching in other ways.

#

but thats it

#

im gonna read

hearty shell
#

There was a post recently that had a nice map of all the features

#

let me find it

rustic gull
#

well i need to upgrade to 3.10, ive been meaning to do it for the shorter str | int syntax in the type hints

grave fjord
rustic gull
#

ha, i never knew what that import was for and now i know

hearty shell
#

Ah found it

hearty shell
rustic gull
#

ahem code should be in utf-8 encoded text not rendered in jpegs ahem, but cool ๐Ÿ™‚

hearty shell
#

If you want code you can just look at the pep, this is just an overview of what it might have that switch stmts might not :P

rustic gull
#

i like it. this way of doing a large amount of conditionals is very good to have

#

i already did read the docs on it

#

its cool

hearty shell
#

I forgot that there was that deferred idea

rustic gull
#

so im just starting with mypy type checking and the first thing im finding is that i didnt expect the emphasis on imports. All of the modules i wrote in my project have something like

error: Cannot find implementation or library stub for module named "cli"

How do I resolve this aside from installing stubs, which my files wouldnt have, and silencing it?

#

are stubs just "type hints but in another file"

#

Wow, type checking is a lot of work to just throw at a code base you have never done it on before

#

damn, i wish there was a way that i didnt have to keep track of types packages for every lib and where they can be found or if i have to stubgen them but i think once im used to it will be nothing to do as i build

oblique urchin
rustic gull
#

yeah i figured that out... what im having trouble with now is the packages that will not find the stub i generated even though it generated.

(venv) PS C:\Users\x\PycharmProjects\bandcamp-flac-get> stubgen.exe -p bs4  
Processed 9 modules
Generated files under out\bs4\
(venv) PS C:\Users\x\PycharmProjects\bandcamp-flac-get> mypy.exe -p src.bcfg
src\bcfg\context_scraper.py:2: error: Skipping analyzing "bs4": module is installed, but missing library stubs or py.typed marker
src\bcfg\context_scraper.py:3: error: Skipping analyzing "fake_useragent": module is installed, but missing library stubs or py.typed marker
#

some of them worked.

#

but the ones that dont want to dont want to

#

it might be better to try to learn this on a new project rather than one i havent been thinking about this on

blazing nest
#

PEPE 665 wasn't accepted until now?!

#

Wow I've been using it for so long I thought it already was a thing

hearty shell
#

655?

#

And yeah gotta love typing extensions xD

blazing nest
#

Required and NotRequired

hearty shell
#

Yeah thats 655

blazing nest
#

Oh lol whoops

shy narwhal
#

I have a question, im just starting out and I dont know how to make this work

#

I want all of these words to be accepted for this variable

Screenshot_3.png
hearty shell
noble pilot
#

Ok I'm back and my solution works, but type hinting still not doing what I want it to do. Keep in mind I'm not using mypy. I did a series of inheritance where I narrow down the entity types with generics. Obviously I'm doing this wrong or perhaps PyCharm just cannot infer the types. In the test section, you can see that I use "ConvertedObject" I would like for pycharm to typehint to that object properly. Right now its still resolving to the generic Item_T. So my question is if I'm doing the generics wrong or perhaps this is a limitation of pycharms linters

https://mystb.in/InsertedSwordPros.python

hearty shell
#

Could you provide a minimal example of the problem?

noble pilot
#

the screenshots kind of illustrate it. I would like for it to resolve Item_T to ConvertedObject

#

or Container_T to be the Dict[int, List[ConvertedObject]]

hearty shell
#

Yes, it is just hard to tell if the problem are your generics or pycharm without looking at the lengthy, mostly unrelated code x)

noble pilot
#

Ok I think I boiled it down to the following

Container_T = TypeVar("Container_T")
Entity_T = TypeVar("Entity_T")
Item_T = TypeVar("Item_T")


class TestABC(Generic[Container_T, Entity_T, Item_T], ABC):
    item: Container_T
    
    
class SomeAbstract(TestABC[Dict[int, List[Entity_T]], Entity_T, Item_T], ABC):
    # does item now have that type?
    
    def get_items(self, id) -> List[Entity_T]:
        return self.item.get(id, [])

class SomeEntityProtocol(Protocol):
    some_attr: int
    

class SomeImpl(SomeAbstract[SomeEntityProtocol, Item_T]):
    # Does item now define as Dict[int, List[SomeEntityProtocol]
    pass

from dataclasses import dataclass

@dataclass
class MyItem:
    stuff: str

class SomeOtherImplThatDefinesItem(SomeImpl[MyItem]):
    # Does get_items now return List[MyItem] ?
    pass
#

I'm very sure I'm approaching Generics wrong

#

@hearty shell

hearty shell
#

that looks right I think, you can see it in mypy

#

However on that last part, it wouldnt return MyItem because you said SomeAbstract to return List[Entity_T]

#

and you already defined Entity_T to be SomeEntityProtocol

noble pilot
#

oh dang. I was hoping it would substitute it

hearty shell
#

higher up the hierarchy

noble pilot
#

Do you see what I was trying to achieve? I assume no way to do it that way then?

#

would really like the entity to type hint having a member 'some_attr' and the item type hint as MyItem without having to override everything

hearty shell
#

I mean, why would the other implementation inherit from another implementation?

#

This seems weird even outside the typing context

noble pilot
#

The only reason would be to specify Item_T

#

and I was really just thinking in the sense of autocompeltion etc

#

it works as now because duck typing etc

hearty shell
#

Oh so you would like SomeOtherImplThatDefinesItem just to implement the logic of SomeImpl as to avoid code duplication, but not be an actual suptype, allowing you to just change the expected types

#

Right?

noble pilot
#

yes

hearty shell
#

Yeah, sadly this is not supported as of yet

#

however

noble pilot
#

noooooooo haha ๐Ÿ™‚

hearty shell
#

What you could do instead is make your implementation generic as well, and then just have two classes inheriting from it to define the types

noble pilot
#

hmm, what do you mean?

hallow flint
hearty shell
#

Although I think I might be misunderstanding your example, how do you want the Item_T variable to be used?

#

There is no use of it in the example you gave

noble pilot
#

I was thinking Item_T would just be substituted later on some inheritance

hearty shell
#

What is its use though? Just asking so I understand what you are trying to achieve better

noble pilot
#

well suppose

cache = MyItemCache(SomeImpl[MyItem])
items = cache.get_items()  # List[MyItem]
# do stuff
hearty shell
# 
class TestABC(Generic[Container_T, Entity_T], ABC):
    item: Container_T
    
    
class SomeAbstract(TestABC[Dict[int, List[Entity_T]], Entity_T], ABC):
    # item: Dict[int, List[Entity_T]]
    
    def get_items(self, id) -> List[Entity_T]:
        return self.item.get(id, [])

class SomeEntityProtocol(Protocol):
    some_attr: int

class SomeImpl(SomeAbstract[SomeEntityProtocol]):
    # item: Dict[int, List[SomeEntityProtocol]]
    
    # get_items(self, id) -> List[SomeEntityProtocol]:
    pass

from dataclasses import dataclass

@dataclass
class MyItem:
    stuff: str

class SomeOtherImplThatDefinesItem(SomeAbstract[MyItem]):
    # item: Dict[int, List[MyItem]]
    
    # get_items(self, id) -> List[MyItem]:
    pass
#

What is wrong with this?

noble pilot
#

is it actually resolving correctly?

#

pyCharm won't recognize SomeOtherImpThatDefinesItem.get_items() as List[MyItem] It says List[Item_T]

hearty shell
#

main.py:42: note: Revealed type is "def (id: Any) -> builtins.list[__main__.MyItem*]"

#

Item_T? There is no need for Item_T though

noble pilot
#

oh so this is a pycharm thing then. Dang

#

But I do use EntityT so the item container is different than the get_items

#

but this does tell me atleast my approach was right in the scope of typing... just pycharm can't infer it

hearty shell
#

It looks like it does?

noble pilot
#

and I made a mistake on my code block above. I meant List[Item_T] instead of Entity_T

#

oh crap wtf

hearty shell
#

I am very confused on what you actually want to achieve xD

noble pilot
#

No, that is what I want to achieve

#

the toy example works, my actual not

#

in this case, I wanted it to be dict[int, list[DiscordGuildRelation]]

hearty shell
#

Lets see

#

class PgListenerCache(Generic[Container_T, Entity_T, Item_T], ABC): ...

#

class ManyToOneCache(PgListenerCache[Dict[int, List[Entity_T]], Entity_T, Item_T], ABC): ...

#

class DiscordGuildRelation(Protocol):

#

class DiscordEntityManyCache(ManyToOneCache[DiscordGuildRelation, Item_T]):

#

class ManyTextChannelCache(DiscordEntityManyCache['TextChannel']):

#

class ManyConvertedObjectCache(DiscordEntityManyCache['ConvertedObject']):

noble pilot
#

ok interesting so the get_items() would work in the example. I can't get the item attribute though to type correctly though. It just shows Dict[int, List[Entity_T]]

hearty shell
#

item is Container_T right?

noble pilot
#

yes

#

the goal would be to replace Entity_T with DiscordGuildRelation

hearty shell
#

Oh you are right I think that should resolve

#

Overall pycharms builtin type checker tends to be unreliable imo

noble pilot
#

hey thanks for the help!

hearty shell
#

No problem! Hopefully one day typing in python will become less of a "I am wrong, or is my tool wrong"

#

Sadly for now this is the current state, even with mypy and pyright, although to a lesser degree

noble pilot
#

I realized I was doing some thing wrong. And I like this mypy site to test

hearty shell
#

Yeah it is really useful

noble pilot
#

trying to figure out why I'm getting those

#

I did change for the Container_T to be a Dict[int, List[Item_T]] this time

#

ignore the method names. I did some find/replace and it mixed them up

hearty shell
#

Ah right

#

class ManyToOneCache(PgListenerCache[Dict[int, List[Item_T]], Entity_T, Item_T], ABC):

#

Here you are flipping the order from Entity_T, Item_T to Item_T, Entity_T

#

So when you do

#

class DiscordEntityManyCache(ManyToOneCache[DiscordGuildRelation, Item_T]):