#type-hinting
1 messages ยท Page 4 of 1
uhh, isn't it the other way around?
the other way around
oh yeah right
If I yield as well as return the same type, should it be Iterator or Generator
If you return something or expect something to be sent, you'll need to use Generator
well generators in python can do some interesting stuff
Generators can be sent values to
def foo():
r = yield 5
assert r == "a"
x = foo().send("a")
assert x == 5
yes, that
Wtf
sent_value = yield ...
that's why we have 3 type vars in generator
I understood exactly 0 of this code
lol
and you generally won't need to understand it tbh
I mean wtf does r = yield 5 even mean
this can be useful in some cases, and it was especially useful in early versions of python where we didn't yet have async keywords so we used yield for await and generators as async functions
it means that you yield 5
from the generator
but then it pauses and goes back to where you process the 5
where you send('a') back to the generator
and it "exits" from the yield statement
as in, it returns a value you can assign to a var
hence the sent_value = yield ...
it doesn't yield immediately
Rut
you need to call send on the generator
which is done automatically when you iterate over it
but the iterator uses send(None) IIRC
I am sure I won't need this ever
so you can't really do anything inside the gen with it
that's my point
I never used this in production
Basically you alter the functions local state from outside
via calling send and throw methods on it
I have, and I don't want to repeat that xD
but again, it was useful when we used generators for async in early python
that's how coroutines work btw
in asyncio
under the hood
Interesting
well, they used to be generators yeah
I never touched asyncio itself
I am stuck with too much grassroots level stuff
I made some free monad nonsense or something... So the application could technically work with both a sync and an async runtime. Did I need that? Of course not!
a bit is an understatement
!e
async def foo(): pass
print(foo())
@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.
001 | <coroutine object foo at 0x7f6f696089e0>
002 | <string>:2: RuntimeWarning: coroutine 'foo' was never awaited
003 | RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Like deciding whether I should use a dataclass or property getter and setter based models
so, a special class then
Dataclass has no state, properties have state
written in C or just a subclass of a generator though?
Combining properties with private backing variables is devil worship
!e ```py
import asyncio
@asyncio.coroutine
def main():
yield from asyncio.sleep(1)
print('done')
asyncio.run(main())
@brazen jolt :white_check_mark: Your 3.10 eval job has completed with return code 0.
001 | <string>:4: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
002 | done
My worship? 
3.10 still has asyncio.coroutine, 3.11 no longer does
it does show how asyncio worked with generators though
Yeah
I doubt whether I even should be using TypedDict for kwargs to class ctors
I mean I can just add them to the ctor itself
pyright doesn't complain about LSP when ctor signature changes,
Only mypy crybaby does
Just TypedDict isn't enough so I need to use it with Unpack
what if you want a read-only attribute?
Just use a property without a setter
How can I extract the type params I passed to a generic descriptor used for a class attribute?
prop = EventProp[int]() I want to extract int out of this
self.__orig_class__.__args__[0]
you mean prop instead of self?
wow thanks that worked
@soft matrix Any idea how I might use that to show such an attribute as a property with sphinx autodoc?
Orig class?
Idk how is go about doing that other than just using a property if docs are building
how do i make mypy ignore all the AttributeErrors for db
it's generated by flask sqlalchemy and it has dynamic attributes
&> py -m mypy app
app\forms.py:1: error: Skipping analyzing "flask_wtf": module is installed, but missing library stubs or py.typed marker
app\forms.py:2: error: Skipping analyzing "wtforms.validators": module is installed, but missing library stubs or py.typed marker
app\forms.py:3: error: Skipping analyzing "wtforms": module is installed, but missing library stubs or py.typed marker
app\db.py:4: error: Skipping analyzing "flask_login": module is installed, but missing library stubs or py.typed marker
app\db.py:5: error: Skipping analyzing "flask_sqlalchemy": module is installed, but missing library stubs or py.typed marker
app\db.py:31: error: Name "db.Model" is not defined
app\db.py:76: error: Name "db.Model" is not defined
app\db.py:103: error: Name "db.Model" is not defined
app\db.py:129: error: Name "db.Model" is not defined
app\db.py:166: error: Name "db.Model" is not defined
it gives these errors, but since db.Model is dynamically generated mypy cannot detect it
so how would i do that?
jump to the type source of db
then just copy in
def __getattr__(self, name: str) -> Any: ...
doesn't seem like it fixed it
app\db.py:31: error: Name "db.Model" is not defined
app\db.py:76: error: Name "db.Model" is not defined
app\db.py:103: error: Name "db.Model" is not defined
app\db.py:129: error: Name "db.Model" is not defined
app\db.py:166: error: Name "db.Model" is not defined
stdlib/builtins.pyi line 194
_PositiveInteger: TypeAlias = Literal[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25]```
2**25 is int and 2**26 is Any

int(sqlite3.connect(":memory:").execute("SELECT pow(2,32)").fetchone()[0])
how would i typehint this?
from itertools import repeat
def repeat_all(*values):
return tuple(repeat(value) for value in values)
i want to convey the fact that the output types correspond to the input types
this probably helps, but couldnt figure it out: https://peps.python.org/pep-0646/
Python Enhancement Proposals (PEPs)
from typing import TypeVarTuple # requires python 3.11
Ts = TypeVarTuple('Ts')
def repeat_all(*values: *Ts) -> map(Iterable, Ts):
return tuple(repeat(value) for value in values)
i kinda want this, but this is not valid because you cant call functions in type hints
It's not possible yet. A type-level Map operation PEP may come for 3.12
cant wait for it :P
(seems to be a common trend where i always want the newest features for typehinting)
custom_types.py:14:33 - error: Expected class type but received "() -> float" (reportGeneralTypeIssues) isn't that weird?
!d typing.NewType
class typing.NewType(name, tp)```
A helper class to indicate a distinct type to a typechecker, see [NewType](https://docs.python.org/3/library/typing.html#distinct). At runtime it returns an object that returns its argument when called. Usage:
```py
UserId = NewType('UserId', int)
first_user = UserId(1)
``` New in version 3.5.2.
Changed in version 3.10: `NewType` is now a class rather than a function.
hmm
@deep pendant What is time and Seconds?
Are you sure you didn't import time from time instead of datetime?
I'm trying to create this CachedFunction type alias, but mypy is throwing errors at me.
T = TypeVar("T")
R = TypeVar("R")
if TYPE_CHECKING:
from typing_extensions import Concatenate, ParamSpec, TypeAlias
P = ParamSpec("P")
CachedFunction: TypeAlias = Callable[Concatenate[T, P], Awaitable[R]]
The first argument to Callable must be a list of types, parameter specification, or "..."
The last parameter to Concatenate needs to be a ParamSpec
ParamSpec "P" is unbound
I'm putting it in if TYPE_CHECKING because pyright didn't like it when I put TypeAlias in quotes.
both are bugs if thats the case
ive seen the first one in the mypy issue tracker but open one for pyright about typealias in quotes
Yeah, looks like the closest I can find for pyright is #2275, but that's about the typealias value, and multi line strings
Yeah, I had imported time from time by mistake. The point was that the Pyright's message is not very meaningful.
After some time I realized what's wrong, but not by Pyright's message help ๐
how do I add type annotations of a property to a generic descriptor at runtime
its for sphinx to correctly autodoc my API
@rare scarab
idk
Is there a way to use a ParamSpec declared inside TYPE_CHECKING other than this? Because this doesn't actually detect that any type vars are being declared at class scope.```py
CacheBase = object
if TYPE_CHECKING:
CacheBase = Generic[T, P, R]
class Cache(CacheBase):
pass
This seems to work better actually: ```py
class Cache(Generic[T, P, R] if TYPE_CHECKING else object):
But I hate that it only works inlined
that wont work with mypy i dont think
if TYPE_CHECKING:
CacheBase = Generic[T, P, R]
else:
CacheBase = object
```might work?
yeah, make the whole definition conditional
that does not work (in pyright)
yo @soft matrix you know a way to modify a generic descriptor's type annotations to property's ones?
mypy says "Base is not valid as a type"
property isnt generic
yes
ik
i just need to know a way that my attribute will look like a property to whatever sphinx uses for autodoc
property is actually evil magic witchcraft
you have 2 options then
- put P in quotes and live with it maybe working (might work)
- define P = TypeVar("P") in the else block of the if TYPE_CHECKING where its defined as a typevar
Putting P in quotes does not work
I am talking this, attributes are so messed up, they don't even have some annotation
https://github.com/Gobot1234/steam.py/blob/main/docs/conf.py#L14
https://github.com/Gobot1234/steam.py/blob/main/steam/_const.py#L21
then
if DOCS_BUILDING:
my_descriptor = property
```may work?
docs/conf.py line 14
builtins.__sphinx__ = True```
`steam/_const.py` line 21
```py
DOCS_BUILDING: bool = getattr(builtins, "__sphinx__", False)```
Are you using sphinx-autodoc-typehints?
yes
what about napoleon?
is napoleon loaded before typehints?
that wont work for generic descriptor
Move napoleon to load before autodoc.
in your extensions list
why not?
dunno but property isn't generic
so if i make my descriptor = property
how will the [] syntax work
property[int] doesn't work does it
even if it did, sphinx would probably not undertand it?
property is usually special-cased. If it uses mypy, it should know what to do with it.
lemme try if it works
ok then other equally shitty idea
note: property is only specialcased when used as a decorator. Manually assembling your property does not work.
property is fucked up
use fishhook and overwrite property.__isinstancecheck__ or w/e its called to make it think its a subclass of property
or maybe just make it a property subclass if thats at all possible?
Why do you need a runtime type for this?
a protocol would work fine as a last resort
i tried this one, but property is way too opaque to understand
class Foo:
if not TYPE_CHECKING and DOCS_BUILDING:
@property
def my_descriptor_field(self) -> str: ...
else:
my_descriptor_field = ...
```?
plus i have got no idea how to convert a descriptor into a property
holy fuck i got like 500 attributes, I do this for every attribute imma better rewrite sphinx instead
yea but I can't just subclass my descriptor into a property
can I?
idk try it
you probably can if its not got a metaclass (why you would need one for a descriptor is beyond me)
and it doesnt have slots(???)
class EventProp(RWProperty[T]):
def __init__(self, *ids: EventEnum, default: T | None = None):
self._ids = ids
self._default = default
def __get__(self, instance: MultiEventModel, owner: Any = None) -> T | None:
if owner is None:
return NotImplemented
for id in self._ids:
try:
event = instance._events[id][0]
except (KeyError, IndexError):
continue
else:
return event.value
return self._default
def __set__(self, instance: MultiEventModel, value: T):
for id in self._ids:
try:
event = instance._events[id][0]
except (KeyError, IndexError):
continue
else:
event.value = value
this is my descriptor, now how do I get it to behave like a nice property?
do i touch fget, fset at all?
or do i just touch get, set dunders?
no actually, napoleon support type hints, idk why this package still exists
if not TYPE_CHECKING and DOCS_BUILDING:
class EventProp(property, Generic[T]):
def __init__(self, *args, **kwargs): ...
this should work?
you dont need the get and set implementations when building your docs do you?
no
i just need the generics to work
which wont work if EventProp becomes itself a non generic class
oh yeah that too
but hang on
there's this in sphinx-autodoc-typehints I can probably use this
you might need a plugin
but everywhere I have wrote attributes like this: desc = EventProp[int](...) this means Python won't even be able to add an annotation
i need a runtime type annotation adder, are there pre-existing plugins for this or do i just do all the dirty work in conf.py
why didnt u use the autodoc process signature hook?\
hmm, the hook i mentioned fucks up the formatting even more
@soft matrix hey
def add_descriptor_annotations(app, what, name, obj, options, lines):
if what == "attribute" and isinstance(obj, EventProp):
obj.__annotations__ = {"return": obj.__orig_class__.__args__[0]}
```this doesn't work
where am I wrong
idk use a debugger
idk how your script works
it does basically the same thing
How do I access the class obj is the attribute of?
uh actually i don't need to
i could just iterate here over classes instead of attributes, and assign annotations to the class object as i go
def f(args*) -> bool:
return sum(map(bool, args)) == 1
How would you type the args* parameter here?
object?
Yep
It would work even if args is heterogenous right? Because everything is object
Yep
Perfect, thank you ๐
@soft matrix Succes!, Thanks
@property
def prop(self):
return 100
now sphinx cannot think of a type automatically, but pyright knows what the return type is. since there's no annotation, is there a pyright Python API which will give me the type of prop?
how to use pyright inferred return type programmatically
nope not really
:((
you can do it but its a butt tonne of work
i already feel like I climbed a mountain to get here
i had been finding an answer for that for past 2 days
youd need to write a webserver and query the types from a pyright language server
vscode lets you insert type hints from inlay comments.
thats the next thing i plan on doing to improve this
i think you should make a sphinx extension or file a pr for autodoc with this
how can I do code style in Discord?
!code
Here's how to format Python code on Discord:
```py
print('Hello world!')
```
These are backticks, not quotes. Check this out if you can't find the backtick key.
thanks!
wdym
where's that option
if it can be automated i'd bless u 2 wives
python.analysis.inlayHints.functionReturnTypes: true
Then click on each function you want to inline
lemme try
neat
i might if i get the pyright idea working
Note it doesn't actually put anything in your code unless you double click it.
Sometimes it gets a bit specific.
that's good
pyright's inferences are god level
this doesn't work :((
you put that in .vscode/settings.json?
no.
you made it actually json, right? not as is
{
"python.analysis.typeCheckingMode": "basic",
"python.analysis.inlayHints.functionReturnTypes": true,
"python.analysis.inlayHints.variableTypes": true
}
yes
i did
is it only for functions or properties also
if i had to guess both
yea it doesn't work
i checked pylance settings for the both, not disabled there either
@soft matrix You have any solution for this https://stackoverflow.com/q/73693982 ?
Nope sorry never used tables for anything important in docstrings
how about cross references to built in type docs with intersphinx
like in this, I would like to cross ref pathlib.Path to official python docs
every pathlib.Path
@soft matrix
yes?
ahh nvm sorry
Guys, how would you do this
It can be easily done with dataclasses, but I don't think one is needed
It doesn't have to be a list
This snippet is more similar to TS than py haha
TypeBar: [number, number, str] = [1, 2, "Foo"]
This doesn't seem to solve it haha
TypeBar: List[float, float, str] = [1, 2, "Foo"]
you need to use a tuple to encode length into something like this
Isn't any other Iterable[] option for this? I wanted to use any iterable, but tbh, tuples would work well
Maybe it's not worth the hassle to try to make it any kkdn of Iterable xD
Thank both guys btw
hmm pyright is a bit too helpful here ๐
https://pyright-playground.decorator-factory.su/?gzip=H4sIABmYIGMC_0vLz7dSKCktyEmNTsvJTyzRUYBSxSVFsQq2ChqGOgpGOgpKafn5SppcSYlFGGJcRallqYk58SWVBakaQCFNFAGgDk0uAEJ7R_9lAAAA
Iterator/Iterable would require implementing "session types" I think. Which sounds enormously complicated
because right now, a mutation on an object cannot change its type.
so as of now a tuple is the thing
why do you need Iterable[float, float, str] though?
Basically, it's
I need a minimum and maximum, and a mode to handle them
not iterable needed, but seemed nice to be able to pass any kind of iterable
I would probably use a dataclass or a NamedTuple here
those tuple things are kinda cryptic tbh
It must feel horrible not being able to do {min: float, max: float, mode: "foo" | "bar"} ๐
NamedTuple is preferred if you're going to make a LOT of objects.
Why?
I think there is no difference in memory between tuples and namedtuples
Custom classes with slots requires 8 bytes less memory
i think just having a lot of anonymous tuples floating around in your annotations isnt a great idea
Yes, yes it does
I wonder why py didn't take the same route
Ts wya seems very intuitive
Mypy is good tho, ngl
But it can be better
B)
have you tried pyright?
(or pylance if you're on vscode)
I've tried pylance
Not too much into it
Just the basic stuff, I guess
When yielding, is the type annotation generally def foo() -> typing.Generator or what I'm yielding?
-> Generator[YieldType, None, None] usually
hopefully just Generator[YieldType] soon
If you don't use the send thing or return from a generator, just use Iterator IMO
Alright
once typevar defaults are implemented i wouldnt recommend that but yeah for now iterator is probably good enough
imo i would still recommend Iterator unless you specifically intend it to be used as a generator
Generator is a weird api to constrain yourself to if it's not necessary
That's because the decorator actually uses methods from generator, but for most applications when you're creating a generator it's only for the iterator and typing that as Generator just seems like leaking your implementation
close may be useful if it's managing resources so I could see the merit there, but that's not as common as just directly yielding something
Number = TypeVar("Number", bound=Literal[1, 2, 3])
foo: Dict[Number, str] = {
1: "one",
2: "two",
3: "three",
}
```What's the issue with this? I'm getting ```error: Type variable "Number" is unbound [valid-type]```on line 3 (where foo is defined)
`Number` seems to function as a valid literal in every other instance, as shown in ```py
def bar(baz: Number) -> None:
return None
bar(1) # Works
bar(4) # Expected type 'Number', got 'Literal[4]' instead
@ me if responding
@torpid yarrow why are you using a TypeVar here, they dont make sense here?
use a type alias
ofc 
Thanks
Dunno why I thought to use a typevar
Hello people! I don't know if this is the right place to post this but I got a question: how should I write a docstring for a function that may return two different types?
To illustrate, I have this function that i wrote:
def get_counters(data: list, k = 1):
"""Counts each term from chats that received a label assuming the dataset is already filtered by an intention.
Args:
data (list): a list of words.
k (int, optional): size of the ngrams. Defaults to 1.
Returns:
collections.Counter | nltk.FreqDist: a key-value structure with the frequency of each ngram.
"""
# code goes here
When I hover my cursor over the function, Pylance says it returns (Counter | FreqDist) which is nice, but is there a convention on how should I write this in my "returns" field in the docstring? I searched on stackoverflow but I couldn't find a consensus
why not put the return type in the function signature itself? i don't bother writing the types at all in docstrings when i have them in the function signature, and sphinx is able to render those properly
also it seems really weird to possibly return 2 different types here. what causes it to switch?
def get_counters(
data: list[str], k: int = 1
) -> collections.Counter | nltk.FreqDist:
"""Counts each term from chats that received a label assuming the dataset is already filtered by an intention.
Args:
data: A list of words.
k: Size of the ngrams. Defaults to 1.
Returns:
A key-value structure with the frequency of each ngram.
"""
...
i believe this is still valid as per the google docstring style, and is also more conventional
basically, if k > 2 the function returns a FreqDist, else, a Counter
i feel stupid, i didn't thought about it ahahah
because im working with ngrams and freqdist makes my life a lot easier ๐
Is it possible to convert a list[Union[A, B]] which only contain As to a list[A]? Here is my concrete use case:
I have some code that creates a list[Foo] one element at a time, but not in order. The way I do it now is like this:
my_list: list[Optional[Foo]] = [None] * KNOWN_LENGTH
for element in stuff:
# set every element of my_list to a proper `Foo`
my_list[index(element)] = Foo( ... )
At the end of the loop, there should be no Noneelements left. Is it possible to somehow "convert" my_list to a list[Foo]
hmm, why not just do
my_list = [Foo(...) for element in stuff]
or at least
my_list: list[Foo] = []
for element in stuff:
my_list.append(Foo(...))
Because the order of the final array matters, and the for loop of stuff is in the wrong order
how do you know that all the indices are set? could you tell more about the algorithm?
Succinctly, my_listis a int -> Foo mapping, which could of course be a dict[int, Foo], but which I've implemented as list[Foo]
Essentially, it (and many other variables in the code base) is a dict[str, Foo], but for performance reasons, in the beginning of the program, I assign an integer to each string and henceforth treat them as simply integers, then at the very end fetch the original strings when printing
have you checked that it's actually faster?
also, do you have a fixed set of strings? how many of them?
Not a fixed set - between 10e5 and 10e7, realistically. Maybe a few tens of millions
I ended up doing:
my_list_asserted = [i for i in my_list if isinstance(i, Foo)]
assert len(my_list_asserted) == len(my_list)
return my_list_asserted
which I guess is good enough, except it allocates a new list. It probably won't hog too much memory
I'd just do typing.cast(list[Foo], my_list)
but have you actually benchmarked that doing it in this way is faster than just using the strings?
and what kind of time/memory requirements do you have?
what is the program doing, in general?
I haven't actually benchmarked, though. I mean, it is faster, but the real issue is memory consumption and I haven't measured that.
I wouldn't expect it to be faster, really. Strings cache their hash value upon first computing it. And integer hashes do tend to have more collisions.
!e
because for small integers (in your range of tens of millions), the hash is the integer (in CPython):
print(hash(65536 + 27))
print(hash(27))
print(hash(65536 + 27) & 0xff)
print(hash(27) & 0xff)
@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.
001 | 65563
002 | 27
003 | 27
004 | 27
In general, what the program does is:
Given N clusterings of observations, each observation with an associated length and each cluster with a completeness/contamination score, whenever two clusters A and B have an intersection with a total length more than X% of min(length(A), length(B)), remove the cluster with the lowest score.
The result is a new set of clusters which is not entirely disjoint, but where no two clusters are "near-identical" by the definition of overlaps as above
ok I'm probably too stupid to understand that
Anyway thanks, typing.cast was exactly what I was looking for! (I can assert that is it actually correct without incurring any memory loss at all)
it's mostly the same as a # type: ignore comment, but more explicit
you could theoretically leave an assert and then run the program with the -O flag
dont add the protocol bases to your class, theres no reason to do so
why not
it tells anyone that a class implements a protocol
the pep also describes it as a valid use case
so does just writing def __hash__(self) -> int: ...
but yes i agree, that does seem like a bug
somewhere i remember python 3.8 changed metaclasses of one of these protocols
try the typing extensions versions of these classes if theyre available
plus if a protocol is generic you need to subclass it
specifically for 3.7?
i think imma better remove them
yeah
are you storing these all in a list and dynamically deleting elements from the list?
No, I'm deleting them in one pass. I'm aware that there are unhandled edge cases where cluster A may overlap with cluster B and B with C, but A and C do not overlap. In that case the order of removing clusters matter, but i do not handle that.
eh, that's typical for a greedy algorithm
i agree that typing.cast is the way to go here, however i do wonder what the two cluster "types" are
Whoa generic typeddicts in 3.11
that's nice
should help a bit with the current issues around subclassing and extending typeddicts
It's a shame it's a runtime thing
Although I guess it can be backported through typing extensions without any problem
I think we backported it in typing-extensions
nice
I kinda wish there was a public way to access dataclass fields by name without dataclasses.fields which only returns a tuple of fields you need to loop through
There's the private __dataclass_fields__ but maybe I shouldn't be trying to abuse the metadata field in the first place
with so few fields, looping through a tuple and filtering on name should be nearly instant. although i agree it'd be a little tidier if there was a mapping/dict-like interface
(frozendict in the standard library when?)
you would have liked the typing talk last night @mortal fractal from tin
ill see if i can dig up the recording
Oh yeah, what was it about?
Next Steps for Static Analysis in attrs, Tin Tvrtkoviฤ.
We will be looking at ways to project and select fields from ORM/ODM
classes in a type-safe manner.
Nice
i'd want to see this too!
What's the proper way to type a generic type with a default value? mypy doesn't like this. ```py
TNode = TypeVar("TNode", bound=Node)
def resolve_node(cls: type[TNode] = Node): # Incompatible default for argument "cls" (default has type "Type[Node]", argument has type "Type[TNode]")
...
@overload
def resolve_node() -> Node: ...
@overload
def resolve_node(cls: TNode) -> TNode: ...
def resolve_node(cls = Node) -> Node: ...
Too bad pyright doesn't like untyped overload implementations
!pep 696
you have my blessing to # type: ignore ๐
@mortal fractal @fierce ridge https://docs.google.com/document/d/17iqV7WWvB0IwA43EPlIqlUS6Xuvk08X3sEudAA-gQIo/edit?usp=sharing
they dont seem like related features
sort of. more complexity in any type signature means more complexity when subclassing or adhering to an interface
(until we have some typing.inherit decorator that does it for us)
oh also, thank you for sharing this!
Might be able to hack it if we had an infer function. In ts, you could use ```ts
type Parameters<F> = F extends ((...args: infer P) => any) ? P : never;
That's a built-in type btw
In python, that could be something like this. ```py
F = TypeVar("F")
P = Infer("P")
Parameters = Extends[F, Callable[P, Any], P, Never]
Args for Extends would be input type, test type, true type, false type
what does infer do? take the type of ...args from P? like ParamSpec.args i think
Yes
It extracts a generic type argument and lets you use it in another type or return it outright.
I guess infer could also be an attribute of typevarlike so we don't have to add a new type
it's like pattern matching
Adding the match statement to typing would be neat
Though isn't that a statement, not an expression?
anyone know what mypyc_task/tasks.py:256: AssertionError: unexpected type <class 'mypy.types.DeletedType'> means?
https://github.com/graingert/mypyc-task/actions/runs/3059980689/jobs/4937970759#step:4:21
this is the line that's complaining https://github.com/graingert/mypyc-task/pull/1/files#diff-12dfb1ef034a6ae85d7b81cd787a4c02f9cd0b53cd6deea3a51c36a8c19cf819R256
mypyc expects your var to have some type, but at runtime it has type mypy.types.DeletedType
mypy.types.DeletedType means that your var is NULL (you deleted var (del var) or never assigned it)
https://github.com/python/mypy/blob/master/mypy/types.py#L1135-L1139
i think it is very similar to UnboundLocalError
mypy/types.py lines 1135 to 1139
class DeletedType(ProperType):
"""Type of deleted variables.
These can be used as lvalues but not rvalues.
"""```
Is there a way to solve this without type: ignore? I'm using conditional imports as my code supports 3.7+ but needs typing.Literal which was added in 3.8. py try: # Python 3.8 and above from typing import Literal except ImportError: # pragma: no cover # Python 3.7 and below from typing_extensions import Literal # error: Incompatible import of "Literal" (imported name has type "typing_extensions._SpecialForm", local name has type "typing._SpecialForm") [misc] my requirements look like this, if it's relevant```typing_extensions>=3.10.0; python_version < "3.8"
@ me if responding, thanks
use sys.version_info >= (3, 8) and skip try/except statements completely
That works I suppose
Just always import from typing_extensions, it internally checks versions and then just re-exports the typing object.
Should we depend on typing_extension's implementation details?
Use sys.version_info checks
Works perfectly ๐
I mean it's not really an implementation detail, the point is that Literal is available regardless of Python version from typing_extensions.
It'll just not define its own copy in new enough versions.
Depending on the constraints on typing_extensions Python's version might be better
And different as well
Yep, so just import from there always.
If you look at my requirements, I don't always require typing_extensions if the python version has the features i need
That's a super old version of typing extensions btw, currently it is at 4.3.0
so it might not be installed
Ah yeah. I'd just unconditionally depend on it.
I'm aware, Literal was added in 3.10.0 so I allow any version after and including that
It's for a package, so I'm being as lenient as possible on the requirements
You are aware that placing a constraint like that will not allow the next major version ryt?
Your package can cause incompatibilities with other packages
No, ~= requires the same major version, >= accepts any version afterwards.
=3.10 also means <4.0
it does...?
Oh yea
k that's what i thought ```
โฏ pip install "typing_extensions>=3.10.0"
Requirement already satisfied: typing_extensions>=3.10.0 in ./venv/lib/python3.10/site-packages (4.3.0)
pip version resolver is messed up pretty bad
Here's the actual spec:
https://peps.python.org/pep-0440/#version-specifiers
I had an incompatibility with sphinx and flake8 today
I don't have too many issues with it, except it sometimes takes forever on legacy python versions
It does have issues
that's usually an issue with the libs themselves
Lot of them, to be called stable yet
A lib defining its constraints is not a lib problem
Its the package manager's responsibility to sort out such situations, like npm for example does
A lib not being able to handle the most recent version of a requirement is a lib problem
By suffixing version number next to dependency
since it can cause issues with dependency resolution with other libs
That can't be said for sure, it can be a compatibility issue for one
Pretty much what I meant
That's why venvs are needed everywhere
npm has it's own onslaught of problems. in fact, i spent a whole day trying to fix some react dependency issues
The best way probably is hiw deno handles deps
Just pull from github
Eliminate package handling entirely
oh but I need to delete the var so it doesn't show up in the traceback reference cycle
commenting out line 326 doesn't help me (that's the self = None line)
Guys, for example, i have this abstractions
class AbstractValue(abc.ABC):
@abc.abstractmethod
def some_method(self) -> int:
...
class AbstractCollection(abc.ABC):
@abc.abstractmethod
def first_value(self) -> AbstractValue:
...
@abc.abstractmethod
def add_value(self, value: AbstractValue) -> None:
...
And extended abstraction implementations:
class ExtendedValueImpl(AbstractValue):
def some_method(self) -> None:
...
def extended_method(self) -> int:
return 1 # For tests
class ExtendedCollectionImpl(AbstractCollection):
def __init__(self) -> None:
self._values: list[AbstractValue] = []
def first_value(self) -> AbstractValue:
return self._values[0]
def add_value(self, value: AbstractValue) -> None:
self._values.append(value)
def extended_method(self) -> int:
return 2 # For tests
And class, that accepts Value and Collection types:
class SomeClass:
def __init__(
self,
*,
value_len: int,
value_cls: typing.Type[AbstractValue],
collection_cls: typing.Type[AbstractCollection],
) -> None:
self._value_len = value_len
self._value_cls = value_cls
self._collection_cls = collection_cls
def get_collection(self) -> AbstractCollection:
coll = self._collection_cls()
for _ in range(self._value_len):
coll.add_value(self._value_cls())
return coll
Then we will test SomeClass for mypy errors:
self = SomeClass(
value_len=10, # Any value for example
value_cls=ExtendedValueImpl, # Flexible
collection_cls=ExtendedCollectionImpl, # Flexible
)
collection = self.get_collection()
collection.add_value(ExtendedValueImpl()) # Ok
meth_callback = collection.extended_method() # "AbstractCollection" has no attribute "extended_method"
assert meth_callback == 2 # It works, only type-hint error.
collection.first_value().some_method() # Ok
meth_callback = collection.first_value().extended_method() # "AbstractValue" has no attribute "extended_method"
assert meth_callback == 1 # It works, only type-hint error.
Is this a violation of some principle?
I want to make it clear at the types level that extended implementations have an extended_method method, but I don't know how to implement this in annotations
I thought that this could be done somehow using typing.TypeVar, but I didnโt succeed
you've defined the type of get_collection to just return AbstractCollection, this ABC doesn't specify that an extended_method should be present
do you expect all AbstractCollection classes to have this method?
so, what you want is just a typevar?
so that the get_collection function will actually return an instance of the collection class you passed in?
you'll want to make this class generic then
so implementations must fully comply with the abstraction and not have additional methods?
they don't have to, the typing system will just downcast them into the lower type
so if you have a specific type that you casted as the abstract type that's fine
but you can't access a method of that specific type
the typing system now no longer knows this extended method exists, because you downcasted the variable to a type that doesn't specify this existence
class Animal():
...
class Cat(Animal):
def meow():
...
def foo(x: Animal):
x.meow() # Error animal can't meow, even though `x` is a Cat, the typing system doesn't know that here
c = Cat()
foo(c) # valid, foo function takes any Animal, can be a cat
you can solve this with a generic class though
by making SomeClass as generic?
from typing import Generic, TypeVar
T = TypeVar("T", bound=Animal)
class SomeClass[Generic[T]]:
def __init__(self, x: Type[T]) -> None:
self.x = x
def get_animal(self) -> T:
return self.x()
consider this example for instance
in here, the SomeClass is generic over a typevar that is bound to Animal class
that means you can't pass in say a str, it only accepts variables of Animal type (or more specific types, like Cat)
the advantage of type-vars here is that they get bound to the specific type though, rather than it getting downcasted to the less specific type
so if you do ```py
a = SomeClass(Cat)
x = a.get_animal()
x.meow() # valid, we know x is a Cat now
this is because type-wise a is now SomeClass[Cat] not just SomeClass
in your example, it looks like you'll actually need 2 typevars, one for the collection, and the other class for the value one
to do that, you can use this syntax: ```py
T = TypeVar("T")
K = TypeVar("K")
class Foo(Generic[T, K]):
...
@rustic gull If you have a general Python question, see #โ๏ฝhow-to-get-help and claim a help channel.
i have something like the following:
class Base:
pass
class A(Base):
pass
class B(Base):
pass
i'd like to write down the type of a dictionary like {'a': A, 'b': B}. i tried dict[str, Base] but it doesn't work, obviously, because the values are class objects, not instances of Base. is there a way to name the type "class objects that inherit from Base"?
Make Base inherit from abc.ABC so it's not instantiable on its own
ah, thanks
they're constant
looking at TypedDict, i still need to write down the name for "class objects that inherit from Base", right?
Yes the same thing still applies
But if you know the exact type of each value then that's more precise ๐
ah, actually dict[str,Base] works fine and i just had a mistake in the class definitions, nevermind
thanks!
well, TypedDict can still make it easier for users!
What do you all think about overloads?
I think I've asked this before - surely - but the more I use them the less I want to
They are completely unwrappable, which is extremely common. The end result is that my code becomes less typed because of type: ignore compared to had I joined it all together and potentially not prohibited a weird corner case
What do you mean unwrappable?
you can't use it with AnyFunc = TypeVar("AnyFunc", bound=Callable[..., Any])?
or ParamSpec?
Hi when writing a code with typing
Which one do you mostly do? Need readable code too
from typing import List
a : List = []
Or
import typing
a : typing.List = []
I don't have much experience with putting typing in my projects and im starting to do so. Need some suggestions which one do you mostly do most of time.
Nevermind ill just use the first one because in 3.9 you don't need to import it
I would import things directly from typing, otherwise the types get really messy
typing.Optional[typing.Callable[[typing.Union[int, typing.Iterable[int]]], typing.Any]]
Optional[Callable[[Union[int, Iterable[int]]], Any]]
the usual convention is from typing import
but yeah, since 3.9 you need to use list and such
some people also do import typing as t
!pypi flake8-pep585
๐ minor plug
I see thanks.
is lisp very fun
yes
import typing as t
import typing_extensions as te
Do you use one for collections.abc ? ca feels weird
I don't use collections.abc
technically the types in collections.abc are aliases for the ones in typing
or they're effectively aliases
the other way around
isn't collections.abc not generic?
i.e. you can't do collections.abc.Mapping[str, str]
They're generic since 3.9 (I think?), but they were always just generic aliases to collections stuff
since 3.9 they are generic, and their friends from typing are deprecated
๐ v2
Why is Callable in collections?
cause a Callable is a duck typeable class like the other things in collections.abc?
it's not an ABC though
oh wait
it is an ABC!
but pyright says no
@overload
def func(a: int) -> int:
...
@overload
def func(a: str) -> str:
...
def func(a: Union[int, str]) -> Union[int, str]:
return a
@overload
def wrapper(a: int) -> int:
...
@overload
def wrapper(a: str) -> str:
...
def wrapper(a: Union[int, str]) -> Union[int, str]:
return func(a) # Error
You need another decorator that changes the signature to the input
I am not sure I am following, but this also applies to when you for example only allow certain mixes of kwargs
@overload
def func(*, a: Optional[bool] = None) -> None:
return
@overload
def func(*, b: Optional[bool] = None) -> None:
return
def func(*, a: Optional[bool] = None, b: Optional[bool] = None) -> None:
return
@overload
def wrapper(*, a: Optional[bool] = None) -> None:
return
@overload
def wrapper(*, b: Optional[bool] = None) -> None:
return
def wrapper(*, a: Optional[bool] = None, b: Optional[bool] = None) -> None:
return func(a=a, b=b) # Error
def copy_params(src: Callable[P, R]):
def decorator(func: Callable[..., Any]) -> Callable[P, R]:
return func
return decorator
@copy_params(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
That problem isn't unique to overloads
How to i resolve this error from pylance?
sounds like you forgot to import Any
(also, Sequence[Any] already includes str so you could remove str)
That did the trick!
Thx
I am not talking about the issue of repeating parameter typings, I am talking about the error that I get inside of wrapper
hm...
would it make more sense to use typing.cast for this?
oh never mind, i totally misread your code
what's the issue?
does that return func actually work?
i would have expected that you do need to typing.cast it to the specific callable type
I think it should, ..., Any should meet everything
Yeah, Any and Unknown always implies a cast
probably pointless to even specify it
from typing import Any, Callable, ParamSpec, TypeVar
P = ParamSpec('P')
R = TypeVar('R')
def copy_signature(src: Callable[P, R]):
def decorator(func: Callable[..., Any]) -> Callable[P, R]:
return func
return decorator
def func(x: float, y: float) -> float: ...
@copy_signature(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
huh, that's useful
could you link to a message where you described the issue @rare scarab there seem to have been a lot of conversation in between, or perhaps quickly explaining what's the problem
i think bluenix started it off by saying that overloads were "unwrappable" and killjoy demonstrated the above as being a general-purpose solution
I see
from typing import Any, Callable, ParamSpec, TypeVar, overload
P = ParamSpec('P')
R = TypeVar('R')
def copy_signature(src: Callable[P, R]):
def decorator(func: Callable[..., Any]) -> Callable[P, R]:
return func
return decorator
@overload
def func(x: str, y: str) -> str: ...
@overload
def func(x: int, y: int) -> int: ...
def func(x: str | int, y: str | int) -> str | int: ...
@copy_signature(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
e.g here's an example of it working
and @rare scarab i had no idea you could implement @copy_signature so easily. i've been wanting this for a really really long time so thank you a ton for showing it
yeah, ParamSpec is a relatively recent addition
It's really just a typing hack
i knew about ParamSpec but my complaint was always that you couldn't "copy" a ParamSpec from one function to another
You could probably even use TypeVar("AnyFunc", bound=Callable[..., Any])
but this hack does the job. i don't even think using type narrowing here is a hack!
where would you put that?
oh instead of copying the P and R separately
Use that typevar instead of Callable[P, R]
most type checkers support type vars like that
couldn't you? I'm pretty sure I did something similar a while before already
I don't recall paramspec changing somehow to add this
there's no copy_signature in the stdlib and i never thought to do it this way ๐
ah, that makes sense, but it was possible since paramspec was added
you can copy it with functools.wraps, but static type checkers wouldn't see it.
you will lose docstring there though, unless you incorporate functools.wraps internally
right. also im running this in mypy-play and getting revealed types of Any, so there is still some debugging to do here
wraps doesn't copy doc?
let me iterate down to the bottom of this and post
wraps does, but copy_signature doesn't
with just copy_signature you wouldn't actually even have the copied parameters in help
type checker would know about them, but on runtime, you wouldn't see them
The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.
you do apparently need the cast() with the TypeVar version
that's interesting
and the revealed type isn't quite right in the non-TypeVar version
the TypeVar seems to preserve the overload while the Callable[P, R] doesn't. possibly mypy bug?
Callable alone shouldn't require you to cast if you're returning a more specific callable
yeah i'm not sure what's going on here, i'm not educated enough on how it all works internally
hmm, that's interesting, but I'm not sure that's actually a bug, considering there's something like Concatenate which can't respect overloads anyway
say with something like this: ```py
def copy_signature(src: Callable[P, R]):
def decorator(func: Callable[..., Any]) -> Callable[Concatenate[int, P], R]:
...
return decorator
PEP 612 unfortunately didn't contemplate how ParamSpec would work with overloads, so different type checkers treat it differently
i see. so maybe the TypeVar version is more robust in general, at least until that issue is fixed/standardized
interesting
pyright just straight up fails
it simply won't allow overloaded functions to be passed into callables with paramspecs
yeah, though that might take another PEP until it's resolved, so that it could be standardized everywhere
makes sense. but it's still really useful to know that the TypeVar technique works in all cases
i wonder if this is implemented in decorator and/or wrapt already
Interesting, in TypeScript an overloaded function is more or less an intersection between all overload cases
which does make sense
oh, that's interesting yeah
tbh most typecheckers already represent some kind of Overload[...] generic. Might as well fully add it at that point
# pyright
# warning: Unsupported escape sequence in string literal (reportInvalidStringEscapeSequence)
"\/"
What's wrong with this?
you need to use a actual escape sequence. \/ will become \/ because there is no / escape.
okay but it's for regex so I think its fine
actually it works without it too so i guess ill remove it
Unknown is basically just object
Nevers already a thing
Are you sure that's not NoReturn?
Never is an alias to NoReturn in typecheckers
To me, Never is completely different from NoReturn.
why?
NoReturn means the function doesn't return normally. Never would be used to say "This signature is invalid"
i.e. used on an overload, no-args is invalid.
arent those 2 things the same?
this signature is invalid -> it will not return normally
NoReturn is for functions that actually don't return. like an infinite loop
My point is that Never (similar to typescript) would be used to tell you calling this function this way is probably an error.
NoReturn was just poorly named from the start
afaik it has always been the bottom type and doesnt have to be used in function return types
What would be something you use Never for if it's different from NoReturn?
the default is None
It would be more useful with conditional types.
if only ๐ข
@overload
def sum(*args: str) -> Never: pass
@overload
def sum(*args: T) -> T: pass
T = TypeVar('T')
def test(result: T | CustomError):
reveal_type(result) # T@test | CustomError
if isinstance(result, CustomError):
return ...
reveal_type(result) # object*
any way to get the second reveal_type to know that result : T?
cast it
T includes CustomError
it's object because it doesn't know what T is
i'm surprised it's not using some placeholder instead of object though
actually it does use a placeholder. i can't reproduce @prisma flower
from typing import TypeVar
class CustomError(Exception):
...
T = TypeVar('T')
def test(result: T | CustomError) -> T | None:
reveal_type(result)
if isinstance(result, CustomError):
return None
reveal_type(result)
return result
main.py:9: note: Revealed type is "Union[T`-1, __main__.CustomError]"
main.py:12: note: Revealed type is "T`-1"
Success: no issues found in 1 source file
The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.
i see, that appears to be a pyright-ism: https://pyright-playground.decorator-factory.su/?gzip=H4sIAOjFKWMC_3WOvQrCQBCE-32K7XIHmgcIaCNprQ5bOeJGDpK7Y3cjBnx4Y35AC6f8mPmYllOPOuYQ7xj6nFjRjZkungGazovgaRBNfc2c2NTPhrKGFG0FOKUsSwCHh21jCldYgBu1qCRqmGTotEKHr2-Pxf1xZucU6WNiepDvrtMPWjd2oqHFICGK-thsfPfjWV4w6cBxlv11rZ0FvAGKqoDm9QAAAA%3D%3D
the * probably has some meaning here
it means its inferred iirc
sounds like pyright is doing exactly the right thing then
yea I guess I expected pyright to be a little smarter about using the information it was previously given
it is being smart. it's saying that it can't infer anything about result because it doesn't know what T is.
if you set a bound= on T or had some other constraint involved, maybe it could help
sorry, I wasn't being clear. I meant I wanted it to know result : T and not result : object
looks like it's using object* is used to mean T - CustomError?
I'm creating a server that handles incoming and outgoing HTTP requests and responses - I'm using flask to handle inbound API requests and responses and requests to handle outbound.
If I wanted to type hint the respective Request and Response objects both libraries provide but didn't want to fully specify their paths, is there a good convention here? e.g.
RequestIn: TypeAlias = flask.wrappers.Request
RequestOut: TypeAlias = requests.Request
ResponseIn: TypeAlias = requests.Response
ResponseOut: TypeAlias = flask.wrappers.Response
I could of course just name them FlaskRequest and RequestsRequest but that seems... ugly lol
I was also considering the flask objects aliased as just APIRequest and APIResponse and then use the request objects without qualification.
That's one way, yes
Does python typing have a shorthand for None, T, or collection of T?
None | T | Sequence[T]
replace Sequence with Iterable, List, ... depending on what kind of collection it is
in earlier versions, typing.Union[None, T, Sequence[T]]
That's not a question related to type hinting. You can ask it in a help channel #โ๏ฝhow-to-get-help
class StructProp(PropBase[T], NamedPropMixin):
def __init__(self, *ids: EventEnum, prop: str | None = None, **kwds: Any):
super().__init__(*ids, **kwds)
NamedPropMixin.__init__(self, prop)
this works, but how would i make this work without getting PropBase[Unknown] errror:
class StructProp(NamedPropMixin, PropBase[T]):
def __init__(self, *ids: EventEnum, prop: str | None = None, **kwds: Any):
NamedPropMixin.__init__(self, prop)
PropBase.__init__(self, *ids, **kwds) # type unknown errors
this works perfectly except for the type errors
Why do I get this error? W2602: typing.final is not supported by all versions included in the py-version setting (using-final-decorator-in-unsupported-version)
For this code
import sys
if sys.version_info >= (3, 8):
from typing import final, Protocol, runtime_checkable
else:
from typing_extensions import final, Protocol, runtime_checkable
When running pylint under Python 3.7?
Final is from 3.8
Yes but I import it from typing_extension for 3.7
I have got no errors when I run unittests via tox
on 3.7
seems like a pyright bug to me
That's a pylint warning, not a pyright one
yea thats what i meant
Why do you want this though?
T | Sequence[T] is kinda tricky since you won't be able to differentiated between them at runtime
Can you show an example maybe?
Been staring at a problem for a while, can anyone point me in the right direction? I'm trying to work with the cls passed into a classmethod, and am trying to figure out if it's me or mypy. It's basically the issue described in https://github.com/python/mypy/issues/9183, where python and mypy think this is ok:
@classmethod
def generate_subclass(cls):
class Subclass(cls):
pass
return Subclass```
but as soon as you start adding type annotations it starts complaining that `Variable "cls" is not valid as a type` etc. Anyone see this before?
i just wouldnt have code that looks like this, the reason it wont complain is because mypy doesnt type check untyped functions
so just assumes cls is Any
I guess my question is more how can I annotate it
AFAICT it's an open issue. see https://github.com/python/mypy/issues/5865
Cool, that's where I ended up, was wondering if I'd missed anything - but I'll take that as permission to give up then ๐ thanks!
But why
Why am I trying to subclass a class from within itself? Long story short I need a class factory and I wanted to put it as a classmethod on the base class.
a class factory is a "but why" scenario in my opinion. unless you're dynamically generating bindings for something, then i guess it's unavoidable
Yeah it's a weird edge case library thing - I'm doing odd things like defining classes in yaml and loading base classes over http. I'm not saying it's a good idea ๐
i've found that __init_subclass__ has mostly obviated any need i have for metaclasses
most things can be done with init_subclass.
though with the class factory, implementing type.__new__ would be enough.
i think people used to think such things were good ideas in the 90s and early 00s
java?
log4j moment
Well, what if I want to decouple what my base class is? That's so extensible
right, at least use https
Hey guys, how can I hint that the generator has the same amount of elements as the size paramter
def sized_generator(x: Any, size: int) -> Sized[Iterable]: # Sized[Generator]
""" yield x n amount of times"""
for _ in range(size):
yield x
when we get proper async module loading, someone could make a http import handler
I don't think PEP 646 is powerful enough for that
You can't
Why do you want this?
Would this work ?
from typing import NewType
Length = NewType('Length', int)
def sized_generator(x: Any, size: Length) -> Iterable[Length]:
""" yield x n amount of times"""
for _ in range(size):
yield x
no, that means you get an iterator that produces Length objects
sorry Iterable
btw it works
it = sized_generator('a', 4)
next(it)
Out[84]: 'a'
next(it)
Out[85]: 'a'
next(it)
Out[86]: 'a'
next(it)
Out[87]: 'a'
oh so is there a way to add the length of an iterable to the type hint
it would work even without any typing information
no
why do you want this?
Heh yeah it sounds odd but makes more sense in context: it's loading manifests (in py and yaml) from remote git repositories to manage multi-project docker-compose deployments from a single configuration. There are obvious security issues but there are things you can do to mitigate it (private repos, pin to hashes etc).
So it's a bit weird, and classes subclassing themselves was something I've not done before, hence asking here if I was missing an obscure typing trick. I can move on happy now. Thanks for your input
just started experimenting with this kind of type hints
so im just trying to see what works and how I should go about it
well there's no way to add information about how many elements are there, it would be pretty hard for type-checkers to check that
also, there's generally no need to
ah yes, I see now that arithmetic using pep-646 was excluded. so you can't use it to create [T, T, T, T] from 4 and T
inb4 [T] * 4
In TypeScript that's definitely doable
in general, this isn't possible without something called "dependent types", which are currently an active area of research and development in computer science. the only way you can do something like this is if the size is specified as a literal number in the source code. iirc there is some work being done on that, e.g. when converting a statically-known tuple into an iterable.
Scala 3 is the most mainstream language with dependent types
It even looks like Python, with indentation for block building and so on.. ๐
...scala has dependent types?
hi I'm a beginner programmer, now i'm using the library 'imageioo' but not understand a think in its documentation.
LINK DOCUMENTATION :https://imageio.readthedocs.io/en/stable/_autosummary/imageio.v2.imwrite.html#imageio.v2.imwrite
TOPIC: imwrite with parameter kwargs
MY PROBLEM : i tried to find all the possible parameter , i found only this site : https://docs.opencv.org/3.4/d8/d6a/group__imgcodecs__flags.html#ga292d81be8d76901bff7988d18d2b42ac , but on stack overflow and another site in a code a see a different type of kwargs.
QUESTION: i want find all type of kwargs . some one have any suggest for this or another libraries?
As well as higher kinded types, existential types, refinement types and so forth
whoops discord dropped the reply
when refining the types, do you get a lot of bi-products
lots of coproducts too
ew
Use Annotated
from #help-potato
I have this code ```py
from typing import Annotated
class Class:
x: Annotated[list[int], 8]
y: Annotated[list[str], 4]
z: Annotated[list[float], 12]
and I would like shorten it to thispy
class Magic:
???
class Class:
x: Magic[int, 8]
y: Magic[str, 4]
z: Magic[float, 12]```
is this possible?
basically I want to alias Magic[x, y] to Annotated[list[x], y]
This works fine ```py
T = TypeVar('T')
K = TypeVar('K')
Magic = Tuple[List[T], K]
x: Magic[int, Literal[5]] = ([1, 2, 3], 5)
but if I try to use Annotated it breaks
Magic = Annotated[T, K]
how does that help?
then I would just have to use Magic[list[int], 5]
at that point I might as well just do Magic = Annotated
do stub files have a purpose for python only programs or is it for programs with c/c++ extensions mainly?
they can be used if you can't or don't want to add type annotations to the .py files
for example, type checkers ship stubs for the standard library (based on the typeshed project) even though much of the stdlib is in Python not C
is this a known mypy bug?
from typing import Callable, TypeVar
from typing_extensions import reveal_type
T1 = TypeVar("T1")
T2 = TypeVar("T2")
R = TypeVar("R")
def flip(func: Callable[[T1, T2], R]) -> Callable[[T2, T1], R]:
return lambda x, y: func(y, x)
def pair(a: T1, b: T2) -> tuple[T1, T2]:
return (a, b)
@flip
def flipped_pair(a: T1, b: T2) -> tuple[T1, T2]:
return (a, b)
reveal_type(pair)
reveal_type(flipped_pair)
indirect = flipped_pair(1, "a")
mypy type_bug.py
type_bug.py:23: note: Revealed type is "def [T1, T2] (a: T1`-1, b: T2`-2) -> Tuple[T1`-1, T2`-2]"
type_bug.py:24: note: Revealed type is "def (T2`-2, T1`-1) -> Tuple[T1`-1, T2`-2]"
type_bug.py:25: error: Argument 1 to "flipped_pair" has incompatible type "int"; expected "T2"
type_bug.py:25: error: Argument 2 to "flipped_pair" has incompatible type "str"; expected "T1"
Found 2 errors in 1 file (checked 1 source file)
I have this abstract super class:
class Endpoint(ABC):
...
@abstractmethod
async def endpoint(self) -> Coroutine:
raise NotImplementedError()
@property
def route(self) -> Route:
return Route(
path=self.path,
method=self.method,
endpoint=self.endpoint,
tag=self._name,
)
I want subclasses to implement endpoint but it should be up to them what arguments the method has (I only care about that it's a coroutine method in the superclass.
Fx the following should be valid.
from pogo_api.core.endpoint import Endpoint
class Upload(Endpoint):
async def endpoint(self, message: str) -> str:
sender = self.settings.title
return f"{sender} says: '{message}'."
I ended up "solving it" by:
class Endpoint(ABC):
...
endpoint: Callable[..., Coroutine]
...
Your issue here was that calling endpoint already returns a coroutine so the signature looks like () -> Coroutine[Coroutine] (if you skip the unimportant stuff)
There's some descriptor related types in the types module. Does any one of them relate to the descriptor protocol or the property object?
Currently I have made my own ROProperty and RWProperty generic protocols. But if something like that already exists in stdlib, I could use it instead
Also can't dataclass use the Annotated from PEP593 to replace dataclasses.field?
Those are descriptors, but they're mostly not usable from Python code. They're used to implement types in C - each stores function pointers or data offsets, and then uses the descriptor protocol to let you call those.
Dataclasses could use Annotated, but that would tie it explicitly to typing specifically - currently it's functional without importing or requiring you to use typing specifically.
What's the issue with stdlib's modules depending on each other?
If its make them more Pythonic?
generally it slows down all of cpython
and people who dont use typing then get slightly better performance
Hi, I don't understand a type hint thing.
I have this method:
def _exists_check(self, table: SlobyTable | TableName
The table can be SlobyTable:
class SlobyTable(TypedDict):
table_name: TableName
table: sql_table
and TableName(str):
TableName = str
When I try to use a dict:
test = api._exists_check(table={"test": CREATE_USER_DATA})
I getting this warning: Expected type 'SlobyTable | str', got 'dict[str, Any]' instead
what could be wrong? Thanks.
table should look like {"table_name": str, "table": (sql table)}
buy you're passing in {"test": something}
Oh boy, another pyright issue. It doesn't like using pydantic model types as lru_cache arguments.
TNode = TypeVar("TNode", bound=Node)
@cache
def resolve_node_type(cls: type[TNode]) -> Callable[..., Awaitable[TNode]]:
...
resolve_node = resolve_node_type(Node)
Argument of type "Type[Node]" cannot be assigned to parameter "args" of type "Hashable" in function "__call__"
"__hash__" is an incompatible type
Type "None" cannot be assigned to type "(self: Node) -> int"
But is it a pyright or pydantic issue?
probably pydantic
It works with mypy
mypy might not check this cause it doesnt havent dataclass transform support atm i think
Since mypy doesn't catch it, I might just leave it for now.
A dict of tuples that contain a str and a int is typehinted like dict[tuple[str, int]]
dict[str, int]
dict[str, tuple[str, int]]
alright
Thans
Thx*
Should i declare it like that and after assign the values? Or is it not needed at all
@soft matrix
I have a class AsyncIterObject which implements __aiter__ and __anext__ to itterate over a list and then reload some data from an api.
Which list and what Type that list is is defined by a class that inherits from AsyncIterObject.
Is there any way to correctly typehint that __anext__ so that editors pick up which type is returned?
Since its determined at runtime which list to itterate I am guessing that this would not be posible but maybe there is something funky possible with TypeVars or something
Yes, you'll need to make the class generic, and then inherit from a subscripted class:
from typing import Generic, TypeVar
T = TypeVar('T')
class AsyncIterObject(Generic[T]): # this indicates T is scoped to the class.
def __aiter__(self) -> "AsyncIterObject[T]": ... # or typing.Self if everything supports that.
def __anext__(self) -> T: ....
class SomeIterator(AsyncIterObject[float]):
...
How do you pass an empty parameter list into a variadic generic? It says it can't be empty at runtime
While this is allowed by pyright and the runtime
Foo[Unpack[tuple[()]]]
``` ๐ฅด
Does Foo[()] work?
great! does that still work when AsyncIteratorObject inherits from a different class itself? woueld that just mean multi inheritance?
It does. If that parent class also is generic over that same typevar, you'd need to re-subscript it again, to indicate the child is also generic and the parent uses that same var: class Sub(Parent[T], Generic[T]):.
!e
from typing_extensions import Unpack, TypeVarTuple
from typing import Generic
P = TypeVarTuple("P")
class Foo(Generic[Unpack[P]]):
...
Foo[()]
@trim tangle :warning: Your 3.11 eval job has completed with return code 0.
[No output]
Looking at the code, I think generic protocols won't work...
!e
from typing_extensions import Unpack, TypeVarTuple
from typing import Generic
P = TypeVarTuple("P")
class Foo(Generic[Unpack[P]]):
...
print(Foo[()])
@trim tangle :x: Your 3.10 eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 9, in <module>
003 | File "/usr/local/lib/python3.10/typing.py", line 312, in inner
004 | return func(*args, **kwds)
005 | File "/usr/local/lib/python3.10/typing.py", line 1328, in __class_getitem__
006 | raise TypeError(
007 | TypeError: Parameter list to Foo[...] cannot be empty
That's unfortunate
I guess I'll use this
Ah, it's that 3.10 Generic isn't aware of TypeVarTuple at all. Probably should submit an issue to typing_extensions, perhaps it should monkey-patch this support in?
Could make it a type alias at least.
Are it correct?py async def exist_articles(self, link:str, link_to_author:str) -> AsyncGenerator[bool]: exist = await self._db.session.query(Articles.id).filter_by( link=link, link_to_author=link_to_author).first() is not None yield bool(exist)
just to give feedback: it worked, thanks for your help!
what the better framework for make RPA?
RPA?
yes, i was using pyautogui but have much problems
What is RPA?
is an acronym for program automation
Probably they mistaked this channel for keyboard typing
yes, now that i saw
why doesn't the kwargs override base class kwargs type hint?
The base class of this class defines **kw: Any, I tried changing it to object just in case it catches the correct type but it didn't
I need to cast to correct types everywhere because of this
This sounds correct to me
The type of the attribute was set in the base class and was never changed.
Pyright would have to trace the new argument through to __init__ which might call infinitely many other functions
In fact changing the type might be breaking LSP here, because the base class could do
def do_something(self):
self._kv["keyNotPresentInInsertKw"] = {"๐คก"}
(Or this could happen in a function accepting any object of the base class)
And it would be 100% correct
Can you show more code/context?
class ModelBase(abc.ABC):
def __init__(self, **kw: Any):
self._kw = kw
class MultiEventModel(ModelBase):
def __init__(self, *events: AnyEvent, **kw: Any):
super().__init__(**kw)
class _InsertKW(TypedDict):
index: SupportsIndex
max_slots: int
params: NotRequired[list[_MixerParamsItem]]
class Insert(MultiEventModel):
def __init__(self, *events: AnyEvent, **kw: Unpack[_InsertKW]):
super().__init__(*events, **kw)
I tried to change all **kw: Any to **kw: object but it didn't detect the correct type in Insert when I typed self._kw below it call to super class ctor
@soft matrix @trim tangle
Do you agree that this method in ModelBase should pass type chexking?
def do_something(self):
self._kv["keyNotPresentInInsertKw"] = {"๐คก"}
Unfortunately it seems like you can't make a TypeVar bound to TypedDict
I will not do anything in ModelBase
type hinting is already complicated for these bits and pieces :((
Why does the class even exist? Just set _kw directly
It serves as a base class for other subclasses
But why does it exist? What does it add?
it serves mainly for type hinting and an abstractmethod
You could remove the __init__ from the ABC
How would you type a multiprocessing.Pool parameter? Please ping me if you reply
import multiprocessing as mp
with mp.Pool(processes=2) as pool:
foo(pool)
def foo(pool: ???):
...
oh i didn't realise it was self._kw
mp.Pool should work
T = TypeVar('T')
class Wrapper(Generic[T]):
obj: T
def __init__(self, obj: T = None) -> None:
self.obj = obj
error: Incompatible default for argument "obj" (default has type "None", argument has type "T") [assignment]
Found 2 errors in 1 file (checked 1 source file)
is there a workaround for this? i was hoping that mypy would be able to figure out when T is or isn't None
i also tried TypeVar('T', , bound=Any | None) but it didn't seem to change anything
you need overloads
ah thanks. i just found this answer on SO which i think is exactly my question https://stackoverflow.com/q/64858261/2954547
will that actually set T to be None though? it seems like now i have to overload the entire class, so to speak, in order to work with either T or None
you do self: Wrapper[None] on the None overload
oh
or just don't have the default
missed that detail
oh and T will have to be covariant if you want the overload thing
otherwise you'll only be able to have Wrapper[None] and not Wrapper[None|str]
that doesn't seem to be necessary in my case: https://mypy-play.net/?mypy=latest&python=3.10&flags=show-error-codes%2Cstrict&gist=3f14bad29e24938e01011f40c34d48b5
or am i just not seeing where the problem might arise?
The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.
I think I don't get what the point of the thing is
oh ic
well, why do you need the default?
sounds kinda silly to print None innit
it's for when you don't actually want to print None, but you still want to construct some arbitrary repr or str
with python's lambda syntax this is already such a verbose clusterfuck that i figured it'd be easier if you didn't have to pass in the first argument if you didn't need it
but maybe in that case it'd be better to just have a different class altogether, that calls a function of zero parameters
Why do you even need this to be generic?
Just accept str and repr to be Callable[[], str]
_str = str
_repr = repr
class LogWrapper:
def __init__(self, str: Callable[[], _str], repr: Callable[[], _str]) -> None:
self._str = str
self._repr = repr
def __str__(self):
return self._str()
def repr(self):
return self._repr()
!pypi lazy-string
well heck
i don't mind 0 stars when i can audit the code in 3 minutes because it's 80 lines
(modulo supply chain attacks and/or compromised author)
true, they don't have tests anyway
i gave em a star
21 projects on github use it!
Hello! I have got a question related to type hints. #help-chestnut message
(I have asked the question in one of the help channels, but as this channel is specifically made for type hints, I figured i would ask it here as well) : )
Does anyone know if there's a clean way to type hint for a variable being of any subclass and not the parent class? For example:
class A:
...
class B(A):
...
class C(A):
...
Class D(C):
...
I want to accept B, C, D, but not A
mark it as an abstract base class with abc.ABC
Unless you want A to be instantiable on its own
Why do you want this?
Hi all, I'm new to typing but struggling with this:
from typing import List
class Grain:
def __init__(self, position, size):
self.position = position
self.size = size
class GrainWithColour(Grain):
def __init__(self, position, size, colour):
super().__init__(position, size)
self.colour = colour
class GrainsCollection:
def __init__(self, grains_list: List[Grain] = None):
if grains_list is not None:
self.add_grains(grains_list)
else:
self._grains: List[Grain] = []
@property
def grains(self) -> List[Grain]:
return self._grains
def add_grain(self, grain: Grain):
if grain in self.grains:
raise ValueError("Grain already exists in the list!")
else:
self._grains.append(grain)
def add_grains(self, list_of_grains: List[Grain]):
for grain in list_of_grains:
self.add_grain(grain)
class GrainsCollectionWithColour(GrainsCollection):
def __init__(self, grains_list: List[GrainWithColour] = None):
super().__init__(grains_list)
If I run this with mypy I get the following error message:
error: Argument 1 to "__init__" of "GrainsCollection" has incompatible type "Optional[List[GrainWithColour]]"; expected "Optional[List[Grain]]"
how can I effectively restrict GrainsCollectionWithColour to only accept GrainWithColour types, but still utilise the methods under GrainsCollection?
you could make the collection generic on the list grain type, then use that for the subclass
None is not a valid List
So you'll need to do ```py
grains_list: Optional[List[Grain]] = None
Optional[T] just means Union[None, T]
oh wait, it's not it
(but it's a good idea anyway)
Do you know about variance?
Basically I'm using A as a "sieve" class: when A is instantiated with certain vars (of themselves varying numbers/types), instead of actually instantiating A, it will instantiate an appropriate B, C, etc.
The underlying reason is a bit much to get into, but the short version is that the library I'm writing may need to change its API very quickly, and each subclass has very different end-user functionality. It could be that what would once be a C would instead become a D, implementing the same basic methods as A but having different outcomes.
My best thought right now is that I'd just create a new dummy class, UsableA, and each B, C, etc all also inherit from UsableA, and I let that be the type
is there any way to use Literal.__args__ in a type safe way?
I don't want A to be instantiable on its own, but not all of its subclasses will need to re-implement every method from A, so AFAIK that means ABC wouldn't be useful
so kinda like pathlib.Path?
I'd just make a function like ```py
def make_a(...) -> B | C | D:
...
I think I've done that:
from abc import ABC, abstractmethod
from typing import List, TypeVar, Generic
import numpy as np
class Grain(ABC):
def __init__(self, position: np.ndarray, size: float) -> None:
self.position = position
self.size = size
class GrainWithColour(Grain):
def __init__(self, position: np.ndarray, size: float, colour: str) -> None:
super().__init__(position, size)
self.colour = colour
GrainTypeVar = TypeVar('GrainTypeVar', bound=Grain)
class GrainsCollection(ABC, Generic[GrainTypeVar]):
def __init__(self, grains_list: List[GrainTypeVar] = None) -> None:
if grains_list is not None:
self.add_grains(grains_list)
else:
self._grains: List[GrainTypeVar] = []
@property
def grains(self) -> List[GrainTypeVar]:
return self._grains
@abstractmethod
def add_grain(self, grain: GrainTypeVar):
if grain in self.grains:
raise ValueError("Grain already exists in the list!")
else:
self._grains.append(grain)
@abstractmethod
def add_grains(self, grains_list: List[GrainTypeVar]):
for grain in grains_list:
self.add_grain(grain)
class GrainsCollectionWithColour(GrainsCollection[GrainWithColour]):
def __init__(self, grains_list: List[GrainWithColour] = None) -> None:
super().__init__(grains_list)
@abstractmethod
def add_grain(self, grain: GrainWithColour):
if not isinstance(grain, GrainWithColour):
raise TypeError("Must be a GrainWithColour!")
super().add_grain(grain)
@abstractmethod
def add_grains(self, grains_list: List[GrainWithColour]):
super().add_grains(grains_list)
@void panther I have a problem though - what if I want to make multiple subclasses of GrainsCollectionWithColour
how do I make that generic too?
the inheritance I want looks like GrainsCollection -> GrainsCollectionWithColour -> [GrainsCollectionWithColourAndShape, GrainsCollectionWithColourAndSize]
if that makes sense
I guess I need a GrainWithColourTypeVar
Use Protocols to define the actual methods and attributes which will be present?
with my package being marked as typed, and having py.typed, should I be expected to be deprecating the type names if I rename something internal? Say: ```py
Internal class, not a part of my package's public API
Users shouldn't initilize this class themselves or subclass from it,
it's just here to hold the structured response so that people can
access things from it, like say Response.name
class Response:
...
Function that I expect people to be using, part of the public API
def foo() -> Response:
...
``` and I wanted to rename the Response class to something else, as it's just something internal, however because it's directly returned from the public foo function, I suppose people may be relying on it in their type-hints now, so my question is whether I'm expected to now do something like Response: TypeAlias = RenamedResponse and keep that for some deprecation period. The biggest annoyance here is that I don't think it's even possible to produce a deprecation warning on the use of this type-alias. How do other typed libraries handle this?
theres a typing issue somewhere about adding a typing.Deprecated or something but that never went anywhere
@twilit badge Have you considered marking the return type with a Protocol that implements what you want your users to know about the Response type?
if it's part of a public function's interface, then it's not really internal ๐
Yeah, I thought about that, maybe that is a way to go about it
I mean, fair point, but the internal logic in this class is expected to change often, and I wouldn't want users to use the class directly themselves, the exposed interface i.e. the attributes it has, etc. does stay consistent, or goes through deprecation periods, but all of the internal stuff, including initialization is not expected to be used
Then Olivia's suggestion is good
How do I type a wildcard version of a variadic generic?
Like, I have a Foo[*P] and I want to specify a type that accepts Foo[()], Foo[Any], Foo[Any, Any] and so on
Foo[*tuple[Any, ...]]?
class KeyMap(Widget):
def __init__(self, keymap: Mapping[str, Event[Unpack[tuple[Any, ...]]]], target: Widget) -> None:
super().__init__()
self._keymap = {k.lower(): v for k, v in keymap.items()}
self._target = target
self.register(E_KEY, self._on_key)
def _on_key(self, key: str) -> None:
if control := self._keymap.get(key.lower()):
self._target.dispatch(control.key, control.payload)
# ^^^^^^^^^^^^^^^
``` error: ```
Argument of type "tuple[Any, ...]" cannot be assigned to parameter "payload" of type "tuple[*P@dispatch]" in function "dispatch"
TypeVarTuple cannot be bound to a tuple of unknown length
Event is:
@dataclass(frozen=True)
class Event(Generic[Unpack[P]]):
key: EventKey[Unpack[P]]
payload: tuple[Unpack[P]]
oh til
sounds like we're moving into # type: ignore city
yep
