#type-hinting
1 messages ยท Page 64 of 1
like this guy: https://en.wikipedia.org/wiki/Jelle_Zijlstra (my parents assure me I wasn't named after him)
@oblique urchin removing the callable check fails 10 tests
7 tests: checking that an int literal isn't a valid type
2 tests: checking that a list of tuples isn't a valid tuple for typeddict/namedtuple *kwargs
1 test: I don't understand yet
is that mypy?
typing.py itself
ah
@oblique urchin the 10th I think might be a bug in cpython combined with a test case that was testing a feature not actually implemented and was only working by accident
I'm still investigating whether or not the cpython behavior is buggy, but I'm confident the testcase is buggy either way
which testcase is it?
Why does this raise a TypeError? Probably not for the reason you think!
Lib/test/test_typing.py lines 227 to 230
def test_cannot_subclass_vars(self):
with self.assertRaises(TypeError):
class V(TypeVar('T')):
pass```
class Base:
def __init__(self, *args):
for arg in args:
print(repr(arg))
pass
b = Base()
class D(b):
pass
this outputs
'D'
(<__main__.Base object at 0x7f38e9c44730>,)
{'__module__': '__main__', '__qualname__': 'D'}
I totally expected that to tell me that I was passing an invalid constraint to the TypeVar
so this raises a TypeError because TypeVar.__init__ is called with that output as *args, and those don't pass the callable() check in _type_check, which TypeVar runs on the *args passed to it because it expects them to be constraints
so this test is testing that collectively each of these
(<__main__.Base object at 0x7f38e9c44730>,)
{'__module__': '__main__', '__qualname__': 'D'}```
do not pass `_type_check`
which is why removing callable() causes that testcase to no longer raise a TypeError
it seems highly unlikely this is the codepath the author of the testcase thought they were testing
and frankly that cpython behavior is bizarre, remove *args from __init__ and the D class initialization error is different whether or not you set the metclass=type
yes, I guess it's because creating a subclass of X basically just does type(X)(name, bases, ns)
this passes without errors for example class X(map(len, [])): pass
it doesn't? ```>>> type(TypeVar("T"))("T", (TypeVar("T"),), {})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jelle/py/cpython/Lib/typing.py", line 816, in init
self.constraints = tuple(_type_check(t, msg) for t in constraints)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jelle/py/cpython/Lib/typing.py", line 816, in <genexpr>
self.constraints = tuple(_type_check(t, msg) for t in constraints)
^^^^^^^^^^^^^^^^^^^
File "/Users/jelle/py/cpython/Lib/typing.py", line 184, in _type_check
raise TypeError(f"{msg} Got {arg!r:.100}.")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: TypeVar(name, constraint, ...): constraints must be types. Got (~T,).
I mean in the toy code with D and stuff; I thought type("D", (b,), {}) would be same as
class D(b): pass (when the *args is removed), but they raise different runtime errors
right, because it delegates to the metaclass of b in order to create the type
oh, what's the correct way to do it the same then
I think I've confused myself on the type() dynamic creation then I don't do it often
the way I did it in my sample above. In the general case there's complexity around picking the right metaclass and calling __mro_entries__
I see, so type(b)("D", (b,), {})
okay I understand now
anyway, I really doubt this is what the testcase was intended to test, and we understand it now anyway
We could maybe make the TypeVar thing work better if we override __mro_entries__ like @soft matrix did in https://github.com/python/cpython/pull/30268
>>> help(types.new_class)
Help on function new_class in module types:
new_class(name, bases=(), kwds=None, exec_body=None)
Create a class object dynamically using the appropriate metaclass.
!doc types.new_class
types.new_class(name, bases=(), kwds=None, exec_body=None)```
Creates a class object dynamically using the appropriate metaclass.
The first three arguments are the components that make up a class definition header: the class name, the base classes (in order), the keyword arguments (such as `metaclass`).
The *exec\_body* argument is a callback that is used to populate the freshly created class namespace. It should accept the class namespace as its sole argument and update the namespace directly with the class contents. If no callback is provided, it has the same effect as passing in `lambda ns: None`.
New in version 3.3.
so:
removing the callable check fails 10 tests:
7 tests: checking that an int literal isn't a valid type
2 tests: checking that a list of tuples isn't a valid tuple for typeddict/namedtuple *kwargs
1 test: TypeVar instance subclassing testcase that (presumably accidentally) tested that some literals and dict instances weren't callable. I agree with you that this can/should be fixed with the __mro_entries__
leaving aside the int literals for now, what is your opinion on these @oblique urchin https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_typing.py#L4262-L4263
Lib/test/test_typing.py lines 4262 to 4263
with self.assertRaises(TypeError):
NamedTuple('Emp', fields=[('name', str), ('id', int)])```
this isn't obvious what's happening; it's expecting something like name=str in the kwargs
so NamedTuple is passing the RHS to _type_check
so this test raises a TypeError because the literal [('name', str), ('id', int)] doesn't pass callable()
I'd be OK leaving that to a static type checker to complain about
there were 2 test cases like that, and that's it
@oblique urchin okay I propose
-
I open a bpo bug about the TypeVar testcase and propose adding mro_entries to TypeVar to fix the testcase/give it a better error; the testcase clearly isn't testing what it thinks it is
-
open a bpo bug and/or typing ML discussion on if we can remove the callable check in type_check, pointing out that it's :
a) both a source of existing bugs in cpython that are hard to catch (can require both introspection or uncommon stuff like using Annotated) and likely to continue to be the cause of bugs in the future
b) doesn't make any particular technical sense to require
c) makes an awkward (and very non-obvious) requirement to add__call__method to all typeform classes
and then bring up the stuff it affects
sounds good!
@oblique urchin this is meant to be cls and not self isn't it? https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/typing.py#L352
Lib/typing.py line 352
def __init_subclass__(self, /, *args, **kwds):```
not that it changes anything
yeah the first argument to __init_subclass__ should normally be called cls
maybe I make PR without bpo for this as a typo fix
@oblique urchin skimming the source of typing.py I can identify other things that are bugged/impossible only because of the callable check
from typing import Literal, Annotated, ForwardRef, TypeVar, ParamSpec, Callable
import logging
T = TypeVar('T')
P = ParamSpec('P')
def add_logging(f: Callable[P, T]) -> Callable[P, T]:
'''A type-safe decorator to add logging to a function.'''
def inner(*args: Annotated[P.args, "meta"], **kwargs: P.kwargs) -> T:
logging.info(f'{f.__name__} was called')
return f(*args, **kwargs)
return inner
@add_logging
def add_two(x: float, y: float) -> float:
'''Add two numbers together.'''
return x + y
can't annotate paramspec; why? ParamSpecArgs isn't callable()
oh, nice catch
this is jsut what I found from a very quick skim auditing typing.py for callable() compatibility
I've probably missed stuff
I'll make issues for stuff
@oblique urchin would we really need the stuff gobot did or can't we just raise an error?
where would you raise the error?
you could probably just have raise TypeError right?
def mro_entries:
raise
oh yes, that's right. you don't need the indirection here
for NewType that helps give a better error but that's not necessary here since there's no obvious thing to suggest that the user meant to do
Oh no ahhh, Python 3.5 support dropped in 0.920 ๐. That doesn't support Python 3.10
Oops, I am talking about Mypy
cython issues i assume
ah
# May be a bit hard to read but essentially means:
# > Returns a callable that takes another callable with these parameters and *some*
# > return value, then returns another callable with the same parameters but the
# > the return type is the previous 'type' parameter.
def returns(__type: _T) -> Callable[[Callable[_P, object]], Callable[_P, _T]]: ...
@overload
def cast(__t: Callable[_P, _T], *args: _P.args, **kwargs: _P.kwargs) -> _T: ...
These are my primary use-cases right now
I will probably need to drop it, or use version_info checks.
why are you still supporting 3.5?
Cython supports everything from 2.7 to 3.10 ๐ฅด
why dont you do sys.version_info checks?
Yup, those are my options
@oblique urchin the self->cls thing seems like a silly thing to make a PR for, would it be uncouth to include it in another typing PR?
I think a separate PR is fine
Oh hey do you know if I need to guard imports and my _P = ParamSpec('_P')?
Mypy only errors on its usage though (when I put it inside Callable) ๐ค
@oblique urchin okay i assume other people would add news skip and stuff? https://github.com/python/cpython/pull/31135
no you shouldnt do
any way to do this properly
T = TypeVar("T")
S = TypeVar("S", bound=SomethingSubscritable)
def func(cls: type[S] = SomethingSubscritable) -> Callable[[T], S[T]]:
return lambda x: cls(x)
TypeVar "Type[S@func]" is not subscriptable
What do you want to do?
wait till HKT is a thing?
Can you show an example of how you'd use it?
Is SomethingSubscritable Generic?
That feature is called "Higher Kinded Types"
and Python's type system doesn't have it
trying to think if there's any reason someone would object to this being annotatable
seems clearly intended to be a type
any alternative to keep that pattern?
well, that's definitely something someone might want runtime introspection for I guess
write a pep for it ๐
@brisk heart Can you show an example of how you actually use it? Maybe there's an alternative, simpler solution?
its something people have wanted for a while unsurprisingly
yup but it will be a bit different to the example
sure
# user uses it like this
@deco(metadata="goes here", cls=Foo)
def callback(self, thing: Foo, interaction: Interaction):
...
# the actual types if fully annotated
@deco(metadata="goes here", cls=Foo)
def callback(self, thing: Foo[Self], interaction: Interaction):
...
kinda like this I suppose
the library provides a standard implementation of a component
Do you guys think I should guard it to 3.10 or 3.5 (the last version available to Mypy doesn't support it, newer versions has dropped support)? ParamSpec was introduced in 3.10 but only Mypy checks only fail on >3.5 ๐ค
users should have the option to change this component to their will
so as long as they subclass (or possibly implement a protocol?) they can use their own implementations
Can you show a more complete example?
you can use typing_extensions.ParamSpec for 3.6-3.9
and maybe 3.5 too, forgot when we dropped 3.5 support
sure lemme dig something up
3.5 i think sort of works but not for generics off the top of my head (on 3.10.x)
# before
class BasicView(miru.View):
@miru.button(label="Click me!", style=hikari.ButtonStyle.SUCCESS)
async def basic_button(self, button: miru.Button, ctx: miru.Context) -> None:
await ctx.respond("You clicked me!")
@miru.button(label="Stop me!", style=hikari.ButtonStyle.DANGER)
async def stop_button(self, button: miru.Button, ctx: miru.Context) -> None:
self.stop()
# after
class BasicView(miru.View):
@miru.button(cls=BetterButton, label="Click me!", style=hikari.ButtonStyle.SUCCESS)
async def basic_button(self, button: BetterButton, ctx: miru.Context) -> None:
await ctx.respond(f"You clicked me {button.total_clicks} times!")
@miru.button(label="Stop me!", style=hikari.ButtonStyle.DANGER)
async def stop_button(self, button: miru.Button, ctx: miru.Context) -> None:
self.stop()
maybe something like this?
this is the lib for reference https://github.com/hypergh/hikari-miru
This is for PYI files for Cython. I don't know if I can add that dependency. I think most people using these typings will typing_extensions so it might sort-of work?
not mine, I'm just a contributor
typing_extensions is assumed to be in the venv isnt it?
yep, as Gobot says. Type checkers assume typing-extensions always exists
You can get ParamSpec in an if TYPE_CHECKING on any version of Python afaik
So why do you need to subscript anything?
@oblique urchin https://bugs.python.org/issue46643 and https://bugs.python.org/issue46642
I will wait for agreement on https://bugs.python.org/issue46643 before doing anything, but also maybe we want to wait to see what the discussion will be around my suggestion to remove callable. I am making that issue next.
well because BetterButton can be subscripted with the view
so if you subclass the button you can just use it as a standalone thing
and reference the parent view with self.view
hmmm, won't mypy complain that it's not entire?
?
like it always complains on strict mode that I don't provide the subscript
that seems correct to me
I don't want to be bitten in the arse later
It absolutely is correct but now idk if it won't complain about the same thing
It does not currently but I can't tell if it will stay like that
so i did a bit of digging and ParamSpec supported in 3.5 just probably not very well at runtime
Wait I am confused how this will help me, changing the import to from typing_extensions import ParamSpec still fails on Python 3.5 where only Mypy 0.910 is available (lacking ParamSpec support).
Runtime does not matter to me as this is in a stub-file.
I guess it bundled an old version of typeshed that didn't have ParamSpec yet
I have the exact same problem on 3.9 btw
I just keep it in TYPE_CHECKING and it kinda just works out
Looking in my venv there's ParamSpec in the included typing_extensions file and this is in my typeshed. I thought Mypy just didn't support it up until after that?
that's typing, not typing-extensions
ParamSpec is referenced in typing-extensions as well:
venv35 is the venv (standard module) I created for my Python 3.5 installation
is it in mypy/typeshed though?
Ooops, that's the one Mypy specifies as a dependency. Here's the one in Mypy's vendored typeshed:
https://mypy-play.net/?mypy=0.910&python=3.5&gist=381ce86025691f11733d32a5c869a655 seems to let you import it
The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.
Yes, I have no issues importing it. What fails is when Mypy typechecks it, sorry if that wasn't clear: https://mypy-play.net/?mypy=0.910&python=3.5&gist=caefc60b95736940c450b832ccfb1817
The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.
oh yeah that's expected
do you really need to care about hypothetical users trying to type check their Cython code under Python 3.5?
Yeah that's what I am unsure of. Seeing as there is no other option available then I'll ask about it.
It's pretty reasonable to ask someone to use the latest update of their type checker, even though the code it's meant to run isn't gonna be under that version I'd say.
you can also just type ignore the definition where we use ParamSpec
that's what we did in typeshed for a while
mypy will just fall back to Any for those parameters
Wouldn't I loose type information for all versions then?
no only for the versions where it doesnt work with mypy
Ah, the type: ignore comment would be on the ParamSpec import?
no it would be on the def returns line
Ah okay, gotcha.
Honestly I have no idea what higher-kinded type vars are -- my eyes glaze over when I hear that kind of talk. :-)
Lmao, same, Guido
x)
@oblique urchin if you wanted to comment https://bugs.python.org/issue46644
now an actual test failed in https://github.com/python/cpython/pull/31135 but it's obviously nothing to do with my patch
lol why
I think macOS tests are just failing right now on the main cpython branch in general (maybe inconsistently); heh
it retriggers the tests, because the azure pipelines thing was from before the appropriate tags were added
ah
ahah I just saw this https://bugs.python.org/issue42353
dataclasses.py uses re.match and the regex starts with a ^
@oblique urchin I found another callable() bug
@oblique urchin https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_types.py#L834-L838 this doesn't test that union can't substitute parameters, this tests that int literals are not callable
Lib/test/test_types.py lines 834 to 838
def test_union_parameter_substitution_errors(self):
T = typing.TypeVar("T")
x = int | T
with self.assertRaises(TypeError):
x[42]```
if you change 42 to int then it no longer raises a typeerror
that might be what it's supposed to test
x[int] should work
ah, right
so this probably needs a real implementation not an incidental one
you know, if we wanted to prevent int literals we could easily just, you know, prevent int literals in _type_check instead
added to the bug this additional test
I'll wait for feedback before doing anything else
maybe just the mro thing
yeah some socket-related test is just failing on main periodically in general right now
I'm still not sure if there are people that would argue you shouldn't be able to annotate paramspec
to me it seems like you "obviously" should be able to
Looking at cpython main I'm not even sure what change if any caused test to start failing
anyone got an example of the most perfect code in the world?
like a github project or smtihng
That is impossible to define lol
๐ก
you know what i mean
like
most efficint
most neat
most pep-8 compliant
etc
It looks like maybe the problem with the test machines is resolved if you need to retrigger the tests or maybe it's fine
I will try to audit typing.py for additional bugs of that sort later
Since I found the bug very quickly by just directly examining the code I might be able to find more
And if we don't remove callable we will need to fix all of them anyway
And they would be worth test cases
What's the difference between typing.Iterable and typing.Iterator? 
Iterator has the __next__ method
Also they're both deprecated
Docs say they derive from abc
Iterable has iter but Iterator has iter and next, yeah
You want the versions in collections.abc
i nees some help i am getting an error whome should i ask ?
hey, new pyright release
Enhancement: Allow "--watch" to be used in conjunction with "--outputjson" command-line options.
I wonder if I could use this to speed up the playground, but sounds complicated
cpython main is very broken right now
๐คฃ
whether or not all your tests pass is luck
it's not obvious to me what commit broke the tests
I don't get this one at all
from typing import TypeVar
class Foo:
...
T = TypeVar("T", bound=Foo)
def func(cls: type[T] = Foo) -> T:
return cls()
main.py:8: error: Incompatible default for argument "cls" (default has type "Type[Foo]", argument has type "Type[T]")
Found 1 error in 1 file (checked 1 source file)
pyright understands it tho
this is a case that someone was proposing adding a new SpecialForm for but I don't see why https://github.com/python/mypy/issues/9773
Did anyone encounter confusions betwee type aliases, TypeVar and NewType?
I was initially confused as to when you'd need a type alias to a normal (unsubscripted) type and why you wouldn't just use NewType
you mean like ```py
Email = str
yeah
ah
yeah, there isn't much point to it
I have a coworker that does this, it's very annoying
surely that should be a new type
I don't see much need in NewType either to be quite honest
I guess it prevents one from mixing two strings together...
i think its alright in cases like this
(but you can also just do Email("not an email") in a module that doesn't own Email)
In Haskell they're useful because you can export the type Email but not export the constructor
I would rather have a proper Email object that does validation in its constructor, so that you can actually be sure that an Email object is valid
Isn't the example given stuff like currency?
Or- for example meters vs. foot
I found this explanation:
https://justincaustin.com/blog/python-typing-newtype
it seems to explain what the point of it is
Even though they're both integers, you cannot pass a meter where a method expects a foot.
Well they atleast to me are useful as sudo phantom types
NewType could prevent typos: https://github.com/RobertCraigie/prisma-client-py/issues/91
While the example I've used there is obvious there could be cases where it's not so obvious
This is also a niche use case
good point, values of different domains conceptually, but with the same representation at runtime
I've found NewType pretty useful
A class can feel a bit boilerplatey for a single piece of data
You can still do validation, you'd just have a function that takes a string and returns an Email
yeah
I guess it's more of a tool to hint at silly errors/typos, rather than providing air tight safety (which is more like Haskell's newtype)
I don't think I've ever used NewType. Maybe I should try it
we use it heavily for typed ids
Yeah it's great for integers that serve as IDs and can't be mixed
We also use them for strings that essentially serve as IDs
It provides value with basically zero cost
With a class you'd have more cost in boilerplate so it makes you hesitate more
It also makes function signatures communicate waaaaay more information
One qol use of it I found was for working with external languages inside python code such as SQL or graphql, you can make a gql = NewType('gql', str) wrap your queries and not only you get that 'typed' but you can configure your IDE for syntax and completion inside the newtype
How can you configure your IDE for that?
I configured it in pycharm some time ago, but it is somewhere in the section to add custom highlighting, let me see if I find it again
Should be under language injections
@oblique urchin I would be okay with adding a type(typ) is int: raise in place of the not callable(typ): raise, but I don't think it should be any more complicated than that
And even that is stretching it I think
it makes a lot more technical sense than the callable check though
or check typ.__class__ directly, don't know if one method would be more preferred
Not sure we even need the int check. I guess that's just the example people put in for testing
It doesn't make much sense to me to disallow int specifically
In my head the justification is "types are not int indexing" line of argument, but overall I agree with you that I'd rather remove it all
There is a risk that people get confused about list[int] vs. actual lists, but I'm not sure that risk is higher with ints
and besides list[] allows anything anyway
No feedback on the issue yet https://bugs.python.org/issue46644
I will draft the mro fix for that weird test in a little bit
I gave typing.py code another glance and there weren't any additional callable() issues I could find that immediately stood out other than the paramspec one I already identified
I haven't checked open typing PEP PRs or typing_extensions yet though
The fact that it's a pain to audit the code for this problem is an argument in it of itself to remove the check
@oblique urchin is sobolevn on irc or discord somewhere? I want to discuss a typing.py testing strategy with him casually if he would have me
this seems like a really good suggestion you dont wanna know how much stuff ive written has __call__ with raise NotImplementedError
Please, leave a comment on the bpo with this :)
@oblique urchin an underlying problem not directly related to this is that typing.py is missing comprehensive test coverage of _type_check reachable from the external API. The big ways to reach this in general are usually with get_type_hints, but also Annotated now and a few weird ways. I know sobolevn has been working a lot on improving typing.py test coverage so I didn't know if he would have thoughts (or ideas) on how to increase coverage in this area
Or maybe interested himself in it
he's on the core dev discord but not here, not sure about IRC. Maybe just email him?
I didn't even know there was a core dev discord, like a private one for triagers and core?
yes
you could probably email them and ask for them to accept your friend request ๐
๐
I will draft PRs soon
Hey @oblique urchin, I started writing what I threatened to write yesterday. Do you have a minute to take a look? It's about 3000 words
https://decorator-factory.github.io/typing-tips/
(or anyone else as well)
great! about to head out, I'll take a look tonight
https://decorator-factory.github.io/typing-tips/pitfalls/typevar-typealias-newtype/ the example stopspy def send_message(email: Email, message: str) -> None: bot.how will i know how to send my email now?????
it's morning here ๐
oops
thank you
No emails for you
this looks very nice from an initial glance
one completely unreasonable idea i just had would somehow be embedding pyright playground into this so you dont have to work with the screenshots and people can experiment with the code themselves
I thought about it
screenshots are definitely janky
you'd need a check for email being None too, right?
why?
the type says it can't be None
oh
right
oh i meant the variable in the 2nd block
does anyone here know why overloads on property dont work?
hm?
can you show an example?
from typing import Generic, overload, TypeVar
T = TypeVar("T")
class This(Generic[T]):
@property
@overload
def something(self: "This[int]") -> None:
...
@property
@overload
def something(self: "This[str]") -> bool:
...```something like this as far as im aware doesnt work
ah
but i might be wrong
If this has some complex logic, I'd make a method
it doesnt though
looks like that check was also being used to prevent tuple literals like this
# with self.assertRaises(TypeError):
# ClassVar[int, str]
if it has overload, it does ๐
re @soft matrix @oblique urchin
@property
def community_url(self) -> str | None: # TODO if defaults for TypeVars happen make this Generic over Literal[Type]
"""The SteamID's community url.
e.g https://steamcommunity.com/profiles/123456789.
"""
suffix = {
Type.Individual: "profiles",
Type.Clan: "gid",
}
try:
return f"https://steamcommunity.com/{suffix[self.type]}/{self.id64}"
except KeyError:
return None
@soft matrix Can you show more code?
i want to be able to have this know that community urls only exist for Individual or Clan accounts
i was just going through my pyright errors and was thinking this would be possible if the class was generic and overloads on properties worked (and id do it if defaults for TypeVars existed)
cause it makes the most sense to me to not have to do user.steam_id.community_url and instead just user.community_url
Can you explain what the class is supposed to do?
its the basic logic for how a steam id is supposed to be handled and all the things you can generate from one
What is a steam ID?
something like 76561198248053954
yes
What are "type", "universe", and "instance"?
its like a snowflake in discord terminology except it encodes more info
type is the type of account the id is associated with like a game server, user, clan
universe is for valve devs to screw around with (basically everything is Universe.Public)
instance is a number from like 1 to 3 and idk what its actually for
Invalid = 0 #: Used for invalid Steam IDs.
Individual = 1 #: Single user account.
Multiseat = 2 #: Multiseat (e.g. cybercafe) account.
GameServer = 3 #: Game server account.
AnonGameServer = 4 #: Anonymous game server account.
Pending = 5 #: Pending.
ContentServer = 6 #: Valve internal content server account.
Clan = 7 #: Steam clan.
Chat = 8 #: Steam group chat or lobby.
ConsoleUser = 9 #: Fake SteamID for local PSN account on PS3 or Live account on 360, etc.
AnonUser = 10 #: Anonymous user account. (Used to create an account or reset a password)
Max = 11 #: Max of 16 items in this field
So steam IDs are not just for users, but all sorts of things?
yep
Which of these types will be actually useful to the users of your library?
ContentServer, Pending, Max don't seem quite useful, but correct me
maybe there are 2-3 which are most useful?
Individual, GameServer, ContentServer, Clan, Chat
are the ones that currently inherit from SteamID
does this mean everything is variadic if you remove the check, or just class var?
they probably wouldnt really need themselves as they should be using the subclasses unless they were parsing an account id into the 64 bit form (76561198248053954) but i definitely need it
alright
so what are the differences between these accounts?
well they have different things associated with them GameServers have players and rules, Individuals have games and friends, ContentServers have files, Clans and Chat have members and text channels
- delete the
community_urlmethod fromSteamID - add this to
Individual:
def community_url(self) -> str:
return f"https://steamcommunity.com/profiles/{self.id64}"
- add this to
Clan:
def community_url(self) -> str:
return f"https://steamcommunity.com/gid/{self.id64}"
yeah but SteamID is meant to be like a raw version of all of the subclasses
Wdym "raw version"?
like some functions accept SteamID over a User/Clan/GameServer
You're already using subclassing to signify the type of an ID
They can keep accepting SteamID.
If they accept SteamID, they should be able to work with a User/Clan/GameServer
or i could just have```py
class SteamID(Generic[TypeT]):
@overload
@property
def community_url(self: SteamID[Type.Individual] | SteamID[Type.Clan]) -> str: ...
@overload
@propery
def community_url(self) -> None: ...
What is the point?
Why would I call a pure function that always returns None?
Why do you need a generic? Just have a subclass for each type
You are overengineering this
I still don't get why you need a generic?
well i dont if it raises
A community URL isn't something an ID has. A community URL is something an individual has, or a clan has
wow this is kinda gross
And an ID is also something an individual/clan/... has, not what it is
Is it even possible to make add support for tuple literals to typing in python (like, for some future undetermined purpose) without breaking runtime introspection?
hm?
the way i see it is its like discord.Object or Snowflake idr
def func() -> (int, str)?
like, imagine I do Final[(int, str)], but this is the same (at runtime!) as Final[int, str] which is disallowed, and there are runtime types that take one or multiple arguments
like this makes the introspection code a hot mess
Right, but a snowflake doesn't have a channel_url() -> str method or anything specific to each kind of object
unless you give up disallowing Final[int, str] at runtime
isn't typing introspection code a hot mess already?
but im trying to fix some of the hot mess right now
ah
I tried it, and it all seems very undocumented and messy
all these special attributes and crap
This restriction is usually met in typeform instances by implementing a
__call__method that raises at runtime
i... what
cheeses
Why is typing such a mess? I expected it to mostly be no-op stuff that can be subscripted?
yeah its gross
it used to be a lot worse
@soft matrix I think you can open an issue on Pyright and ask how to make an overloaded property
and now the msg argument doesn't do anything
but to be honest, I don't see a use for this in a well-designed system
im pretty sure it doesnt work with mypy either, the person who was typehinting all of discord.py had some cool ideas about how to make Command's callback property show the first required param Cog if it was in a Cog
@soft matrix @oblique urchin what is your opinion, should Final[str, int] be disallowed at runtime? we would have to disallow tuple literals as types. That's basically the only option
because types are used in __getitem__ and there's not a clean way to know if you passed a tuple or a comma separated list
i dont think it should be allowed if at runtime if any of the args are type checked at runtime
sorry, so you think we should disallow Final[str, int]?
I think we've basically already committed to disallowing tuples as a type
Union[typeform] -> typeform, but Union[int, str] -> Union[int, str]
yeah
so I can replace the callable check with a check for a tuple literal
Welcome to GitHub's home for real-time and historical data on system performance.
not a good start to the month for github
https://github.com/python/cpython/pull/31151 there's one more commit and the news that hasn't showed cause of github problems
amusingly the explicit tuple check entire coincidentally allows https://bugs.python.org/issue46642 to continue to pass
how to ship a python library to pypi with stubs, i generated stubs to typings/umago/ and added py.typed in root. but can't see the pyi on going to defination
in vs code
So did ya'll see why all the cpython tests are failing?
Apparently some of the tests relied on being able to send a POST to example.com (the actual webserver), and that stopped being reliable
I'd love to see the logs of random requests on example.com's webserver
Where can I read about the Never type? I haven't found it anywhere
was there a pep for it?
and how is it different from NoReturn?
I found this in Pyright release notes:
Enhancement: Added support for
typing_extensions.Neverandtyping.Nevertype and unified the underlying handling ofNeverandNoReturn, which are synonyms.
wait... what's the point of adding a synonym?
seems like a strange change
it is not yet in typing, https://github.com/python/cpython/pull/30842
there wasn't a PEP (I think), there was some discussion in typing-sig https://mail.python.org/archives/list/typing-sig@python.org/thread/CGIYKGMF6XLDX3F2JNRCCE7HC6K2XLZ2/
having two bottom types is kinda confusing... I don't get what the difference is yet
yeah I didn't really get it either
the only reason I see is that NoReturn is a bad name in places other than return type of function
I agree that it's a confusing name
and renaming it to Never would be breaking
I think at some point it is gonna be removed
They're intended to be equivalent, it's that NoReturn is a bad name in those other cases.
yeah but I still think NoReturn should be removed cause there should be one obvious way to do it
"there should be one obvious way to do it" and "backwards compatibility" are in a bit of a clash ๐
it can be deprecated
right
it should've been "There should be one obvious way to do it, but we are humans, and humans are imperfect. So if we find a new way that's significantly better, we will deprecate the old obvious way."
or maybe the real thing that's happening is that people are becoming more Dutch?..
let us propose a more nuanced Zen of Python that is invoked via import this.depends
depends on what you want to do with that response
If you are only parsing it later then, if you know the structure of the json ahead of time you can make a TypedDict or just return dict[str, Any]
TypedDict
Hey, does anyone know why you cant alias Annotated in mypy? I tried multiple ways but mypy never accepts it
The only was I found around that was to from typing import Annotated as Foo
Depends on your type checker. In Pyright, you can do py Json = Union[int, str, float, bool, None, list["Json"], dict[str, Json]] mypy doesn't support recursive type aliases yet, so ```py
Json = Union[int, str, float, bool, None, list[object], dict[str, object]]
But I'm not sure how useful the type is, unless what you want is precisely "any JSON value"
Since you probably want to parse this afterwards anyway, I'd just use object
yes
@soft matrix oof I might have gotten your instance subclassing PR killed by accident ;(
Sorry
I don't think I can make a better argument than I already did that the error is worthwhile @soft matrix :\
The NewType change is likely more valuable because subclassing NewType by accident is more likely
No need to jump to saying it will be rejected, just give it time
ok
Interesting
That would be a helpful feature
Would really help with the issue of having to duplicate parameters in a child class if you want to have good typing
How do annotate forwarding kwargs
def foo(a: int = 1, b: str = ''):
...
def bar(**kwargs):
foo(**kwargs)
currently you can't maybe see https://github.com/microsoft/pyright/issues/3002?
ParamSpec?
ParamSpec could not work out how to bind it to a function that was wasnt a func argument
The coincidence x)
What's better, using ```python
from typing import List
mylist: List[int] = [1,2]orpython
mylist: list[int] = [1,2]```
I know there isn't really any big difference but is there any minor difference between these two?
If you are coding a library and what to be mindful of older versions of python use List, otherwise it is deprecated
at this point with 3.6 being deprecated, id always go with list[int]
There is virtually no difference to the type checker, and at runtime the only difference is that the __annotations__ will have a typing.List object instead of list
oh alright, thanks
It was added in 3.9 though
3.7+ allows you to use the future import to ignore the error it'd produce
list[int] is the way to do it if you're not tied to a version less than 3.9. typing.List and friends are deprecated and will be removed in some future version.
!pep 585
^ see for more info
(btw, if you want to catch these, there's https://pypi.org/project/flake8-pep585 ๐ )
great, thanks!
wait I got two notifications for this
I reordered the messages
.bm
ah
shameless plug moment
let's also not pay attention to that fact that I'm always watching all messages here either
ty
gratz @soft matrix
thanks :)
After https://github.com/python/cpython/pull/31203 merges I'm going to go ahead and write the patch to fix the annotated forms
Accomplished by adding eq methods in a manner that mirrors similar methods in other parts of typing.py.
What is everyone's thoughts on truthiness?
https://github.com/microsoft/pyright/issues/3006
https://github.com/quora/pyanalyze/blob/master/pyanalyze/boolability.py. in pyanalyze I made if Iterable be an error
my coworkers have been asking for even stricter checks, like making if int an error
le huh
I do dislike implicit boolean checks
that would be interesting.
what about or and friends though?
it is basically an implicit if
I largely use the same behavior for those
you disable all coercing to bools?
no, currently it's just for types that are always truthy (like Iterable)
I wish there was a way to write flake8 rules like that. But alas, it has no knowledge of types
the int thing produced a ton of false positives, I may explore it again at some point though
well, [] is an iterable that is not truthy ๐ค
but the check is questionable, with that I agree
im curious, whats the alternative parser to libcst, since libcst doesn't support match statements afaik
it does
!pypi libcst using this personally to check errors in code
i have no clue
I'm definitely against all truthiness, personally, at a language feature level. Pointless feature IMHO that optimizes writing and pessimizes reading.
oh.
once a language already has it, and many consider it idiomatic, the question is should you still go against the grain
they need to update their docs
naming 100 ๐
but I dont' love if len(x) != 0 either
or did they...
x wasn't the actual variable name ๐
what about a isempty method? a lot of langs seem to have them
I'd like to be able to write x.is_empty and x.is_not_empty
0.4.1 does, 0.4.0 does not.
oh, jinx
yeah, was going to say, Kotlin has it.
Even better would be if Kotlin did it as a property, which it doesn't for literally no reason
saves you ()
it's a cool example of extension properties, because you can obviously implement is_not_empty in terms of is_empty, so you only ahve to implement the former once and then it's available on all collections
you could also simply empty is_empty in terms of .size == 0 since all Collections have to have a cheaply-available size attribute
If you made a method that returned an Option, as if empty or not would that not still lead to the same issue?
what do you mean?
If Iterable should return an Option, are you talking about readability?
Nvm, just read again, never thought about the consequences of using truthiness checks on variables such as int and iterables
yeah, truthiness gets dicey when nesting is involved
dicey/confusing, maybe I should say
suddenly you have all these things that can become bool and since you'renot being explicit, you can choose the wrong one very easily and you won't get complaints
though you can have similar issues, at a smaller scale, in languages that opt for the null/none model instead of the optional/maybe model
even being relatively explicit
i don't mind truthy/falsy as long as it's generally consistent
like javascript isn't... [] is truthy but '' is falsy
why?!?!
people's custom __bool__ implementations... might or might not be good
although i agree, an explicit .is_empty() function would be nice @terse sky
although idk if we really need it to be a property, seems unpythonic
I actually have barely used properties in python
They're rather painful to write compared to their equivalents in other languages IMHO
So I'm not even sure what the criteria is
Typically, reading a property should be non mutating and super cheap
is_empty would seem to fit this
the general recommendations that i see are that you usually shouldn't use properties except for compatibility with interfaces that only specify attributes
Yeah maybe this comes back to their awkardness
i think it's more about explicit vs implicit
why would you hide a function call behind not-a-function-call?
mypy recently gained an (disabled by default) trick along these lines https://mypy.readthedocs.io/en/stable/error_code_list2.html#check-that-expression-is-not-implicitly-true-in-boolean-context-truthy-bool
if it's popular, we could maybe enable by default in the future
Well it's all function calls really
But I know what you mean
In the end it always can be hidden behind a function call anyway, and it's really not any of the users business :-) as long as principle of least surprise isn't violated
is it? i mean yeah technically you can override __getattribute__ but
i'm not sure what the benefit is to save a ()
if my_list.is_empty():
...
i guess?
i think the general school of thought is that len itself can be abstracted to some extent with __len__ and len(my_list) == 0 is more or less the same
might be nice if Sequence had .is_empty as a mixin method
Except len(my_list) == 0, at least traditionally, is fiscouraged
yeah, the general advice is to prefer the truthy check
which is honestly kinda ๐ฅด
fwiw pep 8 is like... old
really old
i suppose that there might be a good use case for "empty" that isn't governed by len()
in which case you'd maybe want 2 different methods
but... meh
there is definitely a good argument for there being a .length() or .size() method instead of the toplevel len() + .__len__ method
but i think len()+.__len__() is because __len__() was added later
Does anyone what decides how a TypeVar is displayed? Specifically what makes it plus and what makes it minus. I couldn't really find any mention of it anywhere
T`1 T`-1 T`-2 T`2
that's just some mypy internal logic
Oh, I just wanted to know if there is any structure to it
probably, but to figure it out you'll have to read mypy source code ๐
Humm, I think I kinda got what determines the the sign. I wouldnt know where to start on reading the source code for that so I will live with not knowing about the numbers x)
T = TypeVar('T')
K = TypeVar('K')
V = TypeVar('V')
class M(Mapping[T, K]):
@classmethod
def foo(cls, v: V, t: T) -> K: ...
@staticmethod
def bar(v: V) -> K: ...
def m(t: T, k: K) -> T: ...
main.py:18: note: Revealed type is "def [T, K] () -> __main__.M[T`1, K`2]"
main.py:19: note: Revealed type is "def [T, K, V] (v: V`71, t: T`1) -> K`2"
main.py:20: note: Revealed type is "def [T, K, V] (v: V`-1) -> K`2"
main.py:21: note: Revealed type is "def [T, K] (t: T`-1, k: K`-2) -> T`-1"
One time I opened mypy source code,
and then I closed it
This seems useful. Will catch things like
class User:
def is_admin(self) -> bool: ...
@property
def is_active(self) -> bool: ...
def has_access_to_launching_missiles(user: User) -> bool:
if not user.is_active:
return False
if not user.is_admin: # !!!!!!!!!!!!!!!
return False
...
how to typehint kwargs in a Callable?
make a protocol
you need to use a callback protocol eg
class Callback(Protocol):
def __call__(self, *, kwarg_types: Something) -> ReturnType: ...
yep
ideally one day ```py
(str, x: int, y: float) -> bool
Wasn't there something about using TypedDict instead?
eric traut is thinking about a proposal, I think
its not mentioned in the draft pep
That would be for typing **kwargs. It wouldn't help Callable types
Ahh I see
Callable[[**SomeTypedDict], T] i dont see why that wouldnt work
Though I guess we could make Callable[[Unpack[TD]], ...] work
your variant would be a syntax error ๐
i thought they were updating syntax
oh wait that means supporting kwargs in getitem :))))
I feel like that syntax change would be a harder sell
ig thats not gonna happen then lol
I think it might work if there's a stronger motivation. Seems like PEP 639 was mostly rejected because the argument wasn't strong enough
Maybe with a compelling typing use case it would go through
But Callable[[**SomeTypedDict], T] isn't even indexing, it's ** in a list literal
So that would be yet another new syntax
oh yeah
But shouldn't unpack be equivalent to * according to 646?
Single star unpack
Callable[[*TD], T] == Callable[[Unpack[TD]], T]
Not double star
oh yeah that's true, python.org/dev/peps/pep-0646/#type-variable-tuples-with-callable
This could be rather confusing if you could use Unpack for both variadict and a typeddict
and the consequently * on both
:)))))
Yess
That would've been quite nice, was it deferred or completely rejected
it was completely rejected cause it was too hard to teach or something along with only benefiting typing checking
Numpy could've probably gotten a use?

it's pretty useful in R
The rejection was a bit weird to me, the PEP only really mentioned type checking use cases as an aside, but then the rejection notice was basically all about typing
@oblique urchin what happened to the pep for using +/- for TypeVar variance?
The author has been silent for a while
oh that sucks, hope they are well
linter allows Callable[[*param_types], return_type] but not Callable[param_types, return_type] even tho they both work why??
param_types is typed as list and is one
It is just part of the Callable syntax, your first argument has to either be a list of types or a typing.ParamSpec
๐ it even allows Callable[[int], int] but not Callable[int, int] even tho both work and is same, it always wants the first one to be enclosed in square brackets irrelevant of its type?
and by list I mean literal list, not a variable of type list
ooh
it is so you can differentiate the argument and the return type
Maybe... Or it is for implementation, either way x)
maybe they forgot to update the typehint ๐
What you mean?
allow lists and obj of instance type to be passed as first argument??
cos both works proper
If it works at runtime that doesn't mean a static type checker should accept it
If by working you mean it doesnt cause issue at runtime, than anything works
as long as it is valid python syntax
it doesnt work with collections.abc.Callable at runtime
ooh
By that I just meat that there is nothing that there is nothing that should stop the Callable syntax from being implemented as Callable[arg1, arg2, ..., retunr_type]
but it just isnt what was decided as
yeah but its checking and converting Callable[int, int] to Callable[[int], int] so then why its not specified in the typehint?
What is "it"? What type hint?
for Callable
Callable in typing is the newer one right?
!d collections.abc.Callable
class collections.abc.Callable```
ABC for classes that provide the `__call__()` method.
is the one you want
oh
not typing.Callable cause its deprecated past 3.9.2(???)
yep, typing.Callable is deprecated by PEP 585
In [5]: collections.abc.Callable[int, str]
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Input In [5], in <module>
----> 1 collections.abc.Callable[int, str]
File /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/_collections_abc.py:436, in _CallableGenericAlias.__new__(cls, origin, args)
434 args = (*t_args, t_result)
435 elif not _is_param_expr(t_args):
--> 436 raise TypeError(f"Expected a list of types, an ellipsis, "
437 f"ParamSpec, or Concatenate. Got {t_args}")
438 return super().__new__(cls, origin, args)
TypeError: Expected a list of types, an ellipsis, ParamSpec, or Concatenate. Got <class 'int'>
```and this then doesnt work with [int, int] at runtime
oh
because it should be Callable[[int], str]
why is the typing version and collections.abc different?
Because first there were the collections, for user to subclass
Maybe you could add something about these imports actually being deprecated, not just adding an alternative?
(also I have a compulsion to plug flake8-pep585
)
when support for typing was being added there was no support for class subscription, eg "Class[]"
so then typing.Callable, typing.List and so on were created for the use of type ckeckers
later on support for class subscription was added retroactively to the abcs, hance no more need for the typing.X ones
Oh
So while typing.X are technically newer, they are deprecated xD
note that some things in typing are not deprecated, e.g. typing.Any, because they don't have equivalents elsewhere
oh yes, not everything in typing is deprecated
although deprecating Any would be a power move lol
i wish they would have moved typing.NamedTuple to collections
"now we're in Haskell mode, kids"
i really should start using pyright, not having a properly recursive JSON type really makes type annotations difficult to use for "real" data
how does haskell handle that? some language extension for heterogeneous collections?
i know heterogeneous collections are possible in idris 2 but i admittedly don't know how they're implemented
In Haskell it's easy
data JSON = JList [JSON] | JDict (Map String JSON) | JStr String | JInt Integer | JNull
using pyright with JSON still means you have to type ignore every line cause you arent handling each case
ah right, that should have been obvious
is that so?
as in, you have to make asserts about the types before taking out any data point?
oh thats why my program broke
isinstance(Callable, Callable)
True
assuming you have typeCheckingMode = "basic"
that's the same as in haskell though, you'd have to explicitly pattern match
I tend to agree with this. You still need to parse the data somehow.
"parse don't validate" is a really really important paradigm in functional programming with static types
yeah
parseUserInfo :: JSON -> Either InvalidUserObject User
parseUserInfo :: JSON -> Either InvalidUserObject User
parseUserInfo JDict userData = ...
parseUserInfo JList _ = Left $ mkInvalidUserObject [JsonWrongType JList]
parseUserInfo JStr = Left $ mkInvalidUserObject [JsonWrongType JStr]
parseUserInfo JInt = Left $ mkInvalidUserObject [JsonWrongType JInt]
parseUserInfo JNull = Left $ mkInvalidUserObject [JsonWrongType JNull]
i guess
idk if that's even valid haskell but that's the idea
well, this is going to be abstracted by some nice parsing library with some monad shit, but yeah
right
at which point it becomes opaque and impossible to understand for 99% of programmers ๐
Just to make sure:
- if I specify
TypeVar("Foo", A, B, C), it can hold a type that is either a subtype ofA, a subtype ofBor a subtype ofC. But I can't assignUnion[A, B] - if I specify
TypeVar("Foo", bound=Union[A, B])
right?
hey, I dropped out of college on 1st year and I was able to understand aeson
yeah but some people have an aptitude of this kind of thing
dropping out of school and still being "successful" isn't the norm!
anyway, being forced to do this imo is a Good Thing, but it's alien to a lot of python devs. i think ocaml is great here, ultra-abstracted stuff isn't that common and people tend to use more explicit pattern matching
oh I just meant that I'm not very bright, generally
Lol I started to learn about them yesterday and I think very much fit with the majority
The TypeVar is resolved to one of A, B, and C
but it's not subtypes, right? i always found that a strange design decision
oh so it has to be exactly A, not a subtype of A?
You can still pass a subtype of the resolution
i've found that doesn't work in the past
๐ ฑ๏ธruh I'm writing a tutorial on type variables but I don't know how they work
amazing
TypeVar("T", A, B) can be either A or B, including subclasses. and Typevar("T", Union[A, B]) can be either A, B, Union[A, B], including subclasses
(no)
I am getting confused now
what about TypeVar("T", bound=Union[A, B])
I didnt even know you could bind to an Union
ah I forgot the bound, I think Typevar("T", Union[A, B]) is disallowed
as in the *args has to be length 2 or more
let me see
valid solutions for that are Union[A, B], A, B, or any subtype of those
that's right
TypeVars aren't very well specified in general though
Hmmmm seems like you are not right
from typing import TypeVar, Generic
class A: ...
class B: ...
class A2(A): ...
T = TypeVar("T", A, B)
class F(Generic[T]): ...
Fa = F[A]
Fb = F[B]
Fa2 = F[A2] # no error
Fi = F[int] # error
That still solves the TypeVar to A
Hmmmmm that's weird
x = F[A2](A2())
reveal_type(x) # F[A2]
x = F(A2())
reveal_type(x) # F[A]
``` is this a bug in pyright?
that was my impression as well
i think so
so what should the right behaviour be?
it should be F[A]
the only allowed solutions for T are A and B
wait what? https://mypy-play.net/?mypy=latest&python=3.10&gist=515f9fbca187bc9086edaf0232a3521b this definitely used to not work
The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.
from typing import TypeVar
class Base1: pass
class Base2: pass
class Thing1A(Base1): pass
class Thing1B(Base1): pass
class Thing2A(Base2): pass
class Thing2B(Base2): pass
TheBase = TypeVar('TheBase', Base1, Base2)
def f(x: TheBase) -> TheBase:
return x
f(Thing1A())
f(Thing2B())
this solves so so many problems i had with bound=Union[Base1, Base2]
really, it solves them?
I feel like those kinds of typevars mostly create problems ๐
i remember i tried using bound=Union and it didn't really work, lots of inference and variance problems
i think they have a lot of valid use cases, e.g. a function that operates on either str or bytes and returns the same
and a lot of things related to numpy and pandas
wait in the screenshot what's bad about x being assigned to MyStr instead of str?
MyStr is a subclass of str
the typevar gets resolved to str and not MyStr
yeah, but why not let it be resolved to MyStr?
related ?: If TypeVar("T", bound=Union[str, bytes]) was used in screenshot, then it would be resolved to MyStr right?
yes
Gist
but then the body of the function wouldn't typecheck
why doesn't it want every subtype of Union[str, bytes] to be addable with every other subtype, instead of the particular subtype chosen and itself?
so only MyStr + MyStr being needed in this case, not str + AnyStr and bytes + AnyStr
because it's a generic function
!e that makes sense ```py
class MyStr(str): ...
print(type(MyStr("h") + MyStr("i")))```
@boreal ingot :white_check_mark: Your eval job has completed with return code 0.
<class 'str'>
looks like we found a mypy bug
x and y have the same type
and a pyright bug
the function declaration guarantees it
double kill
but it should error in this case?
in which case?
but reveal_type(x) should only look at the function signature of concat, isnt it?
but you cant concat bytes and str
I think quicknir is right, should error
but you can't deduce to bytes + str
I was referring to <#type-hinting message>
both arguments are always the same type
that said, the function annotation is still wrong
ah that, sorry
but mypy doesnt know that because it is bound to an union
oh wait, the mypy one is right. I missed that @halcyon nexus's example used bound=Union
theoretically if mypy or pyright looked at the call sight then it might not error in above case
it cant resolve
the return type from teh body isn't really compatible with the annotation
but just looking at declaration site it is not always allowed
it only resolves if it is TypeVar('T', Text, bytes)
the return type annotation is saying that the exact type that's passed in is what's returned
however, the bound is not sufficient to guarantee that
otherwise it is agnostic as to what the other T in the argument is no?
having str as an upperbound for a type T means that adding two T's returns a str
not that adding two T's returns a T
it might even mean that adding a T and a str returns a str, in fact
at runtime why does this give a str instead of MyStr?
because that is how __add__ on strings are implemented
right.
you might imagine it looks like py def __add__(self, other): return str(self.value + other.value) just subclassing that will still have it return str
inheritance only works as you'd expect on the first argument; trying to extend inherited arguments into generics, you're going to have problems
because a subclass of str might not have a one-arg constructor? so it'd be sometimes impossible to construct an instance of the subclass after concatenation?
subclass of str are only guaranteed to have the same API as string, on the more derived type
in the MyStr case they could just do type(self)(self.value + other.value) but I see how that would not generalize
__add__ takes a str and returns a str
so if all you know about a type T is that it's bounded by str, you can't guarantee that adding two T's gives you a T
this might be relevant to this false negative (that was fixed) in pyright: https://github.com/microsoft/pyright/issues/2363
You can only guarantee that T + str -> str
assuming the Self annotation existed in the past, why not define __add__ as (self: Self, other: Self) -> Self
instead of (self: Self, other: str) -> str?
because it would be wrong
ah I see
You don't really want to do T = TypeVar('T', upper_bound=Union[str, bytes])
python doesn't provide a "string-like" interface but in the end that's what you want
I guess the easiest way is probably to define a generic protocol, if that would work
Humm, so in this example
T = TypeVar('T', Text, bytes)
def concat(x: T, y: T) -> T: return x + y
class MyStr(str): ...
a = concat(MyStr(), MyStr())
The reason it is fine is because T resolves to str right?
hmm is python able to handle this properly?
In e.g. Kotlin you can mention the type variable in the bound:
fun <T : Comparable<T>> T.coerceAtLeast(minimumValue: T): T
well, mypy just resolves a to str
You need this for interfaces that relate to binary operations
You want to say "I want a T comparable to other T's". In this case, you want a U that is addable to other U's.
How do you formulate this in python?
from typing import Protocol, Generic, TypeVar
T = TypeVar('T')
class Addable(Protocol, Generic[T]):
def __add__(self, t: T) -> T:
...
U = TypeVar('U', bound=Addable[U])
def foo(u1: U, u2: U) -> U:
return u1 + u2
quoting the U passed to Addable doesn't help
maybe a self type on Addable.__add__?
I mean, I think you just have to upper bound by the class you want, inst that what you are doing there?
I don't understand
I don't understand Self at all, but yes, maybe that would work. Shouldn't really need Self though.
You are saying you want (T, T) -> T exactly, but mypy just resolves to the common supertype among all T, but you want it to resolve to the exact same
I opened an issue about TypeVar https://github.com/microsoft/pyright/issues/3010
does it seem like a legit bug?
(I also initially submitted the issue with the name Incorrect like a complete idiot)
Well, I want to be able to define a type variable, where that variable can be passed to the constraint
this is supported in pretty much every constrained generics system, I think
T = TypeVar("T", A, B, contravariant=True) I have little idea what that even means :S
well... it seems legal, idk ๐คท
let me think if this makes sense for a bit...
No, it still doenst bound it
The mypy Playground is a web service that receives a Python program with type hints, runs mypy inside a sandbox, then returns the output.
ackchually I don't know
variance definitely doesn't do anything on generic functions
str.__add__ returns a str, so a would be str right?
it's only meaningful with generic classes
But then how do you make so (T, T) -> T has to have all T be the same type?
and not only share a common supertype
what's the difference?
you cannot
Don't subclass str and make your own add
Isnt that what quicknir was talking about or I am missunderstanding?
I am only here for the types hehe
came for the ducks, stayed for the types
or for the cats
No
definitely not for the pacmans
I was saying that in other languages, you can have generic interfaces
and you can pass the argument itself
the type argument itself, into the generic interface
this doesn't force exactly (T, T) -> T, derived types are fine
what's not fine is (T, str) -> str
You need this mechanism in order to have constraints that make sense on two argument functions. Like +, <, etc
As a simple example: how would you constrain a sort function that doesn't take a comparator?
I guess it would make sense if one constraint was a subtype of another. Not sure
So def concat(x: T, y: T) -> T: where you pass MyStr to both x, and y and then get a str as a return is fine?
No
It should be None here since MyStr returns None when you try to instantiate it.
if T will be instantiated to str, then yes
yeah I was thinking in that direction too
Right, it depends what T is
no it wont?
but in this case neither is a subtype of the other, so idk tbh
Converting to "int" failed for parameter "pep_number".
!e
class MyStr(str): ...
pirnt(type(MyStr()))
@hearty shell :x: Your eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 2, in <module>
003 | NameError: name 'pirnt' is not defined. Did you mean: 'print'?
Now I'm curious what python's type stubs look like for sort
I had a typo my bad
stdlib/builtins.pyi line 820
@overload```
!e
class MyStr(str): ...
print(type(MyStr() + MyStr()))```
@spiral fjord :white_check_mark: Your eval job has completed with return code 0.
<class 'str'>
now I have to find SupportsRichComparison ๐
Yeah I think it's not completely precise
its a _typeshed type i think
stdlib/_typeshed/__init__.pyi line 54
SupportsRichComparison = Union[SupportsDunderLT, SupportsDunderGT]```
how do you do that so fast ๐
chrome url bar autocomplete ๐
Yes, all I was saying is that mypy is alright with substituting (T, T) -> T to (str, str) -> str when you pass MyStr to both arguments. Which is alright, but I thought quicknir was saying that shouldnt be right. I just missunderstood xD
Okay, so we can see that the way python does this is hot garbage
class SupportsDunderLT(Protocol):
def __lt__(self, __other: Any) -> bool: ...
where is your statically typed god now
whats wrong with that?
it doesn't check that the members of the list are comparable to each other
^
just that they're comparable to something
def nietzsche():
del god # (c)
oh
you shouldn't need to fall back to Any, in order to constrain sorting a list
see and this is why we need a Not type
from typing import TypeVar, Generic
class A: ...
class B: ...
class AB(A, B): ...
T = TypeVar("T", A, B)
class F(Generic[T]):
def __init__(self, thing: T) -> None:
self.thing: T = thing
f = F(AB())
reveal_type(f)
reveal_type(f.thing)
``` Is it defined anywhere that `f` should be `F[A]` and not `F[B]`?
what does Not have to do with it?
evil
the way this is solved in basically all languages is with a generic constraint
but it seems like in python, you can't pass the type argument back to the generic constraint, making this useless. Maybe it works better with another type checker?
Here's Kotlin's sort:
fun <T : Comparable<T>> MutableList<T>.sort()
Notice that T is being passed to Comparable
If you do T = TypeVar("T", B, A) it resolves to F[B] instead. But the inheritance order doesn't seem to matter
class list[T](Sequence[T]):
def sort(self: Self[T & SupportsCMP]) -> None: ...
```would this solve it?
is that... valid python?
no
I mean you can already write what is intended using valid python
so much of this doesnt work but i think you can guess what it means
mypy just refuses to handle it
from typing import Protocol, Generic, TypeVar
T = TypeVar('T')
class Addable(Protocol, Generic[T]):
def __add__(self, t: T) -> T:
...
U = TypeVar('U', bound=Addable[U])
def foo(u1: U, u2: U) -> U:
return u1 + u2
this example si with adding not comaprison but it's teh same concept
this is the spiritual equivalent to the Kotlin code I showed before
but mypy chokes on this U = TypeVar('U', bound=Addable[U])
quoting the U there doesn't help either
very odd, but if you drop the Text, bytes constaint it does give the MyStr type
not surprising I guess since it doesn't handle recursive types
if someone has pyright installed maybe they could try running that code example through pyright
thats not gonna work with pyright
Yeah, you can't use generic type aliases as a bound
That would be higher kinded types
pretty sure it's not higher kinded types
from typing import TypeVar, Text, Tuple
T = TypeVar('T', "MyStr", Text, bytes)
def as_tup(x: T, y: T) -> Tuple[T, T]: return x, y
class MyStr(str):...
a = as_tup(MyStr("a"), MyStr("a"))
reveal_type(a)```
Like this it does work, if you add the MyStr to the TypeVar
I guess it changed it to str because it was constrained to str or bytes
Java, Kotlin, etc, do not have higher kinded types, and they handle this perfectly, ergo this is not higher kinded types ๐ Without even getting into definitions
Higher kinded types?
Addable to itself would be
from typing import Protocol, Generic, TypeVar
from typing_extensions import Self
class Addable(Protocol):
def __add__(self, t: Self, /) -> Self:
...
T = TypeVar("T", bound=Addable)
def foo(u1: T, u2: T) -> T:
return u1 + u2
x = foo(1, 2)
``` @terse sky
Like, this is exactly the kind of thing that makes me overall meh on python static type system. It's very expressive, in some places it has all these super fancy features, and then in another places you find a very basic omission that prevents it from correctly typing sort
seems like the / is important
that's pretty strange (the /)
(typing has Self fyi)
o right
Well, (4).__add__(t=42) would indeed not work
only on 3.11 ๐
or without Self
from typing import Protocol, TypeVar
class Addable(Protocol):
def __add__(self: "T", t: "T", /) -> "T":
...
T = TypeVar("T", bound=Addable)
def foo(u1: T, u2: T) -> T:
return u1 + u2
x = foo(1, 2)
reveal_type(x)
3.11 also has the sweet callable type hint syntax
that PEP hasn't been accepted
and there's a serious chance it will be rejected
yeah it's in draft
God damnit, I got static for a second
damn, you couldn't accept self?
I build the demo on my work pc, planning on messing around with it a bit tomorrow
The Self PEP is accepted and implemented
the callable syntax one isn't
oh this was a pun response to Mathias
๐ข why so
like, you know. staticmethod
people don't like so much complicated new syntax
Hmph
I find it less complicated than Callable tbh
yeah, it's more complicated for the people maintaining the parser
Id be willing to sacrifice patma for it ๐ฅด
that's fair
Im disgusted that this is not a syntax error
This channel looks almost as bad as #esoteric-python . I am intrigued