type-hinting

[mortal fractal]

It doesn't know about the constraint brought about by x=x, so it errors because you can't + str and int

[trim tangle]

why?

[oblique urchin]

I feel like I only have so much time before Eric fixes the issue, and I didn't see the problem quickly

[tight cargo]

class SomeClass:
    def __init__(self, some_var: List[T]) -> None:
        self.some_var = some_var

    def get(self, value) -> T:
        return self.some_var[value]

is this correct? T is a TypeVar

[oblique urchin]

No, you need to include Generic[T] as a base class

[tight cargo]

Oh, so just...

class SomeClass(Generic[T]):
    def __init__(self, some_var: List[T]) -> None:
        self.some_var = some_var

    def get(self, value) -> T:
        return self.some_var[value]

the some_var will have same annotation?

[soft matrix]

yes

[tranquil turtle]

add some_var: List[T] to class body

[soft matrix]

and from the looks of things this could be covariant

[tranquil turtle]

and annotate value in get() as value: int

[trim tangle]

nah, List is not covariant

[oblique urchin]

this is not necessary with most type checkers, though I agree it's cleaner

[soft matrix]

no but like it could be since it seems to be a read only type of deal

[trim tangle]

well, the solution here is to probably use Sequence

[soft matrix]

yeah thats what i meant to change

[eager vessel]

I don't think you have to do that 🤔

type of self.some_var is already implied from __init__ parameter, you don't need to duplicate it in class body

[soft matrix]

yeah you dont have to

but pyright requires it

[eager vessel]

I didn't know that 🤔

[trim tangle]

hm?

[eager vessel]

Has i been deceived? 😅

[trim tangle]

unless I misunderstand what Gobot meant

[eager vessel]

I'm sure mypy doesn't require it, is pyright different?

[soft matrix]

i thought it was considered untyped if you didnt add _thing: list[T] in the class body

maybe it was changed

and it used "inference"

[trim tangle]

attributes are automatically typed if they have a known type in __init__ 🙂

[eager vessel]

No, you can provide type info in __init__

Also i think if you annotate the class body type checker would assume that you have this field on your instance?

e.g.

class A:
    field: int

A().field

[trim tangle]

I think so, yes

[oblique urchin]

I think that's only in librariies

[soft matrix]

oh thats it

its weird

[eager vessel]

Is there for a child class to override a field? 🤔 Using type hints ofc and not something like __init_subclass__?

class Base:
    field: int  # I want for subclasses to override it 

[soft matrix]

ClassVar[int]?

[eager vessel]

Well, but i want subclasses to implement/override it

Since it's not assigned a value in base class

[oblique urchin]

There's currently no way to enforce that in the type system

[eager vessel]

Got you, thanks

[oblique urchin]

mypy does enforce this for methods with @abstractmethod, but I don't think you can have an abstract class attribute

[soft matrix]

i mean you could use a abstractproperty right?

and classmethod

[eager vessel]

Yeah, i can have @abstractmethod + @property but it's 3 extra lines 😅

[soft matrix]

well if its what you want surely its fine?

[eager vessel]

abstractproperty is deprecated btw

[soft matrix]

ik

its just shorter to type

[eager vessel]

It's ok if you need a single property, but in my case i need 4

[clear lily]

heya!
I've got a design/architecture question, hope this is the correct channel (otherwise I blame @little hare :^) )

Assuming I have a class XYZ that has a prop: Optional[int] property, and a method that takes some data and spits out an XYZ object.
How do I properly annotate the return type of the method if I know that prop in objects returned by that method will never be optional? The XYZ type is used elsewhere as well, so it can't be changed.
Subclassing XYZ just for changing the annotation seems like the only solution, but it feels a bit weird

class XYZ:
    # defined as optional
    prop: Optional[int]

def from_data(data: Dict[str, Any]) -> XYZ:  # how can the return annotation be improved?
    ...
    # will always be non-optional when returned from this method
    xyz.prop = 42
    return xyz

def create() -> XYZ:
    ...
    # may be optional in other places
    xyz.prop = None
    return xyz

[oblique urchin]

maybe make XYZ generic over a TypeVar bound to Optional[int]

[little hare]

I've got a design/architecture question, hope this is the correct channel (otherwise I blame @little hare :^) )
you sound exactly like @leaden oak 😆

[clear lily]

hmm, that could work, though it might be tedious since there are multiple optional props that are non-optional when returned from from_data in this case
Ideally it'd be nice if editing the signature of create (in the example above) could be avoided, I suppose that's not possible as long as typevar defaults aren't a thing 🤔

[tranquil turtle]

is T a subclass of Optional[T] ?

[potent kayak]

class XYZ:
    # defined as optional
    prop: Optional[int]

def from_data(data: Dict[str, Any]) -> "XYZ":  # how can the return annotation be improved?
    ...
    # will always be non-optional when returned from this method
    xyz.prop = 42
    return xyz

def create() -> "XYZ":
    ...
    # may be optional in other places
    xyz.prop = None
    return xyz
``` the python bot has it defined as this.

https://github.com/python-discord/bot/blob/main/bot/bot.py#L108-L110

[rough sluice]

bot/bot.py lines 108 to 110

@classmethod
def create(cls) -> "Bot":
    """Create and return an instance of a Bot."""```

[oblique urchin]

subtype but not subclass

[leaden oak]

hey I've gotten better at it recently

[mortal fractal]

https://github.com/microsoft/pyright/commit/65f444daecfa3884e082c6cd528c6bd5ee2102df

[leaden oak]

LOL, I just opened pyright's issue tab as Jelle opened an issue

I'm impressed by the extremely low open issue count on pyright

[oblique urchin]

Eric fixes them impressively quickly

Though he's also happy to close things as wontfix

[mortal fractal]

pyright is pretty good overall; it's better than mypy at a good chunk of things, but you do have to keep in mind pyright only has early type binding in the sense of it won't later infer a container type of var init'd to []

Plus it's up to date

I have my toy projects linted with both mypy and pyright strict mode without problem atm

I think all other type checkers have some sort of major issue preventing using them at all (lack of support for | syntax or some other issue)

I was thinking I might try to get pyre to work but last time I looked it had quite a large amount of issues and I didn't want to work through reporting them

Maybe I will sometime though

I want to report everything to pyright though because even if I don't use it, a lot of people use vscode where it might be their first experience with static type checking and ideally the experience is as smooth as possible

[mint inlet]

I try to add bug reports to mypy based on what I encounter but sometimes I get some weird af behaviour and just don't, lol

(I don't really use pyright cause a) sunk cost fallacy and b) I like the attrs plugin kthx)

[terse sky]

Pyright doesn't support attrs?

[mint inlet]

it does, but there are plenty of subtle quirks it doesn't

it mainly just pretends attrs is dataclasses

which like, doesn't work well with factories converters (mypy's attrs plugin will widen the accepted setattr type based on the converter), doesn't change _a to a in the constructor, and reorders attributes in a different way

(this is all known fwiw, but I like these features so... https://github.com/microsoft/pyright/blob/main/specs/dataclass_transforms.md#attrs)

[terse sky]

Does mypy do better than that?

Ah yes, it does

Sorry I missed that

[mint inlet]

yea, and you can do plugins too (which is a positive of doing it in python; although speed kinda sucks)

e.g. i've done a weird hacky plugin to make a "partial" attrs type (think typescript's Partial<T>)

[terse sky]

It's just kind of terrible that all these things have to be special cased into type checkers

You can't even write a decorator that changes defaults

[mint inlet]

lol yep

well you can with plugins for certain type checkers but those are icky

and then you lose compat

which sucks (and which is why i'm not using partials rn :( )

[terse sky]

I wouldn't mind but I don't even find hardly any docs, examples

For mypy plugins

[mint inlet]

ah yeah and the plugin interface imo kinda sucks

[terse sky]

Google was like nope, sorry

[mint inlet]

this is what i had to do, which is just scary

i'm sure there's a better way though

[terse sky]

I can't even really guess what this does

[mint inlet]

it lets me do this

[terse sky]

Ah

That's a bit scary to me

[blazing nest]

I am confused, it made all attributes Optional?

[soft matrix]

The screenshot above this

[trim tangle]

Yeah it's really frustrating. No wonder there aren't more plugins

I was able to hack something togheter https://github.com/decorator-factory/mypy-plugin-attempt

[cold osprey]

hello - i have unannotated lib (uuu) used in my annotated code (aaa) -> i added pyi stubs for uuu. When i do annotatinos in my aaa code on the variables that have types from uuu, should i import the type from stub, or from uuu lib (e.g. Class-es etc.)? Which one is more clean/proper?

[trim tangle]

A stub is not supposed to be run or imported

in fact, I don't think you can import it

[cold osprey]

perfect, thank you!
(my vscode lists it as one of options for importing in helper actions)

🤔 what is the correct location for type aliases then? i've been thinking about putting them into stub files..

e.g. uuu has frequent use of List[Tuple[str, int]], thus my Params = List[...] in stub code, and used in several spots in my aaa
(aaa code is rather thin wrapper around uuu, thus re-using the data types a bit between layers)

[trim tangle]

@cold osprey If aaa is a thin wrapper/adapter to uuu, I would just define the type alias in aaa

Make sure to also use a protected name like _Params in the stub to signal that it's not importable

[cold osprey]

i see, and it's fine to use aaa defined types in stub then i assume...

[cold osprey]

sure thing, thanks

[trim tangle]

No, a stub shouldn't depend on your code. A stub is just a description of uuu

I would just duplicate the alias definition between aaa and uuu

If aaa is a thin wrapper/adapter around uuu, do you really need stubs of uuu? Or maybe you can just add the static types in aaa?

[cold osprey]

Hard to say, i am a bit junior in python per say, not using it very frequently.
Now i started a task of annotating "my" aaa code, that uses lot of methods/classes from uuu.
Thus i started adding stubs for uuu (as one of the steps) to iron out all the mypy warnings from checking aaa code... (like - "calling unannotated function...")

i probably incorrectly stated the wrapper/adapter fact of my use-case.

By stub i mean type annotations stub -> there must be some reasonable way to use deifnition in either direction? i mean if some stubs are created for untypes libs, then client/typed code must be able to use various data-types defined by stub right?

[trim tangle]

No, a stub is not importable or executable

it simply describes what a library contains

You could do something like ```py
if typing.TYPE_CHECKING:
from uuu import _Params as Params
else:
Params = ...

[cold osprey]

i see, code excerpt for aaa. if-case is clear - what would be the use-case of else statement?

[trim tangle]

otherwise using Params will fail at runtime

[cold osprey]

🤦‍♂️ ah, so ... explicitly, not "fill in your code" 😄

[trim tangle]

yeah 🙂

... is just an ellipsis object

[cold osprey]

right, using it in stubs

[trim tangle]

if it's a generic alias, you could do a hack like ```py
if typing.TYPE_CHECKING:
from uuu import _Params as Params
else:
from collections import defaultdict
Params = defaultdict(lambda: ...)

[cold osprey]

What i meant was recycling some more complex list/dics/tuple type annotations both in my aa code, and in stub, to create explicit mapping.
But from your statement i should leave the stub code with raw types, and use my aliases in aaa code only for clean separation.
(despite stub being only private for my aaa code)

[trim tangle]

If type aliases help in the stub, use them in the stub

You could just copy-paste them in uuu and aaa, or if it really gets out of hand make a separate (actual) module

[cold osprey]

i mean i am not arguing, it makes sense for potential separation or changes, just wanted to clarify, thanks a lot! 🙂

[tranquil turtle]

class Truthy(Protocol):
    def __bool__(self) -> Literal[True]: ...
class Falsy(Protocol):
    def __bool__(self) -> Literal[False]: ...

# example:
@overload
def f(x: Truthy) -> Literal[True]: ...
@overload
def f(x: Falsy) -> Literal[False]: ...
def f(x: Any) -> bool: return bool(x)

# another example:
@overload
def assert_(condition: Truthy, msg: Any = None) -> None: ...
@overload
def assert_(condition: Falsy, msg: Any = None) -> NoReturn: ...
def assert_(condition: Any, msg: Any = None) -> None: # if i use `NoReturn | None`, mypy complains `missing return statement`
    if not condition:
        if msg is not None:
            raise AssertionError(msg)
        else:
            raise AssertionError

is using these protocols a good idea?

[terse sky]

what do you want to use them for?

this looks like some kind of tag dispatch taken straight out of C++

[tranquil turtle]

it will help to distinguish, for example, empty collections from non-empty collections

and choose different behavior

[terse sky]

yes, but how is this helping you statically?

[tranquil turtle]

idk)

[terse sky]

Like, suppose you do assert_ on a collection

I don't think that after the assert, mypy will understand statically that it's a non-empty collection

[soft matrix]

Yeah they don't seem useful

[mortal fractal]

+1 on reveal_type @oblique urchin

Speaking of pyright oddities I've been meaning to open an enhancement request to ask Eric to normalize the inferred type in a post PEP 585 world so pyright stops saying List[] and such instead of list[] on inferred types

[oblique urchin]

yeah it won't

[tranquil turtle]

is there any dynamic type-checker?
i mean not runtime type-checker, but type-checker that will run your code with different arguments, trace it and check types of variables

[soft matrix]

Monkeytype?

[oblique urchin]

Monkeytype is a little different, it stores types during runtime executiion. I don't think there's anything like what @tranquil turtle describes.

[mortal fractal]

some sort of pseudo-fuzzing type checker?

[maiden mesa]

Should you type hint self in normal methods and cls in class methods? And should you specify return type as None in init method as well? I have seen some people do this and some not.
like: ```python
from future import annotations

class X:
def init(self: X, value: str) -> None:
self.value = value

[soft matrix]

No

Type hinting self generally isn't something you should do

[maiden mesa]

same for specifying return type as None for init methods as well?

[oblique urchin]

it doesn't add too much but might be useful for consistency if you want a fully annotated codebase

[maiden mesa]

oh okay, thanks

[trim tangle]

There's beartype.

But note that what you want is impossible besides simple types like plain classes (e.g. str) or immutable containers (like tuple[str])

[mortal fractal]

ahhh i was just about to report this https://github.com/microsoft/pyright/commit/6ed39361e3cbafa74ff1c42cc1893fe37cce6ca6

oh

"Behavior Change: Changed the setdefault method generated for TypedDict classes so it accepts only literal key values. Fixed a bug in the get method generated for TypedDict classes so it accepts arbitrary values for the default value parameter."

:\

that means I can't do jsonout.get("library_versions", []) without errors anymore

because the default value will cause unknowns in strict mode

I have to repeat the type like jsonout.get("library_versions", list[Any]()) or whatever it actually is

gross

[soft matrix]

i feel like if the value is an empty literal thats fine

and its only used there

[mortal fractal]

right, but now pyright throws an error because it's partially unknown

[soft matrix]

maybe open an issue?

cause this just sucks

[terse sky]

annotating None is actually important/useful isn't it

if you don't annotate that a function returns None, then it will return Any instead, will it not?

[soft matrix]

its not for init

[terse sky]

ah my mistake

missed that part

[upbeat wadi]

pyright would infer that it returns None

[mortal fractal]

it got changed in response to this issue so I'm commenting there for now @soft matrix https://github.com/microsoft/pyright/issues/2875

cc @oblique urchin

[hollow spruce]

Do pyright or mypy support associated types like in rust

[mortal fractal]

@soft matrix doesn't hurt to add a comment as well if you agree it should be changed

[soft matrix]

i think if its a literal like assuming all you ever do with it is pass it to a function it shouldnt need a type

but id guess from the fact this doesnt work its probably gonna get rejected

[mortal fractal]

well, the annoying part is I did type it in the typeddict!

but because the fallback can be other types it doesn't use that info for the fallback

[soft matrix]

oh yeah i meant in a typed function call

[mortal fractal]

yeah

[soft matrix]

i mean it doesnt matter if you are passing an untyped list to an untyped function

[mortal fractal]

hmmm, didn't get() on typeddict used to work in pyright? I'm getting Any now

[upbeat wadi]

yeah it did

seems like 1.1.210 changed that

[mortal fractal]

I'm filing bug

https://github.com/microsoft/pyright/issues/2883

[terse sky]

from typing import Iterable, Set, TypeVar

_T = TypeVar("_T")

def union_to_set(elements: Iterable[Iterable[_T]]) -> Set[_T]:
    result = set()
    for elems in elements:
        result.update(elems)

    return result

is there a fundamental reason why even the latest mypy complains about result not being annotated

i feel like in some contexts mypy will deduce collection types like this from usage immediately after, but in som ecases it won't

[oblique urchin]

mypy has special casing for some functions like list.append, but not others

[terse sky]

ah, that's weird

ironically I feel like cases like the above have me writing types more in python that I would in some statically typed languages

[mortal fractal]

@oblique urchin sorry you're getting spammed with this issue's comments probably 😅

[oblique urchin]

that's ok, it's pretty interesting

[mortal fractal]

but, I do wonder if the other special casing can be bidirectional like with https://github.com/microsoft/pyright/issues/2875#issuecomment-1014869560

[balmy cobalt]

hello, how would I typehint something that can be converted to a string through str(...)?

[mortal fractal]

I'm trying to think of problems

[void panther]

object I suppose, as anything can be converted to a string

[balmy cobalt]

hmm, isn't there a way to ask for something that isn't the default implementation?

or do I have to manually say that for integers and floats are fine for instance

[oblique urchin]

there isn't (unless you write a mypy plugin or something)

[balmy cobalt]

alright, thanks

[soft matrix]

interesting

[mortal fractal]

if it was that would fix all issues I think?

[mortal fractal]

idk I'm done thinking about this for now, feel free to jump in if you think the special casing can be made to work bidirectionally too or some other fix

[upbeat wadi]

is it intentional that the type of a.foo isn't kept as Literal[3] after the function call?

[soft matrix]

(T) -> T doesnt mean it returns the same instance of something

it means it returns the same type but T can be mutated however the function sees fit

[terse sky]

fwiw I find it to be witch-craft that you get Literal[3] the first time

[upbeat wadi]

ah okay, thanks

[terse sky]

does any statically typed language behave like this?

[blazing nest]

I don't think so necessarily, it looks like Pyright just narrows it down while it can

[mortal fractal]

new pyright published, the problematic case I had is fixed with typeddicts that use list and such now, but ofc using Sequence and such still won't work

it's awkward though

[upbeat wadi]

https://mystb.in/LiesContinuedRod.python
why is pyright giving me this error?

[cold osprey]

hello, is the List[Type[C]] supposed to allow list of various instances of different subclasses of C (what i need), or only all the items being one same specific subclass type?

i keep getting mypy error when i try to return [SomeSubC(), OtherSubC()] from my fn annotated with -> List[Type[C]] (where class SomeSubC(C), class OtherSubC(C) etc.)

In addition, what confuses me is that some q&a's around subclassing refer to TypeVar with some binding's instead...

[acoustic thicket]

i don't see how Type is involved here, but you can put an instance of any subclass of C in a list[C]

[cold osprey]

oops, i may have misunderstood subclass types (Type[C]) vs subclass instance types(C)..., indeed List[C] seems to work for my implementations of fns

[oblique urchin]

it means a list of class objects for subclasses of C

[cold osprey]

@halcyon nexus + @oblique urchin thanks for clarification!

[oblique urchin]

item.callback expects a positional-or-keyword parameter, and you're assigning it something that takes only a positional parameter

[upbeat wadi]

thanks, i see it now

[trim tangle]

why do you want that, out of curiosity?

[brisk hedge]

Any

[terse sky]

I understand the reasoning for Any/object given python being optionally typed

but damn I think it is confusing

object was stuck there before we had a static type system, so that's life, but instead of Any I would have called it Unknown

seems a lot less prone to confusion, and consistent with the fact that it's the "default" type in many cases

[trim tangle]

I agree. I didn't know object was any good as a type hint until very recently.

maybe type checkers could give you a hint, like: "do you really need Any here? it could be object!"

[terse sky]

Yeah. And the fact that you have multiple other languages out there that use Any as a top type, or something similar to a top type, tells you I'm not alone in this interpretation

[trim tangle]

in TypeScript any is like Python's Any, but there's also unknown (which is a type you can't do anything with, like object in Python)

(not contradicting what you said, but maybe that's the inspiration)

[terse sky]

what

that's so backwards

like I'm impressed that TS uses the exact two names that I would want to use for optional typing + top type, but it exactly reverses them

[trim tangle]

Well, Python and TypeScript were born in the environment when you gradually type a dynamically typed codebase. So it's understandable that there's a short and easy way to completely bypass the type checker

I'm not sure what the reasoning behind the name was

maybe any came first, and unknown was added later?

[terse sky]

yes, I agree there needs to be a way to dodge out of typing, after all these are gradually typed/optionally typed languages

But Any is actually the top (pun intended) choice of top type name, in newer languages

It's used in Kotlin, Julia, Scala, Swift

whereas unknown is not used at all in statically typed languages that I know of, because obviously in such languages they are not optionally typed

[trim tangle]

well, in Java there's Object or something, right?

[terse sky]

so if you have an optionally typed language, using Any for the top type and Unknown to "duck out" of the type system, seems by far the most logical

Yes, so in older languages object is the most common top type

in newer ones, it's Any

https://en.wikipedia.org/wiki/Top_type

C++ does not have a true top type but it has a standard library type called any that has top type semantics (all things implicitly convert to it, it does not implicitly convert to anything)

it was standardized in 17 though

i guess the practice starts with smalltalk which is after all kind of the starting point for OO

so java, python, ruby, all have "object" in the name of their top type

[trim tangle]

Rust and Haskell have a variation of any.
https://hackage.haskell.org/package/base-4.16.0.0/docs/Data-Dynamic.html
https://doc.rust-lang.org/std/any/index.html

well, it is a sort of a top type

but you have to explicitly unwrap tit

[trim tangle]

In go the top type is... interface{} 🥴

[terse sky]

yeah, it's pretty random 🙂

yeah what Rust and Haskell have is pretty similar to what C++ has

in order to have a true top type, your language sort of has to default to polymorphism, in terms of storage and dispatch

at least, afaics

which C++ and Rust do not, so having a top type would be undesirable

[cold osprey]

that makes them loveable... 😄 at least Rust for me

[terse sky]

yeah, I don't typically find it important to have a top type, really

I'm happy std;:any exists in C++ but I've used it like, once, for some unit testing utility

[cold osprey]

i'm grinding my teeth now with type annotating some mid-size python code that uses non-typed lib, switching from personal Rust fun projects... 😄

no idea how much traction type hinting has recently for e.g. senior python users, but i as python low/mid-user find it very refreshing and nicely clarifying not that greatly/cleanly previously written untyped code

[terse sky]

I think type hinting in python has been pretty successful, afaics

many people that I talk to have mypy as part of their CI

[buoyant swift]

yeah

[terse sky]

I dislike the term "senior", but, going with it: almost everyone who has more experience, or at least who has more experience and is actually being entrusted with seniority (i.e. can make some decisions at the team lead level or higher)

has probably programmed in some kind of statically type dlanguage with generics

C++, Java, etc

if you have then it's very simple to at least jump in the shallow end of python's type system

[cold osprey]

my task started with around 500 mypy errors, munching through them down to 100 currently after few days, and i identified a few potential bugs already

[terse sky]

yeah

in practice on an existing codebase, IME most of those potential bugs are probably not going to be bugs in practice, because of ways in which the data is constrained etc

otherwise they would have been already discovered and fixed

but some of them might be

but having type annotations helps a lot with development velocity

especially when you have a pycharm caliber IDE

[cold osprey]

it's going fairly slow primarily because of not that greratly documented non-typed lib being used, and C-bindings being inside too 😦

[terse sky]

yeah that can be tricky

though it's weird that you had so many mypy errors to start

[cold osprey]

exactly my case, but leaving fairly loose opening for future code updates to bring in some trouble 🙂

[terse sky]

usually codebases that aren't using types have very few mypy errors

because the default is this Any type that cannot raise type errors

[cold osprey]

unless you aim for --strict 🙂

[terse sky]

hah

does that make any interaction with Any an error?

[oblique urchin]

not really, mypy checks a lot in the global scope too

[oblique urchin]

pretty much

or at least, mypy has strictness options that do that. not sure if --strict enables them all

[terse sky]

yeah, I feel like turning on strict on an existing codebase is probably something I'll basically never have time for

maybe on individual files

but then that's almost pointless unless you also setup CI to do strict on some files but not others, and I doubt I have time to do that either 😛

[cold osprey]

it's a bit of torture, but it's for open-source tool, and sanctified to be done during work-time, so everybody wins 🙂

[oblique urchin]

you can just do that in your config file, right?

[terse sky]

hey, if your work lets you do it, then why not

seems like it would eventually get boring but for a while it's ok

[terse sky]

yeah? That's neat then. Maybe at some point I would try turning it on, file by file.

[cold osprey]

luckilly it's not that huge codebase, and i find it great way to get into python typing 🙂

[terse sky]

but I feel like in practice when I'm working on a file I care about and there's things that aren't annotated I just add the annotations

and tackle it that way. and then it's even more gradual.

there's also probably a lot of almost-dead code, or code that is planned to be removed, etc, that would trip these errors

so handling their mypy errors makes no sense, but then fully removing them isn't high priority either... etc

[trim tangle]

On the other hand...

If your very clever coworker uses type annotations in a very interesting way which is actually incorrect, the code will be 10x worse than without the types

(I'm looking at you Bob)

[coral karma]

index = 1
while (index <= 100):
   print(random.randint(0,999999))
   index +=1```

how do i specify the printed value?

i think this is the right channel

this is just a random idea i thought of, but how do i do it?

[trim tangle]

wdym specify the printed value?

[coral karma]

the print random

[trim tangle]

what do you mean by "specify"?

[coral karma]

i need it to keep going untill it gets the right value

and when it gets it it should stop

[trim tangle]

ah

if it's not related to type hints aka annotations, you should see #❓｜how-to-get-help

[mint inlet]

Does anyone know of any efforts to automatically synthesize falsifying programs to show why something isn't type safe?

I find small examples much more readable than errors xd

[trim tangle]

Agda's proof searcher (Agsy) has a mode where it will search for a counterexample

Not sure about Python. It's pretty vague what 'typesafe' means :) maybe you can define it?

[mint inlet]

Oh that's cool

[mint inlet]

I mean "raises a type error without obvious stuff like Any, or stuff like how kwargs is kinda unsafe"

Hm yeah that's a bit ambiguous tho

[blazing nest]

How do I turn on Pyright's strict mode through Visual Studio Code again?

[trim tangle]

Are you using pylance?

settings -> pylance -> type checking mode -> switch from "basic" (or "off") to "strict"

[blazing nest]

Awesome, thanks!

[ruby ember]

Is it possible to have a typedict which has optional keys?

eg:

class TD(TypedDict):
    a: str 
    b: str 

def f() -> TD:
    return {'a' : 'something'}

here there's no b, which is fine - just wondering if this is possible to typehint or not

[trim tangle]

You can use total=Falseif all your keys are optional

[ruby ember]

ah cool thank you, let me try that

[oblique urchin]

And with PEP 655, you can mark specific keys as required. Pyright and pyanalyze already support this, mypy support is WIP

[trim tangle]

in a lean and compact language like Python, it is pretty strange that simple thinks like required keys or a function type are surprisingly verbose...

maybe I'm just a grumpy old man

And if the callable syntax gets accepted, we'll have at least 3 ways to define a function type

x: There's one obvious way to define a function type.
a: Hm actually, the first way is very restrictive: we have two ways of defining a callable.
x: Fine. There are two obvious ways to define a function type.
b: In fact, those two syntaxes are strangely verbose. Let's now have a third one, now a special form.
x: Good point. The three obvious ways to define a function type, i.e. a "callable", are...
c: Guys I just found a brand new form which is more flexible and readable than all the previous ones!
x: Sigh. Among our chief obvious function type definition syntaxes are...

d: In fact, with a simple mypy plugin you can make a callable syntax that works for morphisms of all categories, not just Set!

[terse sky]

is python really lean and compact? Are we talking about the language syntax, or the specification?

I can't really see either applying that much

I also don't really agree with the premise that this is a simple thing. The whole concept of TypedDict is a weird mix of static and dynamic.
I haven't yet found a use case for it in any new code (i.e. I can only imagine using it for annotating existing code), and the concept can't really exist in almost any statically typed language

[oblique urchin]

I don't think it's really meant for new code, it was built on the observation that a lot of legacy code uses dicts with fixed keys and types and there was no good way to type that

[terse sky]

right, that makes perfect sense

I do think it's a relatively challenging concept to fit into a static type system as a result

FWIW in most cases I would make all the keys mandatory, and just annotate them as optional

where necessary

I'm sure there's some situation where that doesn't work, but in most cases it should work and I think it's actually better

[oblique urchin]

PEP 655 supports that too

[trim tangle]

I mean, one of Python's selling points was originally very simple syntax

not simple as in consisting of very few elements, but like... intended to be easily readable

[trim tangle]

TypeScript is basically TypedDicts all the way down 😛

[terse sky]

well, TS is also an optionally/gradually typed language right

when I said statically typed languages, I didn't intend to include such

"simple" means something different from "lean and compact", I think

[ruby ember]

are they considered a bandaid or is designing with them in mind from scratch generally done?

By the former i mean - i might have something that's returning a dict, and then want to type it - so i'd use a typedict. But if i was writing something from scratch perhaps I should consider something else ? Not sure

[terse sky]

yes, in that sense, they are considered a band-aid

@ruby ember

I mean, there's already a name for something with a specific finite set of "keys" that each can have different types

what most languages would term a struct, or a class

[oblique urchin]

yes. if you're designing such an API from scratch you should probably return an instance of a specific class

[terse sky]

python's more dynamic so classes don't have to be that way, but nowadays it's more encouraged to annotate classes with the whole x: int etc thing

and then if you make it a dataclass, it handles init for free, and even if say your data source returns a dict, you can just construct the dataclass instance using **

[ruby ember]

so in python we'd use an attr's class or something

[terse sky]

yep

dataclass is basically the standard lib version of attr

[ruby ember]

One thing i get confused about there is whether to bother instantiating

[terse sky]

creating a class instance? As opposed to what?

[ruby ember]

class Data:
    x : int
    y : int
data = Data(x = 1, y = 1)
print(data.x, data.y)

or just

class Data:
    x = 1
    y = 1
print(Data.x, Data.y)

i get i can make multiple in the former - but i often only need 1

and that's where i sometimes wonder what's "best"

[terse sky]

the former is very, very, very definitely what's best

I mean in the second, you don't have any static typing annotation which I thought was the original purpose here

[ruby ember]

ok - even if only 1 is used it's still clearly better?

I'm fine with that, to be clear

[terse sky]

and they're also all globals

Like, some function halfway across your codebase could change Data.x and it would be reflected all the way in some other part

no bueno

[ruby ember]

true - i think i've gone a little off topic, sorry

[terse sky]

when you have instances you can always decide whether they should share an instance, or make a copy

you can also make the instance immutable with frozen=True which is often a good idea

[ruby ember]

yeah - my confusion is when I know i only need one

[terse sky]

it doesn't really change anything

the "lower level" functions shouldn't care whether your program only has one

[ruby ember]

yeah i default to frozen and kwargs when i use attrs

ok cool - makes sense... guess i sometimes wondered if i was being a bit redundant by instantiating one and only one

[terse sky]

Well, there isn't really any redundancy right

at least, not in defining, and instantiating the class

[trim tangle]

If you use a function only once per program, would you rather return a value (say, a number or a tuple) from it, or have it set a bunch of globals in its module? @ruby ember

[terse sky]

the place where you start to have differences in the amount of code, is simply, the fact that when you have Data.x, it's simply a global, so it's accessible from everywhere.
So you "save" having to pass it, sometime sthrough multiple layers of functions.

If you have data = Data(x=1, y = 1), data is a local variable (typically) and you need to pass it down

[ruby ember]

i'd always prefer a function to return a value if possible

[trim tangle]

right 🙂

[terse sky]

just curious what is this thing you only have one of

like a Config or something like that?

[ruby ember]

yea - usually in a project there's a module which contains a bunch of metadata

[terse sky]

right. So my program's configs are pretty complicated (they're computer generator from another config, a "meta" config 🙂 )

so the way I handle this is to use nested classes quite a bit

[ruby ember]

i started generating the dvc configs from the metadata.py config today actually lol

[terse sky]

That way, you can pass things along down the way you need to, while typically avoiding that feeling of having to pass a million arguments

that people often complain about (and why they turn to global god objects that hold everything)

before I did that, the last dev had a global struct with like 50-100 attributes

saves some lines of code but makes it much harder to actually see where things go, test things, etc

[trim tangle]

Sometimes people also use "euphemisms" for globals...

[ruby ember]

🌚 i have increased the number of attribute errors in pylint

[terse sky]

yeah, I mean at that point I'd consider some nesting

dataclasses/attrs nest fine and you can, with a bit of elbow grease or using other libraries like pydantic, automatically deserialize a nested json into a nested dataclass

that way each function is only getting passed at least roughly what it needs, instead of a whole huge mess

[trim tangle]

Like, using an inversion of control container in such a way that it's not much different from a global. When you have to monkeypatch in your tests and such.

Or using a "singleton" class...

[terse sky]

makes it much easier to limit the scope of changes

[ruby ember]

for config objects i actually nested recently - instead of using kwargs, and didn't mind it, tho it's no so in line with scientific libraries

[terse sky]

it's not really instead of using kwargs, I'm not sure I understand that part

it does mean that a simple Data(**json_object) no longer works, if that's what you mean

[ruby ember]

"singleton" class here is negative or positive? bc that's what i semi thought i was going for with metadata settings, and that they were a good things for some things

[terse sky]

I'm not a fan

[trim tangle]

a singleton class (a class of which there can only be one instance (which is enforced)) is effectively the same as a global variable

[terse sky]

they were a big thing of design patterns in the 90's, they were supposed to be better than globals, but I actually think they're worse, but that's a whole discussion

that's not quite technically true but it's close enough to true in terms of how singletons are practically used.

[ruby ember]

ah ok - i just meant that instead of having f(data, x1=1,x2=2,y1=1,y2=...) i had f(data, config) where config was an object with config.x.x1, config.x.x2 and so on.

[trim tangle]

I think we should move to #software-architecture 🙂

[terse sky]

hah maybe

[terse sky]

yeah I mean it's generally better to move to dataclasses, because then if f is calling g which also needs config, you just pass config, and you don't repeat the whole .x dance

but it also means that when you pass config, you're obviously passing all of it

that's why it's typically good for config itself to have nested dataclasses

config.foo_config, config.bar_config, etc

the goal is to achieve a reasonably compromrise in granularity

too granular - passing around zillions of individual function arguments over and over, tons of boilerplate, very fragile because to add a single config setting, we have to change every layer of the call stack
too coarse - we pass around all the config settings into every single function, and that means that we're letting every function see the whole config, which is unnecessary, and makes it harder to understand what parts of the codebase need to be examined if we want to change part of the config

[mortal fractal]

https://github.com/microsoft/pyright/issues/2898 someone else also brought up the annoyance

this probably handles my same thing

[clear lily]

oh hey, that's me heh

Yeah, looks like our issues are pretty similar, I managed to hit this problem while trying to use x.get("key", {}) on a TypedDict, where x["key"] would be yet another TypedDict

[mortal fractal]

I think Eric and I were just caught up thinking this needed special handling without realizing we could solve it with both overloads @clear lily

[little hare]

does eric have a discord?

[mortal fractal]

I do not know

[little hare]

or am i searching for a conversation that didn't happen

ah okay lol

[mortal fractal]

We were discussing it in a closed GitHub issue yesterday

[little hare]

context: I'm working on the same project that shiftinv is

[mortal fractal]

@oblique urchin I really like assert_never, were you thinking of rolling this up in a PEP with reveal_type and the others?

[oblique urchin]

yes

[mortal fractal]

+1

[little hare]

and have been present for most of their typechecker rants 🤡

[mortal fractal]

https://github.com/microsoft/pyright/issues/2875#issuecomment-1014869560 @little hare

This and scroll up for when we were discussing it

[little hare]

the real fun thing is we've (shift) has found 2 or 3 pyright bugs with this project

its a bit sad that the typing is so bad in it that this happens...

[sick lynx]

anyone here works with spark sql?

[novel saffron]

How do i typehint an empty dict i.e. null = dict()

it would always be empty

[solid light]

typing.Tuple
Tuple type; Tuple[X, Y] is the type of a tuple of two items with the first item of type X and the second of type Y. The type of the empty tuple can be written as Tuple[()].

I guess you could try dict[()] maybe

[novel saffron]

Hmm, lemme try that

(using mypy btw)

Nope

same mypy error

[boreal ingot]

something like this should work ```python
from typing import *

class Empty(TypedDict):
pass

test: Empty = {}
test2: Empty = {"abc": "123"} # error: Extra key "abc" for TypedDict "Empty"```

[brisk heart]

any ideas on how to make this valid?

FooT = TypeVar("FooT", bound="Foo")

class Stuff(Generic[FooT]):
    ...

class Foo:
    def __init__(self: FooT):
        # this line isn't valid for some reason:
        # Type variable "FooT" has no meaning in this context
        self.stuff: Stuff[FooT] = Stuff()

[blazing nest]

Well I mean, isn't the error valid afterall?

[brisk heart]

it is

I absolutely understand it

the thing is, what's the proper way to do it then?

[blazing nest]

You could probably use self.__class__ or change it to stuff: Stuff[Self] (or stuff: Stuff['Foo'] if you don't have typing_extensions)?

[brisk heart]

I can't use typing.Self cause mypy doesn't fucking support it

[blazing nest]

Why put the typing annotations inside of __init__() instead of on the class btw?

I never got why people do that

[brisk heart]

I do normally

this is just for the sake of the example

stuff: Stuff['Foo'] wouldn't work because the whole point of the typevar is to allow subclassing of Foo

[blazing nest]

Because of scoping, I think it makes a difference no?

[brisk heart]

idk, I'll do whatever works

[blazing nest]

This is the problem that Self solves 😔

[brisk heart]

as long as when I access FooSubclass().stuff.parent or whatever I can get back my FooSubclass instance

[brisk heart]

guess I'll wait for mypy to implement it ig

Any it is then

[blazing nest]

The only way I can think of that is requiring stuff: Stuff[NewSubclass] on each subclass.

[brisk heart]

well I won't require it

[blazing nest]

It should work because it's covariant I guess

[brisk heart]

sounds like a pain for the user

[blazing nest]

It'll be a requirement if you want the typing support that is

[brisk heart]

annotating with stuff: Stuff[Any] should make it user-friendly while still allowing subclasses to ensure type-safety

[soft matrix]

Nothing supports typing.Self cause it's not in typing yet ;)

But I might actually be able to fix it over the weekend

[trim tangle]

Pyright supports the one in typing_extensions

[soft matrix]

Yep but not typing

[trim tangle]

Oh I didn't read your message properly

[coral karma]

so im using

input('=> ').lower().replace(' ','').rstrip('s','?','!',''s')

ik what the problem is, idk if this is the right channel but like i need help knowing how to make these paragraphs so it doesnt interfere with the very common usage, ('s) for instance, "what's up"

[trim tangle]

If your question is not about type hints, see #❓｜how-to-get-help

[grave fjord]

typeshed dropped support for python3.5 https://github.com/python/typeshed/pull/4675
but I still see a check for it in the toml stubs https://github.com/python/typeshed/blob/556e623a8163062eb6929dd530e2bae120ceba07/stubs/toml/toml.pyi#L5-L13

is this still needed?

[rough sluice]

stubs/toml/toml.pyi lines 5 to 13

if sys.version_info >= (3, 6):
    _PathLike = StrPath
elif sys.version_info >= (3, 4):
    import pathlib

    _PathLike = Union[StrPath, pathlib.PurePath]
else:
    _PathLike = StrPath```

[grave fjord]

looks like there was an attempt to add py.typed to toml https://github.com/uiri/toml/pull/347

oh and this is happening https://github.com/pypa/pypi-support/issues/1557

[oblique urchin]

probably not, surprised it's still there, Sebastian diligently grepped for the old versions 🙂

[grave fjord]

why doesn't this type-check? https://mypy-play.net/?mypy=latest&python=3.10&flags=strict&gist=aeffaae1238515c61d8a20bc895f2670

surely if a function takes a dict[str, Any] it could also take a TypedDict of any type?

[terse sky]

no

@grave fjord

Oh, actually, I forgot about how Any works in python compared to most languages 🤣

I'm not sure then

[hasty hull]

This doesn't type check as a dict is mutable which means that it could be mutated within the function

Making the TypedDict no longer valid

[terse sky]

that's what I was thinking, but Any isn't a top type

I don't know how things like invariance work in the presence of Any, which, I guess, is like a top and bottom type at the same time?

[hasty hull]

It's not to do with the Any, it's to do with the keys

[terse sky]

ahh sorry I misunderstood

[hasty hull]

You could add a key that isn't present in the TypedDict if it could be passed to a function that takes a dict

All good

[terse sky]

Yes, you should change your annotation from dict to Mapping

and that's a good tip generally, function arguments should be annotated with Mapping far more often than Dict

[grave fjord]

Oh interesting

[gleaming thistle]

There must be something obvious I am missing here, but I haven’t used typeguard in a while, why would i get an error like this from check_argument_types() ?

must be one of (List[Dict[str, str]], NoneType); got list instead

Argument is: [{‘a’: ‘b’}, {‘c’:’d’}]

[mortal fractal]

Hmm, at some point recentlyish pyright seems to have started complaining about the typing of decorators external to my code

Just noticed when opening up an old flask project

[soft matrix]

is it complaining about the untyped deco obscuring the function?

[mortal fractal]

yeah

[mortal fractal]

I wonder how fundamental pyright's early-binding of types is in the code.

In the case of pyright there's tension between making typing easier by allowing late binding of the type of collections and such (a = []; a.append("hi") => reveal_type(a) # list[str] vs list[unknown] in pyright) and pyright's desire to act as a LSP for people who may not care about typing

Even though complaints or confusions about this perennially come up on pyright issue tracker, I don't particularly think the pyright approach is 'wrong', but I wonder if pyright can have its cake and eat it too with the addition of an explicit auto type or something that instructs it to try to determine type later

[terse sky]

I mean many (most?) statically typed languages just don't allow after the fact usages to determine type

so this behavior isn't going to be unfamiliar; the opposite, almost everyone will be at least familiar, if not solely familiar with languages where things have to have a type based on their initializing expression

it is convenient but the issue for me is, i didn't understand why it worked sometimes but not other, and it turns out that mypy special cases it in. If it's going to be special cased in then I'd probably rather not have it.

[mortal fractal]

rust and c++ work like mypy I believe, what were you thinking of @terse sky

I'm not familiar with any that have automatic type inference but only in an early binding way other than pyright, but I assumed that's just because I couldn't think of any offhand

[terse sky]

@mortal fractal C++ does not work like mypy, no

Rust does, as does Haskell, Kotlin has a very limited version

[mortal fractal]

hmm, the auto type doesn't? I assumed it did but I didn't try

[terse sky]

But C++, and afaik java, C# etc dont

Auto means you don't write the type but you can't do backwards inference

Try auto x = std::vector(); x.push_back(5);

[mortal fractal]

I didn't know Java had type inference :o

[terse sky]

I think it does now doesn't it? Or maybe that's the next one

[mortal fractal]

I do not follow Java

[terse sky]

Me neither

Anyhow, mypy s behavior is unattractive because it's not consistent

[mortal fractal]

Hmm, surely it can be consistent and sound if rust does it, no?

[terse sky]

If it always worked it would be nice though personally I think it's still a band aid

Sure it's possible

But Rusk is using a type system designed carefully from scratch and handles this use case

[oblique urchin]

what's inconsistent about it?

[terse sky]

It hardcodes this behavior for certain methods

Didn't you tell me this

[oblique urchin]

ah right, that's true

[terse sky]

Actually I was very confused for ages why I needed the annotation in some but not other cases

[mortal fractal]

If pyright was capable on a technical level of solving some of the types forward it could probably implement a code action to annotate what e.g. mypy would usually infer

That would be cool

as in, through the LSP provide an action to add the explicit annotation

It could probably do that for the return type now

[terse sky]

the thing with most of these typing problems is

[buoyant swift]

it does, with the var kw

[terse sky]

it's only worth being "clever" if you can actually be clever in a consistent, clear, way

meaning: it's easy to explain exacxtly when you can be clever, and it always works.

If you don't satisfy that, then it's usually more trouble than its worth, because someone depends on the clever-ness, and since it's not clear what the limits of the cleverness are, someone makes a small modification of the code and suddenly you ahve to add type annotations, or checks, etc

[mortal fractal]

n.b. pyright already (always) infers the return type already

(even in strictly typed code)

[terse sky]

Yeah, I mean, in just about every language that can do this

and there are many

this is discouraged

so I'd say this is a pretty bad default for pyright.

[mortal fractal]

hence a code action for explicitly adding it

[terse sky]

Sure, but it's still not a good default. I'd hope that pyright has a flag where you can force such code to not pass type check

[mortal fractal]

(it's an LSP change not a type checking one)

[terse sky]

but these are two different things right

LSP doesn't have to be correct.

I guess I don't follow the details of what exactly you're saying about pyright.

[mortal fractal]

Pyright is (literally) a LSP implementation

[terse sky]

I thought for some reason pyright is also a type checing implementation like mypy but maybe I'm getting mixed up

Pyright is a fast type checker meant for large Python source bases. It can run in a “watch” mode and performs fast incremental updates when files are modified.

[mortal fractal]

The type checking part of pyright drives its LSP

[terse sky]

okay, when you said "pyright infers the return type" it sounded like you were talking about type checking

[mortal fractal]

It does do this for type checking, and you can have an opinion about this, but this is independent of whether or not the LSP in pyright can use that inference to provide a code action

pyright could change its defaults like you want and still use the inference to power a code action

Just not run it in type checking mode as part of the type checking

[terse sky]

yes, I just don't understand how we endedup talking about the LSP I guess

[mortal fractal]

I think it's another way to approach the problem of people being annoyed by pyright defaults while addressing your type checking concerns

The type checker doesn't have to change, but the LSP side can use the type checker's knowledge to help people add the types to their code

[terse sky]

not really, for people who care about type checking the most important thing is going to be CI

having an action to fix it is better than nothing, but the fact that it passes CI is a problem

i guess it's a reasonable choice for trying to retrofit onto existing codebases with partial at best type annotations

for green field code, it's very weird. If you want to infer the return type, it could at least use a special placeholder type, like Auto

[trim tangle]

@terse sky I think there is one case where inferring the return type could be a good idea: function factories, like a decorator factory

If your innermost function has some non-trivial annotation, it gets more and more complex with each layer

it could be good to have an explicit Auto though, I agree

[terse sky]

I'm not sure I understand this exact example, but yeah, in languages with this feature, the idea is not to never use it

there are times for it

but yeah, you opt into it explicitly with Auto

don't make it so that literally forgetting your type annotation allows the code to go in

[mortal fractal]

I think we're still talking past each other; I'm not offering this instead of changing pyright, I'm saying it can happen irregardless of pyright's method, in fact it probably becomes more useful if the pyright default changes like you want

Because presumably people who relied on pyright's old behavior suddenly will need to add a bunch of types :P

[terse sky]

Okay, I think I understand now, rereading our convo

[fierce blade]

What's a good Python IDE?

[terse sky]

i think we branched a bit in the convo and I was focused more on one aspect, and you on another, and I didn't understand how you got to where you were, now it makes sense 🙂

[mortal fractal]

:)

[terse sky]

realistically, Pycharm

[fierce blade]

tyu

ty*

[terse sky]

np

can look in #editors-ides as well

[fierce blade]

ok

[mortal fractal]

Eric has stated before their internal MS python codebases do use return type inference a lot, but as far as I can tell this is really rare in typed code in the wild

I think because most people use mypy or if they use pyright they also use mypy

So they have to add it anyway

I also like adding it for documentation reasons

[trim tangle]

Illustration

def decorator_factory(apple: int, banana: str):
    def decorator(func: Callable[[T, tuple[T, ...]], tuple[T, str]]):
        def new_func(fruits: tuple[T, ...]) -> T:
            ...
        return new_func
    return decorator


def decorator_factory(apple: int, banana: str) -> Callable[[int, str], Callable[[Callable[[T, tuple[T, ...]], tuple[T, str]]], Callable[[tuple[T, ...]], T]]]:
    def decorator(func: Callable[[T, tuple[T, ...]], tuple[T, str]]) -> Callable[[tuple[T, ...]], T]:
        def new_func(fruits: tuple[T, ...]) -> T:
            ...
        return new_func
    return decorator

And if you want to specify argument names or positional-only/keyword-only arguments, you need to define an entire protocol class.

I'm actually not sure you can even annotate this case correctly, because decorator_factory returns a parametrically polymorphic function 🤔 so does the annotation even make sense?

[rustic gull]

Is None the correct return type for a property setter?

[oblique urchin]

yes

[rustic gull]

Thanks

[terse sky]

I think like I said, it's rare because most people just think it's not a good idea :-).
E.g. even in Haskell which has crazy powerful typed inference, people I know who use Haskell in prod say that pretty much all their functions are fully annotated

once you start allowing inference across function boundaries, that means that buggy modifications in one function can cause the type system to flag something in another function

which just isn't really what you want

[trim tangle]

yeah, I think the general haskell advice is to give explicit annotations to all top-level functions

[fierce ridge]

idris has the opposite problem, it has almost zero type inference, even in obvious cases where you just assign foo = bar, you have to copy the whole type

but because type annotations are actual idris2 expressions that are evaluated by an actual idris2 interpreter (with some limitations, e.g. you can't perform i/o), you can easily re-use complicated type signatures by defining them as functions

[grave fjord]

What's the difference between Idris and Idris2?

[fierce ridge]

the former isn't being developed anymore

the latter targets scheme primarily, whereas the former was written in C

[grave fjord]

But Idris supports Idris2 expressions?

[fierce ridge]

although there is a reference-counting C compiler target for idris 2

oh, i was just being lazy about the naming

i was only talking about idris 2

[grave fjord]

Ah

Idris2 doesn't have linear types?

[fierce ridge]

it does

idris 1 does not

[grave fjord]

So why reference counting?

[fierce ridge]

because variables are still un-quantified by default, so most variables in most programs will still not have quantity 1 or 0

[grave fjord]

variables?

[fierce ridge]

bindings? idk the proper CS terminology

there is a lot of work right now on things like linearly-typed arrays

or at least there was a lot of interest in such things from the community a couple months ago

i have been kind of "checked out" from idris stuff since the new year

i needed a break and wanted to focus on my actual data science work

[grave fjord]

I wish they called them valuables

[grizzled smelt]

does anyone know where I can find a comparison of mypy/pyright/pyre/pytype? Or has any opinions?

if you are just interested in static type checking, and not in any other linting... is mypy still the best?

[terse sky]

idk about best

it's just the most ubiquitous

generally speaking

most likely to have tools available for it in your IDE

most likely that libraries have specifically checked their type annotations make sense with mypy

etc

I would just default to mypy unless you have something specific in mind

one example is that mypy for example doesn't handle recursive types well at all, while I think pyright does

so if you care a lot about that, that might be a reason

[oblique urchin]

Pyright right now is best about picking up new features

pyre and especially pytype don't have much use outside their respective companies

[grizzled smelt]

google's stuff is always feels a bit weird I concur... testing out pyright now

error: "InteractionsList" is not iterable
    "__next__" method not defined on type "Iterable[Interaction]" (reportGeneralTypeIssues)

🤔

Iterable not iterable, maybe it's genuine, but a funny error

[oblique urchin]

Iterables don't have __next__, Iterators do

though that error does look pretty mysterious

[grizzled smelt]

pyright is picking up some useful stuff though, just by running pyright .. Meanwhile mypy seems to shit the bed.

Ah right I always get the two confused

[mortal fractal]

ayyy someone brought up the Final thing I mentioned a few months ago

https://github.com/microsoft/pyright/issues/2906

still an open question, but nobody was interested on the ML when I tried to push it

[fierce ridge]

this seems like a possible bug, if only in the error messaage. it looks like the error message should say "InteractionsList is not an iterator"

but it's also very very rare that an API requires Iterator and not Iterable

[grizzled smelt]

it's type hinting __iter__ so Iterator was correct, agree the message is confusing

One thing I've noticed, pyright seems to really dislike pydantic's constrained types, e.g. speaker: conint(ge=1, le=2), it says Illegal type annotation: call expression not allowed 🤔 From the limited documentation I have no idea if this is expected or not

[oblique urchin]

that seems right, that's not a valid annotation under PEP 484

mypy will likely give the same error

not familiar with pydantic but they should use Annotated if they're aiming for compatibility with static type checkers

[grizzled smelt]

Annotated[Optional[int], conint(ge=0)]

this seems to fix it, thanks

one question, I have a library that has a type which is a big union of basic types (type hinting data coming back from a database, i.e. unknown).

I get the error, SQLParameter is just Union[str, int, ...]

Argument of type "SQLParameter" cannot be assigned to parameter "InteractionStartDate" of type "datetime" in function "__init__"
    Type "SQLParameter" cannot be assigned to type "datetime"
      "str" is incompatible with "datetime"
      "int" is incompatible with "datetime"
      ... every other type within the SQLParameter

Is the only way around this to narrow, via assert isinstance(data["SomeColumn"], int) etc, every single place I query the database? (not feasible or useful really). Is there a good way of solving this? I have around 100 of these errors

[oblique urchin]

yeah this is why we recommend against returning Union types

maybe the library can produce more specific return types based on the db column

[grizzled smelt]

it's my little unknown library: https://github.com/invokermain/pymssql-utils so can fix it up.
so sounds like Any type is preferred then? The underlying driver doesn't provide anymore info on the types coming back from the database, so its the Union or Any

[oblique urchin]

Any is the general recommendation yes, though it's unfortunate

If you know the schema you could create a plugin of some sort that infers the right types (we do that internally at my company)

[grizzled smelt]

P.S. thanks for the responses, much appreciated, I don't have many knowledgeable Python people where I work (mainly a c# shop)

ah yeah you could, but probably beyond the scope of the library, I'll update the type hints to Any most likely

[novel saffron]

Nope, mypy is still complaining

mypy.....................................................................Failed
- hook id: mypy
- exit code: 1

src/black/const.py:2:1: error: Module "typing" has no attribute "TypedDict"; maybe "_TypedDict"?
src/black/const.py:15:19: error: Incompatible types in assignment (expression has type "Dict[<nothing>, <nothing>]", variable has type "Empty")
Found 2 errors in 1 file (checked 1 source file)

from enum import Enum
from typing import TypedDict

class Empty(TypedDict):
    pass


class LogLevel(Enum):
    none: Empty = {}
    ...

[oblique urchin]

try typing_extensions.TypedDict

[novel saffron]

thanks Jelle and vivax, that works 👍

[trim tangle]

Why does an attribute not satisfy a property?

i.e. ```py
from typing import Protocol
from dataclasses import dataclass

class HasPosition(Protocol):
@property
def x(self) -> int: ...
@property
def y(self) -> int: ...

class Point1:
def init(self, x: int, y: int) -> None:
self.x = x
self.y = y

@dataclass(frozen=True)
class Point2:
x: int
y: int

p1: HasPosition = Point1(x=3, y=5) # error
p2: HasPosition = Point2(x=3, y=5) # error

[boreal ingot]

it does if you make HasPosition a Protocol

[trim tangle]

oh, that was a typo

but it still doesn't work

This was asked a few times on the pyright repo, and Eric just said "they're different" without elaboration
https://github.com/microsoft/pyright/issues/2072
https://github.com/microsoft/pyright/issues/1368

[boreal ingot]

hmmmm

[trim tangle]

🤔

I will open an issue 🙂

[boreal ingot]

according to this, it was changed a while ago
https://github.com/microsoft/pyright/issues/2490#issuecomment-952041414

@trim tangle ah found the commit https://github.com/microsoft/pyright/commit/fa84505344a0a81da75703884b3f217d243de849

Changed the interpretation of a property declared within a protocol class. It was previously interpreted only as a property (i.e. classes compatible with the protocol must implement a property of the same name). Compatible classes are now able to declare an attribute whose type is compatible with the property getter. Access to the property from the protocol class as a class variable is no longer allowed.

[trim tangle]

well.... it does error in mypy thou

ah ok

[boreal ingot]

hmmmmmm (again)

[trim tangle]

ah

I did something wrong then

yeah, it does make sense

I don't know what Eric's original rationale was tbh

The whole point of a property is that we can replace an attribute with a dynamic computation without breaking stuff

[boreal ingot]

I guess they are technically different

oh interesting A property defined within a protocol class cannot be accessed as a class variable (my attempt at showing they can be different)

[grizzled smelt]

Hey guys, a question... trying to tidy up my library to work nicer with static type checkers.

My problem is I have a class that has a data property that is either None or a list of something: https://github.com/invokermain/pymssql-utils/blob/main/pymssqlutils/databaseresult.py#L77

It all depends on the fetch parameter, i.e. if fetch is true then data is a list, if fetch is false data is None.

Is there any way of type guarding this? Or is the best approach to raise an error if fetch == false and the type hint can just be List[stuff]?

[south topaz]

you can use overloads https://docs.python.org/3/library/typing.html#typing.overload

[grizzled smelt]

interesting didn't know about these, thanks. How would you use it to overload a property based on an attribute?

[trim tangle]

@grizzled smelt That is not possible

Generally, if you have a class with lots of flags or optional fields, you might want to rethink your design

What does the fetch flag represent in your case?

Why can't you make two classes, one "fetchy" and another "non-fetchy"?

[grizzled smelt]

ok cool thanks, just wanted to know my options before I rethought my design. I'll make the data property raise an error if no data was ever fetched. Then the type hint is more specific. There shouldn't be a need to call data if it is None.

it's a slight breaking change, but for the best

[terse sky]

I'd probably consider what fix said. You could remove the fetch argument to the init and remove the data property. Then have a derived class which has the data property, not optional, and does the same call with fetch=True.

[mortal fractal]

https://bugs.python.org/issue45384 latest comment hmmmm

If the interaction of Annotated with some typing stuff is causing problems I feel like this should be considered an important bug if we want to continue to claim typing uses of annotations aren't going to interfere with people using annotations for other things

People already complain on the ML that there's too much tension between typing and other uses of annotations, so we owe it to make our blessed non-typing annotation format (Annotated) work well for people if we want to be taken seriously

[oblique urchin]

interesting, ties in to the current discussion about Annotated vs. Required/NotRequired order

[soft matrix]

What's ML?

[mortal fractal]

Mailing list

[soft matrix]

Ah

[mortal fractal]

I don't use Annotated so I can't speak for those people, but I feel like it's an important issue to take seriously for these reasons

can (and should) the annotated+final/classvar ordering be fixed? Are you familiar with how it ended up in the current situation?

[oblique urchin]

I'm generally not a fan of the typing.py runtime enforcing restrictions

[void panther]

I've always thought Annotated just lets through the first arg and the type checker should see it as that, why isn't that the case?

I agree from what I've seen it just causes weird issues for little benefit

and it's always annoying when I try to be quick and use 0 as an annotation and it raises an exception

[oblique urchin]

But to a type checker things like Final, ClassVar, Required, NotRequired aren't really types, they're special markers around types

[mortal fractal]

I thought Annotated was supposed to be our separator between stuff type checkers look at and stuff they don't look at, and from that POV I assumed Annotated should always be in the outmost scope

And this keeps things simple for non-typing users too

It cleanly separates what they want to look at from what they don't

[oblique urchin]

There are plausible use cases for nesting it. e.g. you may want list[Annotated[int, GreaterThan(1)]]

[mortal fractal]

We can accommodate by not disallowing either situation, I think

People who want to annotate like this have already bought into a need to parse the typing side of annotations

But as long as we never disallow annotated in the outermost scope we can guarantee we're not forcing non typing users to parse typing annotations if they don't want to

[mortal fractal]

idk I kinda want to bring this up on the typing GitHub repo or something?

[oblique urchin]

can also mention it in the PEP 655 thread, where I just argued for the opposite 🙂

[mortal fractal]

Okay, I'm out right now but I'll send a message when I get back summarizing the point

It's not that I disagree with stuff about being a field or not; I just think we need to have type checkers see Annotated transparently without restriction if we are to be taken seriously as allowing other annotation users without unnecessary hindrance

[mortal fractal]

I wonder if annotated on the outside breaks the runtime introspection dataclasses.py does of ClassVar

I actually read how it was implemented 3 or 4 months ago but I don't remember the specifics

[oblique urchin]

yes probably

[mortal fractal]

That's an oof for backwards compatibility, I wonder if a change would be backportable as a bugfix in the hypothetical everyone agreed on changing behavior

By changing we would be committing to typing introspectors being able to deal with annotated always, but they already have to do that in the inside so I don't think it's such a huge deal

I see this as just being a good citizen to other uses of annotations though

[oblique urchin]

that sort of introspection is tricky in the presence of string annotations, unfortunately

[mortal fractal]

the internal ones?

[oblique urchin]

yes, the stuff dataclasses does

[mortal fractal]

okay drafting a ML response right now

[mortal fractal]

okay I think i'm done

[mortal fractal]

posted to mailing list

[mortal fractal]

https://mail.python.org/archives/list/[e-mail]/message/MPMOIBX3XFXCD4ZNDC6AV4CLSI5LN544/

[upbeat wadi]

is there any way to do something like x: '(int) -> None' except with kwargs? (e.g. x: '(delay: int) -> None')

[oblique urchin]

callback protocols

https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols

[upbeat wadi]

ah i was hoping to avoid that, thanks

[soft matrix]

Hopefully someday though that will work

[olive oar]

I have this piece of TS code, how would I do it in Python?

type TextOption = { type: 'text'; def: string }
type SliderOption = { type: 'slider'; def: number }
type ToggleOption = { type: 'toggle'; def: boolean }
type Option = TextOption | SliderOption | ToggleOption

declare function parse<T extends Record<string, Option>>(
    query: string,
    options: T
): {
    [K in keyof T]: {
        text: string
        slider: number
        toggle: boolean
    }[T[K]['type']]
}

const result = parse('', {
    url: { type: 'text', def: '' },
    duration: { type: 'slider', def: 0 },
    loop: { type: 'toggle', def: false },
})
// const result: {
//     url: string;
//     duration: number;
//     loop: boolean;
// }

[trim tangle]

Do you want them to be actual dictionaries in Python?

[trim tangle]

If you're asking about the parse function, no, you can't do anything remotely like that in Python

[olive oar]

Yeah sorry wasn't clear it was about the parse function.

What would be the Python way to do it, just omit typing for its return?

[trim tangle]

There is just no way to type-hint this relation between the input and output

If you use this function just one time here, I would just inline it

(in TS as well)

[olive oar]

It's for a library so that's not really an option.

[trim tangle]

Can you show a bit more realistic example?

I don't understand what the function is doing

oh wait, I think I see now what it's doing

[olive oar]

Sure, not exactly but the idea should come across.

const server = new MyLibrary({
    url: { type: 'text', def: '' },
    duration: { type: 'slider', def: 0 },
    loop: { type: 'toggle', def: false },
})

server.onReceiveRequest = query => {
    // when client hits the server with query "url=foo&duration=5&loop=true"
    // query is { url: 'foo', duration: 5, loop: true }
}

[trim tangle]

yeah, I didn't read your example properly

[trim tangle]

The way people do it in Python is by inspecting the type hints, like

@dataclass
class MyRequest:
    url: str
    duration: int
    loop: bool

server = MyLibrary(MyRequest)
``` I don't know a good way to do this, but you can attach arbitrary metadata with `Annotated` ```py
@dataclass
class MyRequest:
    url: Annotated[str, Text]
    duration: Annotated[int, Slider]
    loop: Annotated[bool, Toggle]

server = MyLibrary(MyRequest)

I agree that it is pretty ugly

[olive oar]

Hmm right, the library can get the type info during runtime

But how would server.onReceiveRequest be checked during "compile" time?

[trim tangle]

MyLibrary could be a generic type, like ```py
class MyLibrary(Generic[T]):
def init(self, request_definition: Type[T]) -> None:
...

def on_receive_request(self, callback: Callable[[T], None]) -> None:
    ...

(type[T] instead of Type[T] if you're on 3.9+)

then you do something like... ```py
@server.on_receive_request
def on_request(req: MyRequest) -> None:
...

[olive oar]

Ah I think I get it now

[trim tangle]

It is still possible to attach completely wrong metadata to a type, but that could be checked at startup time, so not a big deal

[olive oar]

The example in TS, MyLibrary constructor accepts options, and the type of request query is inferred from options' type.
And you are saying in Python, instead it should accept the request query, and in runtime constructor generates the options via inspecting metadata.

Am I understanding it correctly?

[trim tangle]

Yes

In the future (probably 3.11) (or if the user uses from __future__ import annotations) annotations will be turned into strings:
https://www.python.org/dev/peps/pep-0563/
There's also a different proposal with a different way:
https://www.python.org/dev/peps/pep-0649/
So you'll have to use typing.get_type_hints https://docs.python.org/3/library/typing.html#typing.get_type_hints with include_extras=True

and it will not work in some cases, like if the class is constructed inside a function or inside another class

[olive oar]

Interesting.

[trim tangle]

the whole idea of making annotations into strings seems a bit wild to me...

[olive oar]

Yeah that feels a bit weird too

Python even has built in AST parsing during runtime (it's one thing I would really love in JS), so I would think it's just easier and more powerful to leave annotations as is.

[trim tangle]

The AST will still have the proper expressions for annotations

but the actual metadata stored at runtime will be strings

objects don't really store their ast, if you want to inspect the AST of a function, you need to run inspect.getsource to get its source code as text, and then run ast.parse on this source

[olive oar]

Right, I'm saying interpreter has to already parsed the source initially, and it would be nicer to leave the annotations as AST/whatever format rather than forcefully converting into strings.

But I suppose a good reason against it is that that's unnecessary overhead, as annotations might not be commonly used at all.

[trim tangle]

The main reason for turning annotations into strings was the forward referencing issues

[olive oar]

Oh? I assume something to do with circular dependency stuffs?

[trim tangle]

Well, something like ```py
class A:
def compare(self, other: A) -> int:
...

or even ```py
class A:
def f(self) -> B:
...

class B:
...

[olive oar]

Ah

So B is not yet parsed by the time f is resolved.

[trim tangle]

well, parsing occurs before any runtime

but B is not defined yet when f is being defined

[olive oar]

I see.

Back to topic, if I don't want to use inspect metadata at runtime, how would I go about it?

[trim tangle]

Without hacks that will make people jump from their chairs, I don't think it's possible 😄

[olive oar]

Have to lose typing for either the options or the query, I assume still ask user to make a Request dataclass, but also need user to manually pass in the correct options then?

[trim tangle]

Yes, you could also pass a dict mapping keys to widgets, or something like that

You could optionally inspect the type annotations on the dataclass and issue warnings if the type of a field doesn't match the type of the widget

[olive oar]

Cool, I think that's probably a good compromise.

I'm still working on the TS version of my library, but every time I use some complex type design I think about how I would do it in Python

[trim tangle]

Another option (more of a hack) is to use defaults in the dataclass ```py
@dataclass
class MyRequest:
url: str = Widget
duration: int = Slider
loop: bool = Toggle

@dataclass
class MyRequest:
url: str = unwidget(Widget)
duration: int = unwidget(Slider)
loop: bool = unwidget(Toggle)

[olive oar]

It's fun but also a bit frustrating at the same time 😅

[trim tangle]

Yeah, Python's type system is very limited compared to TS

It is very frustrating that you can't do these nice structural mappings like in TS

[olive oar]

Yeah I found it particularly hard to construct new types from existing types.

[cedar sundial]

In what circumstance would generics need to be used? I can’t seem to get my head around them

[trim tangle]

The main use case is for containers. If you had just list, it would be pretty inconvenient because my_list[0] and my_list[3:5] would be just Any. That's why you parametrize it like list[str]

[cedar sundial]

I get that, but that’s not using generics? I thought they use like list[T]

[trim tangle]

list is a generic class

(at least conceptually)

You could make your own container, like ```py
class MyList(Generic[T]):
...

Unless maybe you mean something different by generics??

[cedar sundial]

Like that: Generic[T]. The T represents what exactly?

[trim tangle]

Do you know about TypeVars?

[cedar sundial]

I know of them but haven’t used them before

[trim tangle]

I have a small article about TypeVars 🙂 https://dev.to/decorator_factory/typevars-explained-hmo

[cedar sundial]

Interesting. I guess one use case would be to collapse a bunch of union types down to a generic. Would you always use a generic for even just a union of 2 types?

[trim tangle]

I don't think I understand you

they do different things

[cedar sundial]

Where you are adding constraints in that article. You replace the union with a TypeVar with constraints. Could you do the same for a function which might have multiple types as it’s input and output? As well as if a parameter of a function had a union of just 2 types, could you replace that with a TypeVar, with or without constraints?

[trim tangle]

def f(x: Union[str, bytes]) -> Union[str, bytes]: ...
a = f("foo")   # a: Union[str, bytes]
b = f(b"foo")  # b: Union[str, bytes]

S = TypeVar("S", str, bytes)
def g(x: S) -> S: ...
c = g("foo")   # c: str
d = g(b"foo")  # d: bytes

Could you do the same for a function which might have multiple types as it’s input and output? As well as if a parameter of a function had a union of just 2 types, could you replace that with a TypeVar, with or without constraints?
Can you give an example of what you mean?

[cedar sundial]

Yeah, I’ll get on my computer since it’s much easier to type 😅

def foo(a: str | float | int) -> str | float | int: ...

T = TypeVar("T")
def foo(a: T) -> T: ...

Would this be acceptable to reduce the n number of types from a union down to a single TypeVar. Would you have to declare the types in the TypeVar if you explicitly wanted only them types?

For any function that has a union of 2 types def foo(a: str | int) -> str | int: ..., would you also reduce this down to a TypeVar with constraints or keep as it is?

[trim tangle]

These are different types

See this <#type-hinting message>

If you want exactly the same behaviour. that would be just a type alias

IdSource = str | float | int
def foo(a: IdSource) -> IdSource: ...

In this case, the output type is not linked to the input type in any way. So foo(42) is either a string, a float or an int, not just an int.

[cedar sundial]

So this isn't valid?

T = TypeVar("T", str, int, float)
def foo(a: T) -> T: ...

[trim tangle]

This is valid

but it's different from ```py
def foo(a: str | float | int) -> str | float | int: ...

[cedar sundial]

I think I understand but I wrote this in the sense that the type of a would always be the return type too

[upbeat wadi]

Why is ... generally used for defaults in overloads instead of the actual default value? i.e. ```py
@overload
def foo(a: int = ...) -> str:
...

@overload
def foo(a: str) -> int:
...

def foo(a: str | int = 0) -> str | int:
....```

[trim tangle]

If you want to link the return type to the input type, then yes, this is how you would do it

[trim tangle]

It's used in overloads and stubs to signify that there's a default

it's not really part of the type

[upbeat wadi]

wouldn't this do that as well, while telling the user what the default is?```py
@overload
def foo(a: int = 0) -> str:
...

[trim tangle]

Yes, that is valid as well

[upbeat wadi]

is there some sort of "style" recommendation for which one to use?

[trim tangle]

I don't think so

just pick a style and stick to it 🙂

[upbeat wadi]

alright, thanks!

[trim tangle]

I wonder if we need some kind of "PEP 8 + 3/4" for typing

[cedar sundial]

Ok, makes sense. Where would you use Generic[T] since it seems from the previous examples that only the T is used for parameter/return types

[trim tangle]

A generic type is a type parametrized by another type.
In particular, a generic class is a class parametrized by another type. Example:

T = TypeVar("T")


class Box(Generic[T]):
    def __init__(self, initial_value: T) -> None:
        self._value = initial_value

    def set(self, new_value: T) -> None:
        self._value = new_value

    def get(self) -> T:
        return self._value

box = Box(42)
reveal_type(box)  # box: Box[int]

box.set(69)  # ok
v = box.get()  # v: int

box.set("foo")  # error

def f(box: Box[int]) -> None:
    box.set(42)  # ok
    box.set("foo")  # error

[cedar sundial]

Ok, so just used in classes?

[trim tangle]

Yes, Generic only exists to create generic classes like this

Generic[T] on its own doesn't work

or well, doesn't make sense

[cedar sundial]

Ok, got it since Generic is it's own class anyway

[soft matrix]

if the default value can be returned ie (int, T = None) -> int | T make sure to specify the default

[upbeat wadi]

alright

[trim tangle]

Does anyone know of a flake8 plugin that enforces PEP 585?

Because I was going to write one

[oblique urchin]

we're working on https://github.com/pycqa/flake8-pyi based on typeshed's needs

there's an issue about running it on .py code too

[trim tangle]

I think I'll just hack something together 🙂

Should be simple enough if you're importing names directly from typing

[mortal fractal]

It looks like David bought my line of thinking and Mehdi found the comment, cool

[mortal fractal]

I'm bored this weekend so let's try out pyre

https://github.com/facebook/pyre-check/issues/574

[trim tangle]

How does Pyre compare to Mypy and Pyright?

[mortal fractal]

I'm finding out 🙂

right now it has a handful of false positives in my code, but that doesn't mean it's worse per se, because my code was written with the quirks of mypy and pyright in mind already

and I've reported issues with pyright as I found them

[trim tangle]

This seems nice https://pyre-check.org/docs/querying-pyre/

I can imagine using some preprocessing nonsense magic to derive proxy values from types

[mortal fractal]

it seems to have failed to narrow some optionals in a way that is escaping my attempt to make a minimal example so I'm trying to figure out what's up atm

[trim tangle]

oh hey, they have a piece on variance
https://pyre-check.org/docs/errors/#covariance-and-contravariance

maybe I'll use it when I don't have time to explain it

[mortal fractal]

https://github.com/facebook/pyre-check/issues/575

finally got it

a few weird reqs made finding the minimal reproducer hard

[oblique urchin]

that is pretty odd, it complains about the body of the elif clause but not about the conditional itself?

[mortal fractal]

correct

I have (at least) one more (probably) unrelated false positive I'm working on

[mortal fractal]

huh, this bug is so weird i wonder if it's just a known quirk of how pyre handles asserts (....????)

import subprocess
from typing import IO

def wants_io(var: IO[str]) -> None:
    ...

def do_nothing() -> None:
    ...

with subprocess.Popen(
    "myprogram", text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT
) as result:
    assert result.stdout is not None
    do_nothing()
    reveal_type(result.stdout)
    wants_io(result.stdout)

calling do_nothing() makes pyre forget the assertion

https://github.com/facebook/pyre-check/issues/576

not sure what's going on here

okay I'll leave it at this for now and see if they're interested in more later

yes @soft matrix, very

was scratching my head on why my minimal reproducer wasn't working for quite a while

[oblique urchin]

maybe they forget any assertions on function calls in case the function call assigns to the narrowed variable?

people ask for that on mypy/pyright sometimes

[mortal fractal]

that was my best guess re: known quirk

maybe we can find it in pyre documentatino

that may be the source of the other problem too

[mortal fractal]

if that's the case and it's intentional then only the first issue I opened would be a bug

"The above fails to type-check because Pyre cannot guarantee that data.field remains not None if the interleaving logic between the explicit check and the later reference contains anything that may have side effects, like function calls."

[trim tangle]

I think that's something pyright does differently

[mortal fractal]

so I'm closing my latest 2 issues as intentional since it's documented behavior

[terse sky]

fwiw pyre is correct here I would say, even if it's not convenient

this is the closer to the behavior you see in statically typed languages with built-in nullability smart casting

like Kotlin/Swift

I guess the problem comes because it's a mutable field. If you were directly doing the assert on result, I think that should be ok, since result itself cannot be reassigned anywhere else

[mortal fractal]

it's unfortunate I can't work around it with asserts because pyright will error for unnecessary asserts iirc

[terse sky]

hah

that seems overly aggressive to error rather than warn there

[mortal fractal]

so either one or the other will error

[terse sky]

well, what you should be able to do is take a local reference

x = result.stdout; assert x is not None

this is similar to what you would typically do in Kotlin to get smart casting

[trim tangle]

thank god we don't have variable references in Python

[terse sky]

well, we have basically every other form of mutation under the sun with no safeguards to make up for it 🙂

[trim tangle]

"I just don't mutate stuff that is not supposed to be changed, what's the problem 😎 "

[mortal fractal]

i mean, you can fix this by annotating it Final, or at least you could if it wasn't assigned from a context manager

[trim tangle]

oh speaking of context managers

with my_ctx_manager():
    f()
    y = 42
    g()
print(y)
``` Is there any type checker that marks `y` possibly unbound on the last line in some circumstances?

[terse sky]

annotating result final will not help

well, shouldn't help

you'd need to annotate the field itself

[soft matrix]

i think thats what they meant

[trim tangle]

yeah, you can't really know which methods are mutating and which not

[terse sky]

"they" meant? I was referring to your comment?

[trim tangle]

Gobot is not Gabby

[terse sky]

oh damn

lol I'm sorry

brain not working well

[trim tangle]

Are you in compact mode?

[mortal fractal]

yes there's not a way to do it really

[soft matrix]

do you use compact mode?

[terse sky]

no

my brain is just in compact mode atm

again, apologies

[trim tangle]

lol

[soft matrix]

its fine

:P

[terse sky]

it would be ok if result were an instance of a frozen dataclass

[mortal fractal]

I think the best way to handle this if you wanted to make all of mypy-pyright-pyre happy is to just use intermediate variables

[trim tangle]

Sometimes mutation is sneaky

like

defaultdict 🙂

[mortal fractal]

sometimes you can move the assertion closer to the access, but that won't work for one of the other cases I reported

[terse sky]

it helps to understand the counter-example, btw

def subprocess.Popen(...):
    global global_result
    ...
    global_result = result
    return result

def do_nothing():
    global_result.stdout = None

since do_nothing takes no arguments, this is afaik the only possible way for result.stdout to become None

[oblique urchin]

pyright does. I think mypy does if you annotate exit right

[trim tangle]

ah, right

my colleague had some nasty bug where a random None happened out of, apparently, nowhere

and PyCharm doesn't have this check

[mortal fractal]

I don't see how this one can come down to the same thing https://github.com/facebook/pyre-check/issues/574

[trim tangle]

Hmm... this seems wrong though 😦

class MyCtxManager:
    def __enter__(self) -> int:
        ...

    def __exit__(self, *args) -> Literal[False]:
        ...


def h() -> int:  # error
    with MyCtxManager():
        return 42

(pyright)

huhh, None works though

[oblique urchin]

report a bug 😄

[soft matrix]

what seems wrong?

Literal[False] doesnt work but None does?

[trim tangle]

with None it passes, but with Literal[False] pyright thinks that the ctx manager can swallow exceptions

[trim tangle]

I will not report any more pyright bugs!!!

[mortal fractal]

wellll some context managers can swallow exceptions

[trim tangle]

Right, but if the __exit__ method always returns a falsey value, it never swallows the exception

pyright understands this with None, but not with Literal[False]

[mortal fractal]

src/pyffstream/encode.py:767:4 Prohibited any [33]: Explicit annotation for futurescannot containAny.

hahahaha I think this is actually genuinely a false positive

when writing a function for arbitrary futures that doesn't care about the return type they need to be futures of Any, object doesn't work

I don't think there's a way to spell that without Any, because typevars error when used alone 🙂

(and would be wrong anyway since it can be multiple types)

wow pyre's strict mode is really annoying

it wants you to annotate a lot of stuff even when it can figure it out

for attributes and globals it seems

...and captured variables

but some of these are unfixable without removing certain python patterns, like some captured variables are created in places that do not allow annotations, like context managers

so the complaint isn't even fixable really

it wants -> None on all the __init__ 😄

I haven't thought of a good workaround for this that satisfies all 3 type checkers though https://github.com/facebook/pyre-check/issues/575

[oblique urchin]

assign the attribute to a new local variable?

[mortal fractal]

yeah, I guess; it's gross but I think it's the best solution

[trim tangle]

make a precommit hook that adds # type: ignore to all lines

[mortal fractal]

I don't think I can satisfy pyre strict without refactoring the code to be written ad ifferent way

since it wants annotations for things declared in places that don't support annotations

it also doesn't like that I used Any in the future thing or that I used Any to avoid making a typeddict with a few thousand items

that can't be fixed without something like pydantic or typeddict support for something like a default type for all unspecified fields

as for non-strict errors, the only ones left in my code are https://github.com/facebook/pyre-check/issues/574 and it doesn't like that I initialize some instance variables in a method that's not __init__, instead wanting me to do it in __init__, but unlike other type checkers just telling it the type inside __init__ isn't enough to make it happy 😄

https://pyre-check.org/docs/errors/#53-missing-annotation-for-captured-variables

this one is just annoying

[trim tangle]

wtf

I think I'm not going to use pyre 😄

[mortal fractal]

it only warns about it in strict mode, at least

but it will widen the type anyway

my code (already compatible with strict mypy and strict pyright) was mostly compatible with pyre non-strict mode, after making the changes for how it bails narrowing with functions

but strict pyre wants me to annotate a bunch of stuff I don't want to (and some stuff I can't!)

[little hare]

/pypi pyre

[oblique urchin]

does it want you to use type comments?

[little hare]

!pypi pyre

[rough sluice]

pyre v1.0

An extensible, component based framework for specifying and staging complex, multi-physics simulations.

[little hare]

my bot has spoiled me lol

[oblique urchin]

iirc with x: # type: T was a thing

[oblique urchin]

you want pyre-check

[mortal fractal]

maybe? I could try, but a lot of the other attributes it wants me to annotate are just superfluous and I'm not particularly enthused about adding to my codebase

[little hare]

!pypi pyre-check

[rough sluice]

pyre-check v0.9.10

A performant type checker for Python

[mortal fractal]

pyre almost never infers the type of an attribute it seems