#type-hinting

1 messages Β· Page 65 of 1

acoustic thicket
#

Lmao

dapper yacht
#

Whats the PEP?

hearty shell
#

Yeah I am just looking at it again, and I cant possible imagine who would find this Callable[[Callable[Concatenate[F, P], None]], Callable[P, tuple[Response, int]]]
easier to understand then
((F, **P) -> None) -> ((P) -> tuple[Response, int])

#

677

terse sky
#

I mean I think both of these are a bit rough

#

and I'd use an alias either way

#

but in any case I think the main point was that it makes parsing harder, not that it's harder to read for people (at least that's what I understood)

trim tangle
#

where should this error?

dapper yacht
#

At the "/" I was guessing

#

Didnt know it was allowed

spiral fjord
#

It's for positional only arguments

acoustic thicket
#

It denotes the end of positional only args

soft matrix
#

only thing i find confusing is the binding order for ->

dapper yacht
#

Ooh, so its the partner in crime of "*"?

acoustic thicket
#

Yeah

dapper yacht
#

nice, another thing learned

hearty shell
terse sky
#

err that's not how software works? πŸ™‚

hearty shell
#

I am speaking from ignorance though

hearty shell
terse sky
#

Writing software is generally only a small part of the cost of creating software

#

maintaining it is much more expensive

hearty shell
#

But it is a parser

terse sky
#

Yes, and we're going to make changes to that parser in the future, right?

hearty shell
#

I honestly don't know how it is implemented, I am not doubting anything, I guess if I havent had to work with it, it is harder to undersatnd

terse sky
#

I don't know how it's implemented either, i'm just going by Jelle's comment

#

but also a statement like "why would it need to be revisited" as a blanket statement about anything in active software is just kind of incorrect

#

there's going to be another feature that needs to be there in a week, a month, etc, where the person implementing that feature is going to be impacted (negatively) by the additional complexity in the parser introduced by supporting this feature.

#

That's more or less inarguable; it's only a matter of degrees: a) how much extra complexity is this really? b) How much impact does this complexity have on future work? c) how much work is happening on this in the future?

oblique urchin
dapper yacht
#

Thanks, looks cool actually

hearty shell
acoustic thicket
#

I might be way off on this, but isnt the actual parser codegen-ed from a grammar file?

oblique urchin
#

you still need to write the grammar file, and create the runtime objects that the syntax maps to

acoustic thicket
#

Hmm

dapper yacht
oblique urchin
acoustic thicket
#

inchresting

hearty shell
#

(even though we have an advanced parser in
CPython, that does not mean community maintained projects such as
jedi, parso, black will have one).

#

that completely went over my head

#

Didn't even consider lol

acoustic thicket
#

Same

soft matrix
#

everyone just use libcst smh

oblique urchin
rustic gull
#

Kinda annoying to have to import

little hare
oblique urchin
#

Batuhan somehow made it able to parse match statements

little hare
#

πŸ’€

#

is Batuhan isidentical?

oblique urchin
#

yes

little hare
#

noice

#

I use libcst for my autohelp tool

fierce ridge
#

i wish it was just + or &

#

or at least let me use those as synonyms

rustic gull
#

Good idea. That looking better now? Is it right to indicate that from __future__ import annotations is the proper way to do it for older versions without using deprecated stuff? lemon_thinking

trim tangle
#

but generally, for application code, yeah

#

although I'm not sure

#

I don't think I understand how F[A2] is allowed

soft matrix
#

covariant constraints are a thing cause typeshed uses them

oblique urchin
trim tangle
#

yeah

#

he does that sometimes I think

#

maybe he just hates me and wants to close the issue πŸ™‚

soft matrix
#

he did that to me when i broke ci for a while asking for bidirectional type inference on lambdas

trim tangle
soft matrix
trim tangle
oblique urchin
trim tangle
#

new working hypothesis: I'm bad at english 😎

spiral fjord
#

Don't be so hard on yourself, you are far above average from what I've seen over the years.

trim tangle
#

well

#

I think it's just that I'm having trouble communicating my thoughts

mortal fractal
oblique urchin
#

@trim tangle a way to get more data on variance + constraints could be to write a mypy PR that gives an error in this case, then look at the mypy-primer results

rough sluiceBOT
#

mypy/semanal.py line 3193

def process_typevar_parameters(self, args: List[Expression],```
trim tangle
#

i am allergic to mypy code πŸ‘€

#

but I'll take a look, thanks

mortal fractal
rustic gull
acoustic thicket
#

if you have no information about the keys and values beforehand then yeah, dict[Any, Any] is the only option

fierce ridge
#

do you have any additional information about the structure? eg if it's one of 3 different options, but you know all 3 options

#

you could also use object which assumes that you know "nothing" about the contents

#

object is maximally restrictive, while Any is maximally permissive

hearty shell
soft matrix
#

Self isn't the self parameter it's the type of the self parameter

#

And hence implementing copy as (self) -> Self is ok to type checkers

hasty hull
#

Pyright doesn't like the @task line: ```py
from typing import Callable, TypeVar

Task = Callable[['User'], None]
F = TypeVar('F', bound=Task)

def task(func: F) -> F:
...

class User:
pass

class MyUser(User):
@task
def foo(self) -> None:
pass

#

Is this a bug or am I missing something?

soft matrix
#

I think bluenix had a similar problem but idk how that ended

hearty shell
#

Is it not because of contravariance?

soft matrix
#

I think that might be the issue yeah (Callable's arguments are contravariant)

hearty shell
#

Callable[['User'], None] <: Callable[['MyUser'], None]

#

Although you could do

T = TypeVar('T', bound='User')

def task(func: Callable[[T], None]) -> Callable[[T], None]:
    ...
hasty hull
blazing nest
# hasty hull Pyright doesn't like the `@task` line: ```py from typing import Callable, TypeVa...

Yeah Callables have some odd variance by default (it may not have been invariant, I just remember that it didn't work). I had a super similar case to yours here, with me wanting to accept a callable with a base class that I then pass some subclass of (in reality that is fetched from the typehints so I know for a fact that the user can handle this subclass) but I had loads of issues. Lookup discussions from me on the Pyright repository.

acoustic thicket
#

yeah callables are contravariant wrt args and covariant wrt return type

trim tangle
#

The newest Pyright version has this behaviour: ```py
from typing import Protocol, Literal

class IFalsey(Protocol):
def bool(self) -> Literal[False]: ...

class Falsey:
def bool(self) -> Literal[False]: ...

def f(x: IFalsey) -> None:
if x:
reveal_type(x) # IFalsey
else:
reveal_type(x) # IFalsey

def g(x: Falsey) -> None:
if x:
reveal_type(x) # Never
else:
reveal_type(x) # Falsey

#

mypy doesn't do any narrowing with Never in either case

hearty shell
#

What was wrong with narrowing IFalsey?

trim tangle
#

I'm not sure if this is a problem. I can't really imagine the use of a protocol that's always falsey or truthy

hearty shell
#

Ahh, I see although the list of classes that by defualt implement a bool inst large, why not add exceptions?

#

Particularly to any Container, Number, and NoneType, isnt that all of them?

soft matrix
#

jokes on you int isnt a subclass of Number (to the typechecker)

oblique urchin
acoustic thicket
#

hah

hearty shell
#

Brought to you by Python Typing

hearty shell
#

Or actually just Iterable, either way it seems weird to assume every object is object in the std is truthy when many of them define len and bool

oblique urchin
#

I actually think if iterable and if container should be avoided, since the protocols don't define any meanings for those operations

#

And there are iterables (e.g., generators) for which truthiness doesn't imply that it's nonempty

trim tangle
#

if iterable definitely should be avoided

soft matrix
#

It's fine for implemented classes though right?

trim tangle
oblique urchin
oblique urchin
#

if it's typed as an Iterable it's not

trim tangle
oblique urchin
sweet whale
#

Might anybody know why I can't typehint a linked list node to itself?

class Node:
    def __init__(self, value, next=None) -> None:
        self.value = value
        self.next: Node = next

I can do it inside of __init__ but not actually within. It'll say something weird like:

unknown.png
hearty shell
#

You need wrap Node around quotes

pastel egret
#

At the point the function is being defined, the node class itself isn't, and so Node is an undefined variable.

hearty shell
#

or from __future__ import annotations

sweet whale
#

Wrapping it in quotes works, is that good practice though?

buoyant swift
#

didn't they add typing.Self in 3.11?

hearty shell
sweet whale
#

Yeh I'd like to know the best practice way to do it

hearty shell
#

it usually affects very little, I would say to go with the import

sweet whale
#

How does the import work?

hearty shell
sweet whale
#

I haven't heard of the __future__ module before

hearty shell
#

You could import from typing_extension

void panther
sweet whale
#

So wrapping it in quotes?

void panther
#

future imports are special in that they can be read by the compiler to change stuff before runtime, and annotations does the above

#

basically, but implicitly

sweet whale
#

So, it implicitly wraps it in quotes without me actually doing it?

#

doesn't seem to be complaining now

unknown.png
hearty shell
#

That should be an optional though

void panther
sweet whale
hearty shell
#

=None is just the default value, but the default value is not compatible with the type you are providing

#

They are different things, one is you declaring the parameter to have a certain type, and the other is jsut the default value you give

void panther
#

Optional is for the typing, None is for python

sweet whale
#

Oh i see

hearty shell
#

TLDR xD

#

Although you can use Node | None

#

if you are in 3.10

sweet whale
#

I am actually, I'll give that a try as well

void panther
#

or 3.7+ with the annotations import

sweet whale
#

Seems like the annotations import is working! Thank you all for the help, i've actually seen that import a few times before but wasn't sure what it actually did :p

hallow flint
#

Optional just means "None is a valid value for this argument"; it has basically nothing to do with the presence of absence of a default confusion. this is a very common confusion, and most people acknowledge that Optional was poorly named

fierce ridge
oblique urchin
#

PEP 677 (callable syntax) was rejected

novel saffron
mortal fractal
#
  1. While the current Callable[x, y] syntax is not loved, it does work. This
    PEP isn’t enabling authors to express anything they cannot already. The PEP
    explicitly chose be conservative and not allow more syntax to express
    features going beyond what Callable supports. We applaud that decision,
    starting simple is good. But we can imagine a future where the syntax would
    desire to be expanded upon.
#

I'm having trouble understanding what they're saying

#

Are they presenting the ability of the syntax to accommodate additional items as a negative, or a negative that the syntax was too simple to start with

#

I'm trying to understand what the takehome is. Is it that we we shouldn't leave possibilities open for future syntax expansion?

inner sand
#

any pydantic experts in the house?

#

I have two questions:

  1. I have a model that needs to be extended to four different models, each with different Literals for the same parameter. I can create new classes for each of the four models and subclass the main model (see example below), but it adds a lot of boiler plate when all that's changing is the value of the Literal. Is there a shorthand way to do this?
class MyModel(BaseModel):
   ...

class Model1(MyModel):
   foo = Literal["bar"]

class Model2(MyModel):
   foo = Literal["foo"]
...
#
  1. I am creating a model from a TypedDict using create_model_from_typeddict but also want to modify an existing attribute in the model which starts out with a str but again I want to override it to become a Literal. How do I accomplish this? I have to imagine that the answer to 1 is also the answer to 2, or if not, similar. Perhaps there is something I can pass in to the function to handle this?
inner sand
#

ok solved the second one by passing the overridden attribute and type into the function

#

I guess the question then is is there a helper function to create a model from an existing model

#

oh, I think I got it

soft matrix
#

the syntax proposed was shown to be backwards compatible with any new hybrid syntax if it was added

hallow flint
#

i found this confusing too, so i replied asking for clarification

soft matrix
#

It's not on typing sig or the dev mailing list is it?

trim tangle
#

idk why, but I find contributing to a mailing list discussion really scary

#

seems kinda unnatural as opposed to, say, github discussions

#

maybe I just never used mailing lists

grave fjord
grave fjord
trim tangle
hasty hull
#

πŸ‘€

#

😳

trim tangle
#

ooh

#

nice

hasty hull
#

quite a few people are using it as well, it gets like 10k downloads a week

soft matrix
#

What do you guys think of type defaults for TypeVars?

T = TypeVar("T", default=int)

@dataclass
class Box(Generic[T]):
  value: T | None = None
# This means that if no type is specified T = int

reveal_type(Box())  # type is Box[int]
reveal_type(Box(value="Hello World!"))  # type is Box[str]

It would also solve a common problem for things like Coroutine where people want to only give it one argument (The return type)

SendT = YieldT = TypeVar(..., default=Any)
ReturnT = TypeVar("ReturnT")
class Coroutine(Generic[SendT, YieldT, ReturnT]): ...

Coroutine[str]  # fine
Coroutine[Any, Any, None]  # works fine as normal
Coroutine[None]  # would be equivilent to the above
oblique urchin
#

typing.Generator is the obvious use case I feel like

soft matrix
#

same πŸ˜‰

#

yeah generator as well although that would be for the yield type right?

trim tangle
#

I hate having to google the order every time

oblique urchin
#

One thing I was worried about is whether there would be enough convincing use cases if we can't have named TypeVar arguments

trim tangle
oblique urchin
#

like I'd want to be able to write Generator[int, ReturnT=bool] or something

#

but that's PEP 637 which was rejected

soft matrix
#

that would be nice

#

damn

oblique urchin
#

It might get accepted the second time around with a more convincing use case, but it will be an uphill struggle

hasty hull
#

Why was it rejected?

oblique urchin
hearty shell
#

always the damn syntax blobsad

hearty shell
#

but even if it
were, at this point we’re reluctant to add general Python syntax that only
(or mostly) benefits the static typing language.

oblique urchin
soft matrix
hearty shell
#

Yeah, only because there was evidence that it would be useful outside of typing

soft matrix
hearty shell
#

From the comments that he makes on typing issues, I think he is too pragmatic, we need someone less pragmatic to add new syntax for typing only xD

trim tangle
#

I mean, Generator[{"return": int, "yield": str}] technically works

#

I know it's a bit noisy, but still

oblique urchin
soft matrix
#

The grammar changes weren't that disruptive were they?

oblique urchin
hearty shell
#

I was just thinking of [*T]

soft matrix
#

i think the changes made to getitem were useful in other places

#

yeah that

mortal fractal
#

re use cases

blazing nest
mortal fractal
#

I want the special syntax for covariance/contravariance, and Optional

#

I'm bearish on the chances rn though

#

Particularly the optional one

soft matrix
#

special syntax for co/contra shouldnt be that bad

hearty shell
#

Which pep is that?

mortal fractal
#

Unless none aware syntax pep gets accepted, then the optional syntax is probably a shoo-in

terse sky
#

I feel the absence of the none-aware stuff pretty much daily

blazing nest
#

Didn't the author stop because they felt None would be starting to be used too much?

mortal fractal
#

Someone else posted to ML wanting to pick it up and I haven't heard more since then

terse sky
#

I have no idea, but that seems like a pretty silly reason to me. All the cases in question, I still end up using None.

#

It's just more painful.

blazing nest
#

Because I would agree with that in all honesty. Raising errors in properties are more pythonic than a bunch of Nones.

terse sky
#

i thought it was simply deferred by the steerco for the usual reason; not clear if the benefit is worth the syntactic changes

#

you're assuming that it's for error handling

mortal fractal
#

The SC hasn't taken a position on it

terse sky
#

in fact it's not about error handling at all

blazing nest
#

What are you using daily that's causing you to grab attributes of Optional[X]?

mortal fractal
#

Guido said in the thread that revived it that he wanted to see it accepted and then there was a bit more debate

terse sky
#

they deferred it didn't they, so obviously they're not excited about it?

mortal fractal
#

This was a few months ago

soft matrix
terse sky
#

Daily, you try to get data or calculate things, and some data isn't present, and you keep going. You either omit that data from final output, replace it with a default, etc.

#

maybe a directory is an optional argument, and if it's passed, you want to perform some concatenation operations on it, and if it's not passed, you leave it as None.
you end up with these eyesores everywhere for very simple operations

mortal fractal
#

sry didn't mean to reply

blazing nest
#

It's been like over 30 years of Python and we haven't had a ! or ?, they feel un-pythonic πŸ˜…

terse sky
#

by that reasoning we can just stop accepting proposals for new syntax πŸ™‚

blazing nest
mortal fractal
#

Yes

terse sky
#

Pretty much all languages these days have some kind of support for either Optional, or None/Null, semantics

blazing nest
terse sky
#

Here's an example of a gem like this:

def get_as_date(row: Mapping[str, str], key: str) -> Optional[datetime.date]:
    try:
        return company_date_to_date(int(row[key]))
    except (KeyError, ValueError):
        return None
#

I end up writing functions like this all over the place

mortal fractal
#

I wouldn't mind not having none aware syntax if it wasn't actually the case -in real life- that stdlib and pypi libraries expect me to handle None all the time

terse sky
#

it's just so much boilerplate. In Kotlin I'd probably have something like row[key]?.intOrNull()?.companyDateToDate()

#

when the language gives you tools to nicely compose small things together, you can compose them on the fly and understand what's happening

#

with python, the composition tools suck, so you end up with these pointless, verbose, brittle, 4-5 line functions everywhere

hasty hull
void panther
hasty hull
terse sky
#

well, just handling defaults

#

because of python's spectacularly weird edge case with mutable defaults

acoustic thicket
#

How would yall have felt about a superset language for typing syntax, keeping python itself unchanged? Like js/ts

oblique urchin
#

Some people have tried this sort of thing with # encoding hacks (not for typing though), and apparently it usually doesn't work well

acoustic thicket
trim tangle
#

When TypeScript came around, minifying and babelifying JS was already a battle-tested thing

hearty shell
#

Imagine minified python, with semicolons for all statements that allow it

covert dagger
#

it doesn't have to be minified though, it could be like running black or pyupgrade

hearty shell
#

I am joking x)

covert dagger
#

yeah I'm just speaking about the overall idea

#

problem would be that all other tools would have to do different things for typed-python (tython?) and not-typed python

oblique urchin
#

more like pyc files

covert dagger
#

oh right, you'd not want to debug the compiled code

hearty shell
#

So, Python for no types, Cython for C 'types', Tython for uncensored types everywhere

#

I like the name xD

hallow flint
soft matrix
#

thanks for that

fiery goblet
#

Wondering what peoples thought are for typechecking tools? I've only used mypy.
Any recommendations? I will be wanting to use it both locally and in CI workflows

hasty hull
#

Eric is unbelievably fast at fixing bugs and adding features

trim tangle
#

yeah

#

pyright also provides autocompletion in a variety of editors (VSCode, Atom, Emacs, Vim, ...)

hearty shell
#

Are pytype and pyre that bad, no one ever recommends them x)

oblique urchin
#

mypy gets a lot of usage because it was the first widely used type checker for Python

#

and pyright because it's really easy to use from vscode

trim tangle
#

I used mypy initially, but then switched to pyright

hearty shell
#

Yeah that is true, pyright is just default for a lot of users, and those who dont mypy does seem to be the first option

hasty hull
oblique urchin
#

pytype is kind of weird in my experience. In typeshed CI we occasionally see it crash in very opaque ways

trim tangle
oblique urchin
#

I once tried it on our codebase at work and it ran for hours, then spit out massive amounts of incomprehensible stuff

#

so not a great experience

trim tangle
# trim tangle yeah mypy is _slow_

My biggest gripe with mypy is that it doesn't provide autocompletion/types on hover. I get that that is not its purpose, but it really kills half the benefits of static typing

#

ok maybe not half, but a reasonable part

hasty hull
trim tangle
#

I have a walnut sized brain so I need a smart editor to get anything done

hasty hull
#

same lol

fiery goblet
nova venture
fiery goblet
#

lol it uses NPM underneath?

oblique urchin
fiery goblet
#

Woah

#

why

oblique urchin
#

That way it integrates better with vscode

fiery goblet
#

ah

oblique urchin
#

@hasty hull maintains a package that lets you pip install it, but it just calls out to npm

soft matrix
trim tangle
#

yeah, as absurd as it sounds, it is faster because of JS

#

and it still has cursed performance hacks

#

like putting the core logic into a single 20000-line function

blazing nest
hearty shell
#

Oof, yeah I would not download OCamal just to compile Pyre

hasty hull
#

Yeah @fiery goblet you can use pyright as if it was native python as my package handles downloading node for you if you don't already have it

#

And it also automatically keeps pyright up to date

fiery goblet
fiery goblet
hasty hull
#

Not currently but that is a good suggestion

#

You can achieve the same effect by running pyright --help

fiery goblet
#

ah, i'll do that. ty

#

saves me from this

    && $HOME/.poetry/bin/poetry run pyright /tmp || true  # Installs NPM requirements
hasty hull
#

I actually haven't tested how my wrapper works within a docker container so please let me know if you run into any issues

soft matrix
#

does the wrapper allow you to reveal_type on things without actually having reveal_type over the variable?

hasty hull
#

It doesn't have any type checking features on top of pyright, so if the pyright CLI can do that then the wrapper can do it

#

how would that work? by a command line flag?

#

like pyright foo.py --reveal-type bar?

soft matrix
#

i thought it supported the language server

hasty hull
#

I forgot about that lol

#

but that just exposes the language server entrypoint

#

it finds the JS file to start the language server and runs that

soft matrix
#

have you ever tried to use for anything, id like to switch my very bad type finder script for docs from mypy to pyright so it doesnt crash when i use features mypy doesnt know about

#

how does vsc et al do it then?

hasty hull
#

Oh it just clicked what you mean, yes you could do that but it would be very hacky

#

I don't know anything about the LSP protocol

#

You would need to start the language server and send it commands somehow

#

That's how vsc and co do it

#

I know that one method you can use is JSONRPC but I don't know what the structure of the commands is

trim tangle
acoustic thicket
acoustic thicket
maiden mesa
#

What am I doing wrong?

def sign(self, signers: Optional[list(PublicKey | Keypair)] = None) -> None```
```TypeError: 'types.UnionType' object is not iterable```
hearty shell
#

You are calling list on the Union, you need to use subscription for generics

#

list[PublicKey | Keypair]

maiden mesa
#

Oh my, dang how did i not see that

#

Thanks

hearty shell
#

Np

trim tangle
grave fjord
maiden mesa
maiden mesa
grave fjord
#

tuples are immutable

#

So it's fine if every item in the tuple is immutable

maiden mesa
#

Ohk

maiden mesa
grave fjord
#

Tuples are Iterable

grave fjord
maiden mesa
#

yeah, it will run some validations which will add items to it if it's empty or the required length is more

#

One of the class attributes needs to be appended to it as well, and if I am not wrong; I can't access any attribute in the function header itself

grave fjord
maiden mesa
grave fjord
#

No the caller

trim tangle
#

Why not just return a new list?

rustic gull
#

What does ((str) -> _T@add_argument) mean in this type hint? It's the -> I don't get - I've only seen it used for return types.

unknown.png
hearty shell
#

I think that is how Pyright shows its callables

#

So type would either take a FileType or a callable with str as a param and return some T

#

Not sure though

dapper yacht
#

Whenever I put Iterable as a typehint.. Doesn't that mean that my function should also handle strings? Since theyre Iterables

oblique urchin
dapper yacht
#

Usually I just want to ducktype away on dicts/tuples/sets/lists..

#

Would the correct way be to typehint Union[dict, list, tuple, set])?

oblique urchin
dapper yacht
#

I know, I'd add int or whatever to it, the question is more about: How do I exclude strings from my "Containers"-type hint

#

even though I guess technically str is also a container

oblique urchin
#

I would just stick with Iterable. There is some talk about adding a Not type that would let you say "any iterable except str"

void panther
#

why do you want to exclude it?

#

i.e. why does the difference between a string and an iterable of strings matter?

dapper yacht
#

I'm not sure I understand the question

#

!e

def myfunc(my_iterable):
  for i in my_iterable:
    print(i)

myfunc("string")
myfunc(["str1", "str2"])
rough sluiceBOT
#

@dapper yacht :white_check_mark: Your eval job has completed with return code 0.

001 | s
002 | t
003 | r
004 | i
005 | n
006 | g
007 | str1
008 | str2
hearty shell
#

I think they were just asking what is the situation in which your function could accept any iterable except a string

dapper yacht
#

now if I wanted to print every username in a list, id iterate over a list (like above).
Do you mean I should do something like this?

#

!e

def my_func(my_iterable):
  if isinstance(my_iterable, str):
    my_iterable = my_iterable.split(",")
  for elem in my_iterable:
    print(elem)

my_func("asd,123,bcd")
my_func(["asd", "123", "bcd"])
rough sluiceBOT
#

@dapper yacht :white_check_mark: Your eval job has completed with return code 0.

001 | asd
002 | 123
003 | bcd
004 | asd
005 | 123
006 | bcd
void panther
dapper yacht
#

Yeah, but isnt the point of type hinting to know the contents of the supplied object?

void panther
#

its job is to take care of the types, not the actual runtime contents (most of the time)

#

if you were to exclude strings for that reason, should list("string") also be invalid?

dapper yacht
#

what if I don't want to have strings in my Iterable

#

any object that is not a string

#

What if I wanted to convey, that theres an Iterable with instances of MyClass?

# 
def my_func(iterable: Iterable[MyClass]):
  pass
#

Does that automatically exclude strings?

void panther
#

Yes

dapper yacht
#

I might just be very confuzzled

void panther
#

it'll only allow iterables of MyClass

dapper yacht
#

That makes sense

#

I don't even understand where my misunderstand was to begin with

void panther
#

if you know what you want to exclude then you most likely also know of a generic parameter to give it

trim tangle
#

You could make an overload that returns NoReturn, but that's kinda weird

dapper yacht
#

What if I wanted to accept a list of dict-keys, a Iterable[str] wouldnt be very fitting because it includes str right?

void panther
#

it's still an iterable of strings that can be used as keys

dapper yacht
#

Yeah but would it be a sensible type hint for that function?

void panther
#

If that's what it needs, sure

dapper yacht
#

I think I was trying to hard to make the function scream "Enter sensible dict keys here". But that's not entirely the point of type-hinting/duck typing?

void panther
#

yeah that's more of a job for the docstring

dapper yacht
#

Ah ok

#

Damn

#

Its just to check (dynamically or statically) if types match and the program flow wont stop unexpectedly

void panther
#

if it's crucial that it can't be 1 char strings for some reason, then you can validate it at runtime

dapper yacht
#

be it mypy or some IDE highlighting stuff

#

I hate validation

#

I'd rather have the user (mostly me) validate what he calls the methods with beforehand

#

Cheers though

#

thank you

hearty shell
#

is if pkt.mode != -1: your way of narrowing?

#

from Union[list[Packet], list[str]] to list[Packet]

oblique urchin
#

that's not a design that's very compatible with type checking

hearty shell
#

I mean, you can use a typeguard if you change some stuff around

#

One sec

#

Actually, not sure how to do this one outright x)

#

either way it is supposed to narrow your types

rough sluiceBOT
#

day16/part1.py lines 15 to 23

class _Packet(Protocol):
    @property
    def version(self) -> int: ...
    @property
    def type_id(self) -> int: ...
    @property
    def n(self) -> int: ...
    @property
    def packets(self) -> tuple[_Packet, ...]: ...```
hearty shell
# 
from typing import TypeGuard, Union, Protocol
from dataclasses import dataclass

class _Packet(Protocol):
    sub_packets: list[Packet]

@dataclass
class Packet:
    version: int
    type_id: int
    mode: int
    sub_packets: Union[list[Packet], list[str]]
    
def is_type_packet(pkt: Packet) -> TypeGuard[_Packet]:
    return pkt.mode != -1

packet_heap: list[Packet] = []
if is_type_packet(pkt): # pkt is an object of type Packet
    packet_heap.extend(pkt.sub_packets)
#

Or maybe it would be better to use Generics? Either way this seems to work

#

is_type_packet now is the guard, if the object you passed, pkt returns true, that means it adheres to the type of the TypeGuard, which in this case it the _Packet Protocol, which always has a sub_packets of type list[Packet]

#

Although if you try to do anything else with it that you could with a normal Packet the type checkers will complain since now your object only follows the protocol and not you original Packet

#

In these instances I would honestly just cast and forget about it x)

#

Or type ignore as you said

#

@hidden spoke

fiery goblet
#

protocol is typing where you define what something can do, so you don't have to type by inheritance. imo Protocol works best for typing function signatures, this seems like a weird example

#

i.e

# 

class Edible(Protocol):
  def eat(self) -> None: ...


class Candy:
  def __init__(self, colour: str) -> None:
    self.colour = colour

  def eat(self) -> None:
    print(f"Chewy {self.colour} Candy! Nom nom.")

class Fruit:
  def __init__(self, name: str) -> None:
    self.name= name

  def eat(self) -> None:
    print(f"Yummy {self.name} fruit")



def digest(food: Edible) -> None:
  food.eat()
  del food # or something, 


yellow_candy = Candy("Yellow")
apple = Fruit("Apple")
digest(yellow_candy)  # Implements `eat` so it is correct typing
digest(apple)
terse sky
#

That example is pretty weird because the protocol references the concrete class

#

usually, one of the main points of using protocols, is that the Protocol doesn't have to know about the concrete class, and the concrete class doesn't have to know about the protocol

hearty shell
#

Looking back at it, it also doesnt have to be a protocol, just subclassing Packet was enough and it also made sure it had all methods

#

that was a terrible use a Protocol x)

fiery goblet
#

i usually only protocol type for methods/groups of methods

#

lmk if you disagree ^

terse sky
#

yeah I don't think that the issue here was one of using language features like Protocols, or type checking

#

it's more a matter of design

#

having a static type depend on a dynamic value, basically.

#

you really want to express that within the class, because it's a class invariant

#

well, the idea with type checking is that its static, right

#

but a value like pk.mode is dynamic

#

outside of very very trivial cases, it can't really know if pkt.mode is -1, without running code

#

The type checker can't understand that whenever mode has a certain value, it's going to influence the type

#

So, this is the kind of place where python starts to suffer because the obvious solution here is a lambda

#

but python lambdas suck

#

but, it's probably still what I would do

# 
@dataclass
class Packet:
    version: int
    type_id: int
    mode: int
    sub_packets: Union[list[Packet], list[str]]

    def apply_to_non_str_sub_packets(self, action: Callable[[list[Packet]], None]):
        if pkt.mode == -1:
            raise RuntimeError("Error, stored subpackets are strings!")
        action(cast(self.sub_packets, list[Packet]))
#

maybe something like that?

#

Alternate approaches: a) Just don't allow a list[str] to begin with, process it into a packet. b) maybe some kind of inheritance hierarchy.

trim tangle
#

or a tagged union

terse sky
#

Yeah, i haven't been following pattern matching, so I don't know where that's at

#

this is also kind of a painful situation in python because you start to see breakdowns between the static type system and the dynamic type system

#

Union[list[Packet], list[str]], translates to a dynamic type of just... list

fiery goblet
trim tangle
#

let me dig up something

#

It's an algebraic data type. a.k.a. union type, sum type, discriminated union, tagged union, variant record, also mentioned at times as Rust enums, sealed traits, disjoint union, variant, choice type, coproduct, disjoint coproduct, tagged variant, product dual, tagged product dual, discriminated coproduct, intuitionistic logical disjunction under the Curry–Howard correspondence, and the embodied thought process of Henry VIII

#

Basically, you have a type that can take a few fixed structures, like ```hs
data PaymentMethod = Cash | CreditCard String | Voucher VoucherId

terse sky
#

If the core language is standardizing pattern matching, they should really standardize a proper sum type

#

because depending on the dynamic type, as per the current situation, runs into issues like the one above

fiery goblet
#

time to google sum type kek its the same thing πŸ‘€

trim tangle
#

You can already implement them with some hackery. But, of course, type checkers don't like that @terse sky

trim tangle
terse sky
#

Right. That's why it seems silly to me.

#

The current approach has these weird edge cases

trim tangle
#

You can just make a union of dataclasses

#

or NamedTuples

terse sky
#

yeah it's just weird though

#

and unintuitive

#

and with boilerplate you can't eliminate

# 
@dataclass
class HoldsListPacket:
    data: list[Packet]

@dataclass
class HoldsListString:
    data: list[str]

@dataclass
class Packet:
    version: int
    type_id: int
    mode: int
    sub_packets: Union[HoldsListPacket, HoldsListString]
#

truly, a thing of beauty πŸ˜›

#

i'm not even necessarily pro patma for a language like python, but if you're already doing patma then to me it just makes sense to also be able to handle this properly

hearty shell
#

If I have a packet, and then I get the sub_packers, I still only have a Union, meaning weather I can or can't do something like Packet.sub_packets.data[0].version or Packet.sub_packets.data[0].upper() is still dependant on the mode, which the static checker is unaware about

mortal fractal
#

I kinda wish callable syntax had just been deferred for a few years instead of rejected

#

maybe it can be revisited if it's ever decided type annotations can have their own syntax a long time from now

terse sky
#

if mode only has two values, then at that point, you don't really need to store mode at all; you can derive it from the isinstance results

#

isinstance is basically one of the places where python's static and dynamic type systems meet directly. And in this case, you can see it's not a happy meeting.
When you have Union[list[Packet], list[str]], to the static type system this is all distinct and logical and fine.
The problem is that the way you use Union right now in python, it typically depends on isinstance checks, and isinstance checks only use the dynamic type, not the static type

#

and the dynamic type of that thing is always just list, you can't distinguish between the two cases, unless you start iterating the elements, which is just silly

hearty shell
#

Ohhh It set in now, you are eliminating the need to depend on a dynamic variable by using isinstance checks instead.

terse sky
#

yah. but isinstance doesn't work very well.

#

in this case.

hearty shell
#

That is why you are wrapping it around an object right

terse sky
#

sum/discriminated union types in othe rlanguages don't work this way, they don't just depend on one of the values being there.

#

Yes.

hearty shell
#

I see, I dont really know anything else so it kind goes over my head

#

At first sight I dont see what is wrong with the dynamic variable x) But yeah it would be cool if python could redesign the isinstance check or add some runtime behaviour that conforms to a static Union

terse sky
#

if you have a separate dynamic variable, then the static type checker doesn't understand this relationship

#

how can it, after all? They're two completely unrelated fields from the perspective of the type checker, it can't even begin to verify an invariant as complex as that, globally

hearty shell
#

I mean to say at first I didn't see, I see now, because those two are completely unrelated

#

But I guess the only thing I wonder is what would be some valid uses of TypeGuard in Python

terse sky
#

In a language like say, Rust, a Rust enum stores an integer like this, as part of the block

#

the discriminant

#

and then you can do if statements or pattern matches on the enum, and operate on it.

#

well, now I'm curious πŸ™‚

trim tangle
terse sky
#

The thing is that the typeguard, afaics, is just in the end a fancy wrapper around isinstance, so you still have the same nonsense I was pointing out before:

def is_str_list(val: List[object], allow_empty: bool) -> TypeGuard[List[str]]:
    if len(val) == 0:
        return allow_empty
    return all(isinstance(x, str) for x in val)
terse sky
buoyant swift
#

which is why it leaves the layout unspecified, ✨ optimizations ✨

terse sky
#

that doesn't actually matter because presumably the discriminant function is still available

trim tangle
#

yeah it doesn't impact usage at all

terse sky
#

but yeah, I feel like generics are really where you start to see a lot of the wonkiness in python's "dual" type system

#

it has all of the wonkiness of Java type erasure but much less protection

trim tangle
#

To add to your point: the is_str_list function has a subtle issue. It doesn't guarantee that the list is actually a list[str].

terse sky
#

right

#

list is not covariant so this check isn't even good enough

trim tangle
#

I meant that someone else could hold a reference to it as a list[object] and push some nonsense to it later

terse sky
#

well, that's a separate issue

#

that would be mutating it later on

#

at the time the function is run, it's still what it says it is

#

the problem is that it's still not correct

#

a list[Animal] cannot have [Cat(), Dog(), Elephant()]

trim tangle
#

it can

#

this is ok ```py
things: list[object] = [1, "foo", (1, 2, 3)]

#

This is surely correct: py animal: Animal = Cat() and you can add an Animal to a list of animals: ```py
animal: Animal = ...
animals.append(animal)

terse sky
#

hold on, I'm trying to sort out if I've just confused myself πŸ™‚

trim tangle
#

what you can't do is you can't assign a list[Dog] to a list[Animal] variable. But you can assign a Dog to an Animal variable.

#

(because T is covariant in T, to speak in scary words)

terse sky
#

yeah, I think you're right. This is one of those cases I always get a bit confused on.

trim tangle
#

there is a difference between the type of a variable and the concrete class of a value, I think that's a bit confusing

#

as in, here py animal: Animal = Dog() the (conceptual) type of the animal variable is Animal, but the value it holds is an instance of Dog

terse sky
#

at any rate though. Iterating all the elements in the list for a type guard is pretty bonkers.

#

yeah that part is pretty simple

#

that's just dynamic vs static type

trim tangle
#

beartype does some black magic, I've heard

terse sky
#

it's when I start thinking about covariance/contravariance that things get a bit wonkier

trim tangle
mortal fractal
#

I haven't tried any runtime type checkers

trim tangle
#

Speaking of,

mortal fractal
#

do you plan to cover variance at some point?

trim tangle
#

yep, that's the next article

#

from my (limited) experience it's really hard to explain

buoyant swift
#

the rustnomicon has a good explanation you could try to get inspiration from

trim tangle
#

time for a clone of "concurrency ❀️ with ➿ burgers πŸ” "

#

but for variance

dim flume
trim tangle
buoyant swift
#

"it is well known that"

dim flume
#

I should feel bad for seeing arrow functions

trim tangle
#

we shall leave the proof as an exercise for the reader

trim tangle
#

It literally seems "backwards"

terse sky
#

i mean the simplest way to understand it IMHO is in terms of a function

#

with one argument, and one output

#

suppose you have a function foo. foo would like to accept a function that takes an Animal, and returns a Robot

soft matrix
#

i didnt think contravariance worked in functions?

terse sky
#

it should πŸ€·β€β™‚οΈ

soft matrix
#

oh methods?

terse sky
#

there's no methods here

trim tangle
#

Animal -> Foo is a subtype of Dog -> Foo

terse sky
#

err I think covariant in their parameter

#

contravariant in their return

buoyant swift
#

they're contravariant in the parameters when you want to pass a function to something

trim tangle
oblique urchin
trim tangle
#

i toldya it's hard!!

terse sky
#

yeah, you're right, i just think of it in terms of when is the function usable in the way you want, is the idea

#

but I mixed that up

#

but at any rate it's still the simplest pedagogical tool, because the function does both sides of it

trim tangle
#

yeah I always use a function

#

I'm actually not sure there are other useful examples

buoyant swift
#

it's probably a pretty easy transition. "but what is the type of a function :O?"

terse sky
#

Other useful examples of what?

oblique urchin
#

contravariance, presumably

terse sky
#

there are definitely other useful examples of contravariance

#

output streams and such

trim tangle
#

hmm

#

I guess

terse sky
#

but yeah, it's more rare

trim tangle
#

...well, an output stream of Ts is contravariant because it has a method that's contravariant in T πŸ™‚

terse sky
#

well, that's also true about covariance πŸ™‚ Why is why functions are the natural starting point I suppose

trim tangle
#

yep

#

all of variance can be explained with function variance

#

a type is invariant if it contains a covariant method and a contravariant method

#

Although for covariance it's kind of unintuitive. I think tuple[T, ...] is good for that

#

I think I'll do the variance article tomorrow, it's about 4am here

#

Maybe I'm trying to do the wrong thing here, and a markdown file alone is a bad medium for explaining it.

#

It's like trying to explain music in text

#

I also know some basic set theory, so some explanations make sense to me but not to normal people

#

well, not "normal"

#

idk what's the right word

buoyant swift
#

"people who don't know basic set theory"

#

"uninitiated"?

trim tangle
#

like shapes with holes, or something

#

although I'm afraid that this will turn into "monad tutorials"

#

i.e. using weird analogies when just explaining what's going on would suffice

#

maybe I could pay someone to record a video

#

but that would be like unearthly weird

void panther
#

Is there any way to pass along a signature like with a Callable while requiring something to be a FunctionType?

trim tangle
#

wdym?

#

ah

trim tangle
void panther
#

yeah something like that

oblique urchin
#

I don't think there is. The closest you can get is a Protocol with a __call__ method and other attributes you're interested in

trim tangle
#

yep

#

πŸ˜”

oblique urchin
#

However, not sure how well that would work in practice. I don't think type checkers know that user-defined functions are instances of FunctionType

trim tangle
#

It's kinda sad that returned functions are typed as Callable. This makes working with methods really hard and inconsistent across type checkers

#

because any old Callable behaves differently in a class context than a function, because a function is a descriptor

#

so e.g. mypy has a weird thing where a callable instance variable is treated as a method

#

and I wasn't able to decorate a method in pyright, somehow

void panther
#

ah that's a shame

mortal fractal
#

I think the callable pep rejection does a lot to convince me keeping typing syntax and language syntax in sync is not the way forward

#

I think we may be failing to communicate with the steering council for some reason: for example, there was consensus in typing that the syntax change and possible expansions would have made the situation better and easier to understand/read, but the PEP was rejected because it would have made things harder to read - so there's clearly a disconnect

void panther
#

huh? Isn't the callable syntax used in tools already anyway?

mortal fractal
#

It as implemented as a proof of concept of the (now rejected) PEP, yeah

mortal fractal
oblique urchin
#

That could just be a difference in opinion. Several people on python-dev also brought up that it's hard to read if there are multiple arrows

mortal fractal
#

no doubt

tranquil turtle
#

But i cant remember what is "covariant" and what is "contravriant", i confuse them

blazing nest
#

The way I see it, covariant is the one I expect most of the time and the one I understand

hearty shell
#

I know the difference but cant understand what is being conveyed by those arrows lol

blazing nest
#

If you don't understand it, then it's most likely Haskell

hearty shell
#

Given A :> B then,

Gen[A] :> Gen[B] If Gen covariant on first param.
Gen[B] :> Gen[A] if Gen contravariant.

Where SuperType :> SubType

#

I would still like to understand what the arrows mean though xD

covert dagger
#

those are logic symbols I think, https://en.wikipedia.org/wiki/List_of_logic_symbols

List of logic symbols

In logic, a set of symbols is commonly used to express logical representation. The following table lists many common symbols, together with their name, how they should be read out loud, and the related field of mathematics. Additionally, the subsequent columns contains an informal explanation, a short example, the Unicode location, the name for ...

#

=> is Implies and <=> is equivalent

hearty shell
#

Yeah to me it means implies

tranquil turtle
hearty shell
#

But I dont think it is what it means there

tranquil turtle
#

A=>B means "if A, than B"

hearty shell
#

The meaning of notation changes depending on what you are doing

mortal fractal
#

I think the tree sitter pattern matching syntax parser works now

#

It doesn't support the _ keyword though

#

But whatever

rustic gull
mortal fractal
#

Python actually specifically doesn't use the normalization form recommended by Unicode for programming

#

I have no idea why

#

Maybe it was a mistake

shell wyvern
#

hi
like I used to do with FastApi routes, I want to make a function that is expecting a dict. I want to type hint like in FastAPI with a pydantic model.

Note that I am just using FastAPI as a reference here and this app serves a total different purpose.

What I did:

models.py

from pydantic import BaseModel

class Mymodel(BaseModel):
  name:str
  age:int

main.py

def myfunc(m:Mymodel):
  print(m)
  print(m.name)

myfunc({"name":"abcd","age":3})

It prints m as a normal dict and not Mymodel and m.name just throws an AttributeError.
I don't understand why it is behaving like this because the same code would work in FastAPI. Am I missing something here? What should I do to make this work.

I am expecting a dict arg in the func, I want to type hint with a class inherited from pydantic BaseModel. Then I want to acccess the attributes of that class.

I don't want to do:

def myfunc(m):
  m = Mymodel(**m)

Thank You.

ripe basin
#

@mortal fractal maybe allowing non-ascii letters for identifiers was a mistake πŸ˜‰

mortal fractal
#

I mean, even if you wanted to do that they still did it in a way that seems mistaken

#

they used one of the equivalences that isn't just similar letters, objectively different letters can be equivalent under the system they chose

ripe basin
#

On the plus side, fewer programmers are driven to madness because an identical-looking code point accidentally got copy & pasted πŸ˜‰

trim tangle
#

I recommend taking a look at the articles in the top pin

#

FastAPI inspects the annotations of a function and does something similar to Mymodel(**m) under the hood

shell wyvern
#

oh thank you, I'll look into their source as well as the pinned articles.

trim tangle
#

You can make your own decorator that does this, if that's what you want

shell wyvern
#

sure thanks again

dapper trench
#

hey guys! do you use boto3 or aiobotocore?

blazing nest
#

No. Hope I answered your question! πŸ™‚

trim tangle
#

Question: why would anyone use type annotations? Isn't it just a waste of time?

cunning plover
# trim tangle Question: why would anyone use type annotations? Isn't it just a waste of time?

supposedly it is not
Because
a) It gives your better self-documentation to the code
b) you can use mypy instruments to run automatic tests for correctness of your type hints, to imitate weakly strong typed language, so you are adding additional layer of unit testing layer to your code basically
Which can be an advantage in big python project code bases to fight the python weaknesses of being too freedom language

trim tangle
cunning plover
#

documentation gets old, code usually not

trim tangle
hearty shell
#

playing devils advocate x)

trim tangle
void panther
#

I think the most noticeable benefit right off the bat would be providing information to tooling like auto completion

trim tangle
cunning plover
#

once you write type hints, your IDE knows what your stuff does, and makes correctly shown object types and code colors, and hovering tips in its interface

trim tangle
#

Sure, it speeds up development. But it's not free:

  • you need to invest in tooling: run mypy in your CI pipeline
  • you need to spend extra times on writing types - the time you could be adding more features, or improving the design, or writing more tests
  • you need to code "around" the type checker, sometimes change absolutely valid Python code because mypy doesn't like it
cunning plover
#

less pain to people who use your library

#

we wish to be not seeing the code of the library ;b it should be good to use at docs/interactive IDE tips

hearty shell
#

I have found that often when Mypy complained it was because there was an underlying issue with my design, thus I think that second point is null, sure it makes so you have to code around it, but also prevents you from shooting yourself in the foot

cunning plover
#

Also, potentially type hints could be used as a part of building auto documentation to the code in some auto building tools πŸ€”

cunning plover
#

if our software works stabely and matching the desired 99.99% uptime or whatever we wished, we can risk and to make new features faster

trim tangle
#

Is there any research?

cunning plover
#

if it breaks too often, perhaps it is time to take better care of unit testing / refactoring / type hinting and etc

trim tangle
hearty shell
#

Also, although it doesnt ensure specific contracts, If I just stumbled on a function that I didnt code, it is harder to parse all the contracts that the function adheres to. Knowing the type gives me a general understanding of what the function expects, and what I can expect from it, without having to read into specific contracts. I mean ideally you would have contracts and posconditions on top of the type hints

void panther
#

I usually don't type enough to satisfy a proper checker in the complex cases, just to get proper support from pycharm

#

If used like that I'd definitely say it saved me some time, but there are places where I just said fuck it because I had no idea how to achieve something and didn't think it was worth the time when not exposed as a library

cunning plover
#

we can expect encounter the need for type hints in big enough amount of codes I guess

#

they should be important for scaling the code of the project

trim tangle
trim tangle
cunning plover
#

perhaps it would be a time to just switch to Golang

trim tangle
#

So type hints are a waste of time?

void panther
cunning plover
#

in my opinion? .... perhaps yes.

#

There are better ways to improve the code

#

writing better unit tests

#

integration tests

#

improving CI CD pipelines

#

for the spent time more result in code quality

cunning plover
# trim tangle So type hints _are_ a waste of time?

but if I would be having a goal to make documentation, if i would be leaving the project and needing to transfer to other people, I would probably think to use every advantage for documentation, including type hints

oblique urchin
#

For me the biggest benefit happens when I look at a library and I want to see how the function needs to be called, or exactly what it returns. Without type hints you have to rely on (often vague or nonexistent) documentation.

trim tangle
#

oh, I see now

#

So it seems like they found 3 actual bugs?

  • comparison of strings and bytes
  • PoolManager.connection_from_X() APIs which, as noted, very few people use
  • SSLTransport interface
trim tangle
#

I like type hints and I watch the whole typing world very closely. But it's purely a subjective feeling, so I'm wondering if there's some actual data on whether type hints are a waste of time, or whether they solve some problems at a reasonable cost.

#

I know that there are stories by companies and library writers who got some benefit by adding type annotations. But I think it's subject to the "survivorship bias": nobody will make a story about typing 100k lines of code and losing a million dollars as a result.

void panther
#

maybe more worth it if you do need the safety it provides, but then you can also use a static language and be sure that everything is correct while usually being simpler to use from the typing perspective

trim tangle
#

Yeah, that's also a good point. If you need the benefits of static typing, why Python?

rustic gull
# 
tests = [("ABAB", 2, 4)]
for *input, ans in tests:
    a, b = input
    # print(Solution().characterReplacement(*input), ans)
    print(Solution().characterReplacement(a, b), ans)
# junk.py:24: error: Argument 1 to "characterReplacement" of "Solution" has
# incompatible type "object"; expected "str"
#         print(Solution().characterReplacement(a, b), ans)
#                                               ^
# junk.py:24: error: Argument 2 to "characterReplacement" of "Solution" has
# incompatible type "object"; expected "int"
#         print(Solution().characterReplacement(a, b), ans)
#                                                  ^

mypy doesn't understand the unloading operator?

trim tangle
#

@rustic gull In this case, input will be a list. A list doesn't have information about elements at different positions, only an element type. So input gets inferred as list[Unknown], or list[str | int] depending on what type checker and settings you're using.

#

Why not do ```py
for a, b, ans in tests:
...

rustic gull
#

I see thank you

mortal fractal
#

I haven't yet added types to a codebase where it didn't uncover at least one πŸ›

trim tangle
#

My question is more about whether it's profitable for a commercial/enterprise application

#

Of course finding bugs is good. But there's more than one way to find bugs (like auditing, code review, adding documentation (surprisingly), testing (including more advanced things like mutation testing property testing))

oblique urchin
#

if you have a million lines of code, you don't know what all the functions do

trim tangle
#

I guess

#

I guess I'm just not finding these theoretical/subjective concerns good enough

oblique urchin
#

typing-extensions 4.1.0 was released. Enjoy using TypeVarTuple, Unpack, LiteralString, never, assert_never, reveal_type, dataclass_transform, and is_typeddict

soft matrix
#

@oblique urchin do you know when dataclass_transform is gonna be submitted to the SC?

oblique urchin
hearty shell
#

Is reveal_type just a runtime version of mypys reveal_type?

soft matrix
#

no it's much less complex than mypy's version

hearty shell
#

also wasnt is_typeddict already part of python?

#

I need to read the relevant peps x)

soft matrix
#

it's not got a pep for it, I'm pretty sure it's just got a BPO

#

it's only in python3.11? iirc

hearty shell
#

Isnt it this?

soft matrix
#

yeah it's a backport of that

hearty shell
#

Ah, got it

hearty shell
oblique urchin
#

cpython dcs

hearty shell
#

Oh right, just put the version to 3.11 dev

oblique urchin
hearty shell
#

Yeah I got it my bad

#

Thanks πŸ˜…

soft matrix
terse sky
#

As for the "why python" of it, I think in most cases languages aren't selected for those kind of reasons, practically. It's more the usual boring but important things; which libraries are available, supported languages for APIs you care about, ease of hiring, existing code, etc.
I do agree if you like static typing, and none of those things are a factor, python isn't likely to be your first stop.

#

For me though and probably many others, using python for various things made sense because python was already a lock, for its quantitative stack.
On the flip side I really like having types for the non-quant code in our codebase; a lot of the code is stuff that is very hard to meaningfully test because a lot of the work can be ssh'ing into boxes, running random commands, pulling git repos, etc. Types catch a lot of silly mistakes cheaply and it's not so easy to write unit tests that will catch those same mistakes cheaply without more time spent mocking than is really worth it.

languid aspen
#

So at the moment, I have some code which takes advantage of a function's __annotations__ to cast the provided arguments to what they are hinted as, but this will break once 3.11 arrives and all type hints are now strings - is there some other solution I can use? I'm wondering if something like pydantic might have a tool that solves this, or if I'll have to parse it myself

void panther
#

The feature is not yet set to arrive in 3.11 afaik

pastel egret
#

It was deferred again to 3.12.

void panther
pastel egret
#

What you can use is either typing.get_type_hints(), or inspect.get_annotations().

#

get_type_hints() detects the stringified versions, and execs them to get the real object. It also by default unwraps Annotated, so you can just get the types and not annotations.

little hare
#

-gs typing.get_type_hints

#

brb

rough sluiceBOT
#

Lib/typing.py lines 1796 to 1801

# This is surprising, but required.  Before Python 3.10,
# get_type_hints only evaluated the globalns of
# a class.  To maintain backwards compatibility, we reverse
# the globalns and localns order so that eval() looks into
# *base_globals* first rather than *base_locals*.
# This only affects ForwardRefs.```
languid aspen
pastel egret
#

inspect.get_annotations() is more generic, but handles some specific differences between how annotations work in classes, modules and functions.

languid aspen
#

And if I import annotations from __future__?

pastel egret
#

That still works.

little hare
#

if you do that make sure to not stringify your annotations

languid aspen
#

Since I'm planning on doing that

little hare
languid aspen
#

Alright

pastel egret
languid aspen
#

Ah ok, thanks :D

languid aspen
pastel egret
#

Basically explains why to use these functions and what to do in 3.9 or less.

#

Yes it does.

languid aspen
#

Right

little hare
#

yes

pastel egret
#

So if you manually wrap in quotes, it'll be quoted twice which is a bit confusing.

languid aspen
#

yeah

#

Hm, so inspect.get_annotations wont return stringified annotations?

#

ah, but inspect_get_annotations in only in 3.10+

#

I'd like my code to work on 3.7+

#

ah, this might have a solution for that

#

oh

#

this is nice

#

I don't need to fix this, I'm using pydantic and that already does this for me

hearty shell
#

I thought it was still just in talks

mortal fractal
hearty shell
#

Ah alright

spiral blaze
#

Does anyone know how to type hint pandas column so that they suggest you columns in a DataFrame?
E.g.

import pandas as pd
df = pd.DataFrame({"fruit":["apple","banana"]})

def print_fruit_df(df: pd.DataFrame):
    print(df.fruit)

When I am just looking at the functions that accept a DataFrame, I never know what columns names the accepting DataFrame will have and it is really annoying
Instead, I want to do something like this:

class FruitDataFrame():
    def __init__(self, df) -> None:
        self.df: pd.DataFrame
        self.fruits: pd.Series[str]

def print_fruit_df(df: FruitDataFrame):
    print(df.fruit)

So that the text editor can figure out what coulumns the data frame will have
but the above is inconvenient because I cant set a column data to the DataFrame through the attribute without writing a custom setter for each columns: I.E.

df.fruits = ['new fruit']

Would not update the coulmns in the dataframe and I would have to write a custom setter function for each columns and that is really annoying
Pls help

#

I found out about typedDict, and I would want to do something like

# 
from typing import typedDict
import pandas as pd

class FruitDataFrame(pd.DataFrame, typedDict):
  fruit: pd.Series[str]

df: FruitDataFrame = pd.DataFrame({'fruit': ['apple', 'banana'] })
#

Something like that

trim tangle
#

@spiral blaze I think that's not possible

#

(I'm not familiar with pandas for the record)

#

I think it's possible to make a mypy plugins for this, maybe there already is one. But it will not help with IDE autocompletion

spiral blaze
#

can you tell me a bit about making your own type hint though so that I can craft something that is atleast useful?

#

Like in theory, I could just create a custom class, or use a typedDict as the type hint for pandas correct? @trim tangle

little hare
#

But it will not help with IDE autocompletion
somehow this can be done

#

probably

#

okay so ours works with overloads and __getitem__

trim tangle
spiral blaze
#

like I dont understand how vs code doesnt even have a proper implementation of typedDict

trim tangle
#

It does

little hare
#

this is what we have:

Code_-_Insiders_6Bvwg4sMcW.png
#

custom Range typehint 😳

spiral blaze
#

it doesnt suggest you anything though

little hare
#

it does

spiral blaze
#

it just points out when you are making a mistake

trim tangle
trim tangle
spiral blaze
#

ye

#

basically I want to do something like

trim tangle
#

do you have "Pylance -> type checking" set to "basic" in the settings?

spiral blaze
#

ye but not when:

foo.bar
trim tangle
#

Right, because a TypedDict is a type for a dict

spiral blaze
#

but cant you access value of dict like that?

trim tangle
#

no

#

!e

x = {"foo": 1}
print(x.foo)
rough sluiceBOT
#

@trim tangle :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 2, in <module>
003 | AttributeError: 'dict' object has no attribute 'foo'
trim tangle
#

this is not JavaScript πŸ™‚

spiral blaze
#

lol

#

soz

little hare
vague tulip
#

where do i ask about help with time.time() ?

little hare
#

bs4 comes to mind

spiral blaze
#

ye so I wish I could just use TypedDict but not give me warning when I assign a pandas Dataframe to it

trim tangle
vague tulip
#

ok

trim tangle
#

!pypi attrdict

rough sluiceBOT
spiral blaze
#

I.e.```
from typing import TypedDict
import pandas as pd

class MyType(pd.DataFrame, TypedDict):
x: int
y: int

a: MyType = pd.DataFrame({'x':[], 'y':[]})
a['x']

#

but linter complains when I assign a pandas Dataframe

trim tangle
#

right, because it's not an instance of MyType

#

You can put a # type: ignore at the end if you want to ignore the error

spiral blaze
#

ye but like, arent you supposed to be able to create new types?

trim tangle
spiral blaze
#

? The only reason why I want to inherit is so that it will still hint me the Pandas Method as well as their columns

vague tulip
#

how do i make time.time() show only hour and minute and second in the local time

trim tangle
little hare
vague tulip
#

my english isnt very good, i didnt understand what was in the channel

spiral blaze
#

ye, but its awqward to do that every time, I want to introduce a Data Oriented Programming Paradigm with Pandas Dataframe

#

so basically, I will be passing Pandas Dataframe to lots of functions

#

I wish I could code up some custom Typehing so that I can do what I want

trim tangle
#

@vague tulip You need to click at one of the channels in the " βœ… AVAILABLE HELP CHANNELS " category and just type your question in it

unknown.png
spiral blaze
#

but I guess its not possible

vague tulip
#

ohhhh ok

spiral blaze
vague tulip
spiral blaze
#

but I dont like these external packages, coz you have to install them for runtime as well

trim tangle
spiral blaze
#

type hints are good

#

but I need way more freedom to express my own types

#

its too limited for somereason, and clunky

trim tangle
spiral blaze
#

type hint in python focus too much on primitive types

#

and in practice you dont really need type hints for primitive types any way coz they are obvious

#

you need to start hinting when its more complex

#

And like the creator of data-science-types say, the aim should not be to be correct but be useful in the development process

trim tangle
#

If you have something to suggest, you can post a message on the typing-sig mailing list πŸ™‚
and then maybe suggest a PEP

spiral blaze
#

no, coz they would assault me and call me stupid

#

I never like being too close to development process of language

#

still have my scar from Golang discussions

trim tangle
#

python mailing lists do get heated sometimes, but it's very rare

#

especially in typing-sig

spiral blaze
#

Ye, maybe I will start reading, so that I can actually suggest when I'm ready

#

do you think it would be ok to ask similar question but just rephrased there? And ask for feature/ways I could achieve that with native python?

trim tangle
#

sure

#

If you want an example of a successful gradual typing system, take a look at TypeScript btw πŸ™‚

spiral blaze
#

but maybe they will get angry at me for not reading the manual enough

#

what do you mean by gradual typing system though?

trim tangle
#

'gradual typing' refers to partially adding static types to an otherwise dynamic language

spiral blaze
#

ye ok

#

True

#

maybe I could look at how TS solves it if it does solve it

trim tangle
spiral blaze
#

meh sure maybe I will have a go at asking

#

when I have the time

#

I will probably ask you to proof read before posting

trim tangle
trim tangle
spiral blaze
#

youve ever posted on reading list?

trim tangle
#

no

spiral blaze
#

*mailing list

#

ye well whatever maybe we can post together on this issue

trim tangle
#

!pep 681

rough sluiceBOT
#
**PEP 681 - Data Class Transforms**

Link

Status

Draft

Python-Version

3.11

Created

02-Dec-2021

Type

Standards Track

spiral blaze
#

coz seem to know a lot more about typing and TypeScript

trim tangle
#

although it's still a draft

spiral blaze
#

I will look into it

#

oh, so its still in development

#

ye this looks relevant thanks i learnt a lot

covert dagger
#

the kind of completion you wanted for DataFrames is possible in vscode in the jupyter console

trim tangle
#

but that's just runtime autocompletion, right?

covert dagger
#

but there it doesn't use typing information, it just directly reads the data

#

yeah the dataframe has to exist in memory in the jupyter kernel or whatever it is called

#

after that when you type in the jupyter interactive thing, it gives auto-completions for column names

spiral blaze
# 
from typing import Generic, TypeVar, List
import pandas as pd

T = TypeVar('T')
class MyColumn(Generic[T]):
    def __init__(self, column_name: str) -> None:
        super().__init__()
        self.column_name = column_name

    def __get__(self, instance, owner):
        return instance.df[self.column_name]

    def __set__(self, instance, value: T):
        instance.df[self.column_name]=value

class MyDF():
    def __init__(self) -> None:
        self.df = pd.DataFrame()

class CustomDF(MyDF):
    a:MyColumn[List[int]] = MyColumn('a')
    b:MyColumn[List[float]] = MyColumn('b')

df = CustomDF()
df.a = [1,2,3]
df.b = [2.2, 2.3, 3.3]
print(df.a)
print(df.b)
#

Ok I was able to get this far with trying to type columns, I'm pretty impressed with my own creativity

#

but I still have one problem

a:MyColumn[List[int]] = MyColumn('a')
trim tangle
#

nice solution, I haven't thought of this

spiral blaze
#

I have to specify a twice

#

is there like a way to get MyColumn to figure out that it is called a?

trim tangle
spiral blaze
#

ok I will look into it thanks

# 
from typing import Generic, TypeVar, List
import pandas as pd

T = TypeVar('T')
class MyColumn(Generic[T]):
    def __set_name__(self, owner, name):
        self.column_name = name

    def __get__(self, instance, owner):
        return instance.df[self.column_name]

    def __set__(self, instance, value: T):
        instance.df[self.column_name]=value

class MyDF():
    def __init__(self) -> None:
        self.df = pd.DataFrame()

class CustomDF(MyDF):
    a:MyColumn[List[int]] = MyColumn()
    b:MyColumn[List[float]] = MyColumn()

df = CustomDF()
df.a = [1,2,3]
df.b = [2.2, 2.3, 3.3]
print(df.a)
print(df.b)
#

Oh my GOD its beautiful

# 
class CustomDF(MyDF):
    a:MyColumn[List[int]] = MyColumn()
    b:MyColumn[List[float]] = MyColumn()
#

look at that so beautiful

#

if I wanted to use dataframe methods I could do:

df.df.drop('a')
#

but is there a cleaner way of doing it, so kind of like relaying other method calls to self.df?

trim tangle
#

I think adding an attribute access layer is fine

spiral blaze
#

ok thanks

#

I was looking for ways of maybe inheriting directly from pandas dataframe?

#

class CustomDF(MyDF, pd.DataFrame):

#

but I dont think Im good enough for that but I think where we were able to take this is good enough to be very useful thanks

trim tangle
#

Not sure about that. Haven't worked with pandas

hearty shell
trim tangle
hearty shell
#

πŸ˜…

oblique urchin
#

mypy has 2038 open, 5194 closed

hearty shell
#

I think it is about the same then if you account for the fact that typescript has a bigger team behind it

oblique urchin
#

I bet a lot of the mypy bugs can be closed as duplicates, I might do a round of triage at some point

trim tangle
upbeat wadi
# 
from __future__ import annotations

from typing import Any, TypeVar, Generic

T = TypeVar('T')

def flatten(original_cls: type[int] | type[str]) -> '(type[Response[Any]]) -> type[Response[Any]]':
    ...

@flatten(int)
class Response(Generic[T]):
    ...

class FooResponse(Response[T]):
    ...

Pylance has periodically been giving me errors (x is already specialized) for my code, which was similar to what's above. I realized the errors are because the flatten decorator is returning a specialized version of the type it's decorating. Pyright doesn't flag this as an error and pylance doesn't always (for some reason) either, should they be?

hearty shell
#

What is the use case?

#

Or error

upbeat wadi
#

the errors were either "type x is already specialized" or "expected no arguments for type x" (this is an example of one error)

unknown.png
#

I fixed the function signature and errors, I was just wondering whether this should be reported by pyright

#

although like i mentioned pylance doesn't report it consistently

hearty shell
#

Yeah, I don't know what the error even is x) My only guess as to what that error means doesnt fit the context of the code you provided so I dont think this is it

trim tangle
acoustic thicket
trim tangle
#

thank you

#

I should probably change to Child and Parent because I got confused as well πŸ™‚

acoustic thicket
#

yeah, agreed

#

not really incorrect, but why name the instance of Box pair instead of box?

unknown.png
trim tangle
#

that is indeed a bug

acoustic thicket
#

i feel like the function name and parameter names makes this harder to read

unknown.png
#

since you only need to look at the types

#

make_animal :: str -> Animal πŸ₯΄

#

So our Box class should stay invariant.

unknown.png
trim tangle
acoustic thicket
#

how would you feel about using the (rejected πŸ˜”) callable syntax in your tutorial

trim tangle
#

I would like the idea to be as unambiguous as possible

mortal fractal
blazing nest
# 
try:
    from some_dependency import BaseClass  # type: ignore
except ImportError:
    class BaseClass:
        def __init__(self, *args, **kwargs) -> None:
            raise ImportError("Missing dependency to use this O_o")


class Somesubclass(BaseClass):
    ...

Pyright keeps complaining about this (hold on let me upload it to a playground)

#

I get "Argument to class must be a base class"

#

Any tips for this?

spiral blaze
#

does anyone know why apple:str = 2 is not shown as a error on vs code?

oblique urchin
oblique urchin
blazing nest
spiral blaze
#

and Ive checked the setting as well

oblique urchin
spiral blaze
#

ye pylance dont seem to do it

#

I istalled pyright and it works now

#

thx

hallow flint
mortal fractal
#

huh, pyright doesn't like my attempt to play with its recursion

# 
GraphItem = Union[tuple[str, list["GraphItem"]], int]
def listint() -> list[int]:
    ...
a: GraphItem = ("tests",  listint())  # very unhappy
hearty shell
#

Meanwhile mypy doesnt even support recursion x)

mortal fractal
#

I guess I can just make it list[graphitem] instead of int

oblique urchin
mortal fractal
#

maybe πŸ€” I didn't put too much thought into it

# 
b = [3, 4, 6]
a: GraphItem = ("test", b)

this ^^^ is not okay

# 
a: GraphItem = ("test", [3, 4, 6])

but pyright says this is fine

oblique urchin
#

in the first example it infers b as being of type list[int]

#

which isn't compatible with GraphItem

mortal fractal
#

and the second GraphItem presumably, yeah

#

this interaction with mypy is annoying though

#

like,

# 
    master_list: GraphItem = (form_filt, [])
    assert isinstance(master_list, tuple)

I would have to assert like this every time in mypy (because it takes the annotation as religion), but pyright wouldn't need the assert

#

oh well

mortal fractal
#

I wish argparse wasn't so dynamic so it could be statically type checked

maiden dagger
#

Hello, I'm working on a project to predict congestion at the airport.
We are trying to build a pipeline that connects data and machine learning.
But I don't know how or what tools to use to make it.

summer berry
trim tangle
summer berry
#

Perhaps. But I think it's easy enough to understand for use. Maybe not to understand the theory, but that isn't necessary.

#

In any case, it's worth mentioning None has problems and leave it up to the reader to do more research and make a choice or just ignore it

severe osprey
#

How to recognize element type in list?
i.e data = [42, "42", 42.0]
for element in data:
if element is type (int):