type-hinting

[blazing cobalt]

I'd think it should be: the code doesn't exhaustively check all the cases.

[eager vessel]

I think I misunderstood you then

[blazing cobalt]

Yeah, look at the screenshot from the link i posted above -

Pyright can actually verify that you've covered the range of inputs in certain cases.

[eager vessel]

Documentation states that you need to use assert_never for that

[blazing cobalt]

Right - that's the hack.

[eager vessel]

i.e.

def func(a: int | str | datetime) -> None:
    match a:
        case int():
            return
        case str():
            return
        case datetime():
            return
    assert_never(a)

Does pyright complain about all match statements then? 🤔

[blazing cobalt]

I like Pyright here because the whole point of exhaustion checking is to help me find cases I forgot to include. So, if that feature requires me to not forget something else ... 🙂 it kind of defeats the purpose

(Or also, cases that became possible over time after the match was written.)

[eager vessel]

To be honest I personally rarely make functions that would need assert_never or exhaustiveness checking

[blazing cobalt]

Now that it's possible, i do it all the time — checking a None case, etc.

[eager vessel]

I mostly use match to check for result values/errors:

@strawberry.mutation
@inject
async def create(
    self,
    input_: Annotated[BoardCreateInputGQL, strawberry.argument(name="input")],
    command: Annotated[BoardCreateCommand, Inject],
) -> BoardCreatePayloadGQL:
    validated = validate_from_callable(input_.to_dto)
    if isinstance(validated, Err):
        return BoardCreatePayloadGQL(
            board=None,
            error=validated.err_value,
        )

    board = await command(dto=validated.ok_value, project_id=input_.project_id)
    if isinstance(board, Err):
        match board.err_value:
            case ProjectNotFoundError():  # pragma: no branch
                return BoardCreatePayloadGQL(
                    board=None,
                    error=EntityNotFoundErrorGQL.from_error(board.err_value)
                )

    return BoardCreatePayloadGQL(
        board=BoardGQL.from_orm(board.ok_value),
        error=None,
    )

I just didn't encounter such cases that you did

[eager vessel]

But yeah, I guess I'd need to be extra careful with mypy if I make a function like this

[blazing cobalt]

An extra level of free safety. E.g.:

def first_match(node: SelectorLike, css: str, expected: str) -> str:
    match node.css(css).get():
        case str(result):
            return normalize_nonempty(result)
        case _:
            raise ParseException(f'Could not parse the {expected} using "{css}"')

Also, the functional programming advantage of fewer or no mutable instance variables.

[eager vessel]

You could also return result instead of exception, if that's an expected error and if it should be handled

e.g.
https://github.com/psf/black/blob/main/src/black/rusty.py

https://github.com/rustedpy/result

[blazing cobalt]

I like Rust Result a lot — I think that this Pyright feature can actually help enforce it — i.e., make it practical to use

I.e., The Result type only makes sense when your language does exhaustiveness checking, like Rust does

[eager vessel]

In this case since i expect my result to be Ok after the match statement it would fail if I add an extra error type here

[eager vessel]

It depends on how you use it

[blazing cobalt]

I'll just disagree 🙂

[eager vessel]

def func(result: Result[int, str | float]) -> None:
    if isinstance(result, Err):
        match result.err_value:
            case str():
                return
            case float():
                return
    reveal_type(result) # Ok[int]
    print(result.ok_value) 

def func(result: Result[int, str | float]) -> None:
    if isinstance(result, Err):
        match result.err_value:
            case str():
                return
    reveal_type(result) # Ok[int] | Err[str | float]
    print(result.ok_value) # Item "Err[str | float]" of "Ok[int] | Err[str | float]" has no attribute "ok_value"  [union-attr]

Safety is achieved using ok_value and err_value properties that are only present on Ok and Err types respectively, If you're going to use Ok value later it works perfectly fine

[blazing cobalt]

The Union type makes it infinitely better than e.g. Go's style

[eager vessel]

Even just using union, e.g. int | Err1 | Err2 would work better than returning a tuple with result, err

[blazing cobalt]

IMO, it needs to be as easy as Rust — just jump right in to the match without the isinstance.

Yep, I had a Go programmer co-worker telling me how Great Go's "error handling" is, and the same as Rust's, but had never seen a Union type.

[eager vessel]

Yeah, it could be better

But honestly, it's still better than using exceptions

Now I'm terrorizing my coworkers with Result

[blazing cobalt]

👍

A nice thing about Result is that it shows up in the function signature. Exceptions don't.

[rare scarab]

Go doesn't have auto propagating errors, and I find that annoying

[hallow flint]

yes, pyright has a strict check that will complain about all match statements that let things potentially fall through. i would accept a PR to mypy that added an opt-in check that does the same thing. i put some pointers in here: https://github.com/python/mypy/issues/13597

[blazing cobalt]

Yep. IMO Go simply does not have error handling.

[rare scarab]

Time to panic()

[blazing cobalt]

LOL, yes. I inherited a Go app while at Coinbase from their "principle engineer", littered with calls to panic().

[trim tangle]

I like Rust's error model but I wouldn't force into Python

It's a different language

All the libraries (and the standard library) use exceptions

[eager vessel]

Exceptions are essentially gotos, I wouldn't use them for errors that are expected to be handled gracefully

[trim tangle]

Then you might be using the wrong language 🙂

Exceptions are bad, dynamic typing is bad... Haskell, F# and Rust might be better choices for your taste

You can replicate idioms from other "cultures" in Python. But it's not going to be idiomatic code, and your colleagues might have trouble understanding it

[eager vessel]

They should be used where they make sense IMO, Framework expects you to use one? Go ahead

By that logic since python is dynamic static typing shouldn't be used? Is it really idiomatic?

[trim tangle]

I would say static typing is more or less idiomatic nowadays

I'm not saying there's some technical problem with using this approach. It's a social problem

Are you going to convince the main developer on my team that we should switch to error values? Probably not

[eager vessel]

Adding anything new into your tech stack or development process would present a social problem

If all developers agree to use x for some reason then they just use x

[trim tangle]

Yes

[eager vessel]

black uses it afaik? I didn't really dig into the code

Just saw that there's a small implementation of result type

[trim tangle]

It's not just adding a new technology, it's changing a very fundamental aspect of how you program in a language. It's changing something that's "always been this way"

Static typing was a long uphill battle, and not everyone accepts it.

And the benefits are much less clear than with types IMO

[eager vessel]

We mainly use it in service classes, so all errors are well documented this way:

def craete_user(self, ...) -> Result[User, DuplicateUserError | PermissionDeniedError]:
    ...

Not anywhere else really

If you want you can always .unwrap() 😅

[trim tangle]

I would probably use this style if I was working alone but had to use Python for some reason

Honestly, just documenting what exceptions your function raises is a big step forward

But, of course, documentation is not statically checkable

[tranquil turtle]

You can feed it in ChatGPT

[carmine phoenix]

also Scala can return an Enum that contains the output or the error, and you can easily see all the possible exception that can be raised recursively

[rare scarab]

Do you use for loops? They implicitly except StopIteration

[empty torrent]

anyone know why this snippet doesn't show a error for not being exhaustive in pycharm?

[rare scarab]

because match doesn't need to be exhaustive?

[empty torrent]

I actually meant a type error from a type checker like pylance

(but here specifically from the one from pycharm)

because PEP622 says it should If I understoond correctly

[wicked scarab]

class Foo:
    def __init__(self, s: str): ...
    def __eq__(self, other): ...

def bar(arg: Foo | list[Foo]):
    if arg == Foo("+"):
        reveal_type(arg)

pyright says information: Type of "arg" is "Foo | list[Foo]". Why it doesn't narrow?

[leaden oak]

a custom object could in theory be equal to anything else, actually I don't think it's really possible for equal checks to narrow custom objects

[wicked scarab]

oh yeah that's true, didn't think about that, thanks

[tranquil turtle]

def __eq__(self, other) -> TypeGuard[Foo]: ...
``` ![lemon_exploding_head](https://cdn.discordapp.com/emojis/754441880141561958.webp?size=128 "lemon_exploding_head")

[leaden oak]

I guess yeah, I wonder if any type checkers have support for that

[tranquil turtle]

this is not good, actually

x: Foo | list = Foo('a')
if x == Foo('b'):
  ...
else:
  # typechecker assumes that if TypeGuard check fails, it is not a Foo:
  reveal_type(x) # list
  print(x) # Foo('a')

[echo knot]

Is there a way to create a TypedDict with keys having whitespaces? (Other than using conversors like editing the keys or pydantic)

[soft matrix]

TypedDict has a functional way to create keys and values, use that

[oblique urchin]

yes, using the functional syntax

[soft matrix]

!d typing.TypedDict

[rough sluice]

typing.TypedDict

class typing.TypedDict(dict)```
Special construct to add type hints to a dictionary. At runtime it is a plain [`dict`](https://docs.python.org/3/library/stdtypes.html#dict "dict").

`TypedDict` declares a dictionary type that expects all of its instances to have a certain set of keys, where each key is associated with a value of a consistent type. This expectation is not checked at runtime but is only enforced by type checkers. Usage...

[oblique urchin]

TD = TypedDict("TD", {"weird key": int})

[echo knot]

You saved me. Thanks! 🙂

[open hawk]

PyCharm keeps giving me a type mismatch when I try to index into a dict[type[CorrectionComponent], V], when the keys are all the types of the subtypes of CorrectionComponent. Why doesn't this work?
I mean, it does work, but I keep getting a type warning anyways

Background: I've got a bunch of classes, and I have a binary file with the data needed to make instances of those classes. I don't want the classes to have to know about their binary encoding, so I have the binary stuff in a separate module.

So, while I would like to just have CorrectionComponent be an ABC with a static method called from_binary(), which each instance implementing their own version of that method, I don't want that to be in the class definition. The "simplest" way I thought of to do this is to just have a dict which maps the Type to the pseudo-constructor. This works fine, with the exception of the type hints. Whatever I do, PyCharm warns me that

Expected type 'Type[OrbitBlock | ClockBlock | CodeBiasBlock | PhaseBiasBlock | IonoBlock | GeometryBlock | PCVBlock | SatelliteYawBlock]' (matched generic type '_KT'), got 'Type[CorrectionComponent]' instead

I think the problem is how I type the dict[type[CorrectionComponent], Callable], pycharm still says it's the dict[Type[blah | ...], whatever

Oh, nevermind, I actually had fixed it, all except an incorrect attempt to type hint the dict using TypedDict.

For the record, the solution was to use

component_parsers: dict[type[CorrectionComponent], Callable] = {...}

[rare scarab]

This also works when you have a reserved name as a key, like TypedDict("Reserved", {"class": str})

I wish we could wrap identifiers in `` to ignore naming requirements and keywords

class Foo(TypedDict):
  `class`: str

list of ascii symbols python doesn't use: ` $ ?

[trim tangle]

Python 2 used `

😛

[oblique urchin]

yes, for repr()

that was very rarely used though

[vivid ore]

Does mypy support variadic generics at the moment?

[lunar dune]

It has experimental, partial support for PEP 646

I think you need to use a flag to enable it, can't remember what the flag actually is off the top of my head though

[vivid ore]

Can you enable the flag via mypy.ini?

[lunar dune]

I would assume so, most mypy flags are configurable in mypy.ini files 🙂

[vivid ore]

It says

"TypeVarTuple" support is experimental, use --enable-incomplete-feature=TypeVarTuple to enable [misc]
I tried writing
enable-incomplete-feature=TypeVarTuple
but it doesn’t work :(

oh :(

$ mypy --enable-incomplete-feature=TypeVarTuple .
mypy.ini: [mypy]: Unrecognized option: enable-incomplete-feature = [TypeVarTuple]

oh

I am using an outdated version

ok it crashes :(

[rare scarab]

Support is flimsy. Watch your step

[heavy bloom]

Hi guys, I have this base class that acts as a marker class that doesnt add anything to the derived class. My lsp is popping up an error for a function that a derived class has a narrower scope than the base class, which does make sense in the general case. However in this case I can reason that the base class is actually just empty. Is there a way to solve this, by way of config or just implementing it differently?

class BaseFoo:
  """
  This is actually all there is in the library that im trying to develop
  """
  pass

class Bar(BaseFoo):
  def some_interface(self, arg) -> str:
    return ''


class BaseProcessor(abc.ABC):
  @abc.staticmethod
  def process(foo: Iterable[BaseFoo]) -> bool:
    raise NotImplementedError
  


class BarProcessor(BaseProcessor):
  """
  This class does some processing and for the mental model it helps to think of all objects that are   passed to methods as BaseFoos
  """
  def process(foos: Iterable[Bar]) -> bool: # <- this will be the offending line
    # do some actual processing
    return True

The process method on the BarProcessor will pop up an error saying the types are interchangable. These processor classes are supposed to be implemented by others for their specific case so logic for every item is isolated. My current solution is adding a # type: ignore to this line so it passes ci but I think this is too brittle. It relies on implementors to remember this, or review to catch this.

Ping me if you need clarification btw, I might not be paying attention.

[echo knot]

I am on the phone right now and I could not check anything. Could it be because of the contravariance of input arguments in a method?

Like, Iterable is generic. This one as far as I know behaves as a covariant on its types. However, input arguments in a method behave as contravariant, so when you pass an iterable of Bar (being Bar a subclass of BaseFoo) is not allowed

I might be so wrong about this tho

[heavy bloom]

My background is not comp sci but math, so forgive me if this might be confusing but i think so. It essentially complains about the scope being too narrow for the derived type.

[echo knot]

I will check it out like in an hour once I have a laptop, because I could be wrong too 🙂

[heavy bloom]

Thanks

So i know subtypes might behave differently than their base but in this case the base is just a marker without behaviour or attrs

[trim tangle]

The reason your type checker is complaining is that the BarProcessor violates the Liskov Substitution Principle, i.e. you can't substitute a BarProcessor where a BaseProcess is expected. For example:

class Baz(BaseFoo):
    ...

def process_bazzes(p: BaseProcessor) -> None:
    print(p.process([Baz(), Baz(), Baz()]))

this function is perfectly fine, and process_bazzes(BarProcessor()) is fine with the type checker, but it won't work (because BarProcessor expects only Bars)

One possible solution is making BaseProcessor generic:

B = TypeVar("B", bound=BaseFoo, contravariant=True)

class FooProcessor(ABC, Generic[B]):
    @abstractmethod
    def process(self, foos: Iterable[B]) -> bool:
        raise NotImplementedError


class BarProcessor(FooProcessor[Bar]):
    def process(self, foos: Iterable[Bar]) -> bool:
        ...

[heavy bloom]

Is this just a language thing then, since the Base should always be the same or a smaller scope than the derived in my example right?

Ill try this btw, thanks

[trim tangle]

No, that's just how subclassing/subtyping works. A subclass should fit wherever a base class is expected

So e.g. you should be able to pass any BaseProcessor to this function: ```py
def process_bazzes(p: BaseProcessor) -> None:
print(p.process([Baz(), Baz(), Baz()]))

[heavy bloom]

Can i somehow enforce that it only takes a derived class?

[trim tangle]

I don't think so

I think the generic is the way to go if you have different Foos

[heavy bloom]

Can you explain contravariant to me?

I dont think i understand that conecpt correcly

Oh is it just the opposite of covariance?

[trim tangle]

kinda

[heavy bloom]

Im thinking of this in terms of vectors

[trim tangle]

I have it as part of a tutorial on generics:
https://decorator-factory.github.io/typing-tips/tutorials/generics/variance/

[heavy bloom]

Okay let me read this

Ah okay I think I understand, these types are to make sure all operations inside the method are valid. So the scope has to be the exact same.

Empahsis on exact

[trim tangle]

If you had this

  @abstractmethod
  def process(foo: Iterable[Bar]) -> bool:
    raise NotImplementedError

then you would be able to let BarProcessor.process accept an Iterable[BaseFoo]

Or even Iterable[object] (i.e. it can process anything)

[trim tangle]

(Some people say this is always the case, but in practice of course it can be different. Here's a talk by rhettinger where he argues Python subclassing is a different kind of tool)
https://www.youtube.com/watch?v=EiOglTERPEo

[echo knot]

Dont worry. It also took me some time to understand about this thing. It is not intuitive at first 😅

[trim tangle]

For some reason variance is really hard to explain/teach

[trim tangle]

this article does a pretty shitty job all things considered

but maybe it's just skill issue

[dull lance]

While not in Python, I liked this one in particular: https://www.youtube.com/watch?v=EInunOVRsUU

[trim tangle]

maybe I need some fancy pictures

[fierce ridge]

is there a way to define something like a Protocol but that doesn't act like a parent class? that is, i want to enforce certain constraints on certain classes statically (not at runtime), but i don't want to treat them as children of some particular parent class

ABCMeta is almost what i want

from abc import ABCMeta


class AlgorithmClass(ABCMeta):
    name: str
    # How to require an 'apply' method?


# OK
class Algo1(metaclass=AlgorithmClass):
    name = "algo1"

    def apply(self, thing: Thing, param1: int, param2: float) -> Result:
        ...


# ERROR: No 'name' attribute
class Algo2(metaclass=AlgorithmClass):
    def apply(self, thing: Thing, param1: int, param2: float) -> Result:
        ...


# ERROR: No 'apply' method
class Algo3(metaclass=AlgorithmClass):
    name = "algo3"

i can use @abstractmethod but then i can't do any additional checks on the type signature i think?

actually wait let me try, maybe that just works

[soft matrix]

why do you not want protocol as a parent class?

iirc its just erased at runtime

[fierce ridge]

ah yeah that's the original problem

[fierce ridge]

the problem is Madam Liskov and her accursed substitution principle

in this case i want the apply method to have room to add various additional parameters and i don't want to put *args, **kwargs in the parent type signature

well i suppose i can, but that's a little ugly. because then mypy complains

[soft matrix]

oh ic

yeah fairs ive ran into this same thing before

[fierce ridge]

another option would be to refactor my code to pass in options via init

then it's all nicely substitutable

you know what, i'll just do that

Liskov wins again

oh i know why i didn't want that

one of my "parameters" is kind of time-related thing

so multiple calls to the algo will have different parameter values, and instantiating multiple instances of the algo seemed really silly

so instead of my_algo = Algo3(); my_algo.apply(x); my_algo.apply(y) i need to pass in some time-dependent contextual information, and any alternative interface design i can think of is just really bad

what i want:

algo = Algo3(db_conn)
algo.apply(x, x_prev)
algo.apply(y, y_prev)

what i'd have to do instead:

Algo3(db_conn, x_prev).apply(x)
Algo3(db_conn, y_prev).apply(y)

which is sort of not how i wanted these things to work

[eager vessel]

With protocol you actually don't have to inherit anything

At least that's the case if you use that protocol in some places where you pass your Algo classes

[fierce ridge]

true, i didn't think about that. however then i can't also easily enforce that a particular class implements a given protocol, can i?

[eager vessel]

No, until that class is used somewhere it's valid

e.g. passed or instantiated

Even if it doesn't implement all the methods and fields

[trim tangle]

You can make a "type test" like

def _(foo: Foo) -> None:
    bar: MyProtocol = foo

Or

_: type[MyProtocol] = Foo

Not sure if this works

[eager vessel]

Why inheriting a protocol is an issue for you? 🤔

[brisk hedge]

you also don't need to inherit from a protocol to implement it

[muted iron]

All Inheriting from Protocol does is potentially raising a metaclass conflict. So you probably should do it lol

[trail kraken]

How should I go about adding and using extra types in stubs for convenience purposes? This is for type-hinting a third-party library that I cannot make changes to.

Suppose I want to make use of JsonScalar = None | bool | float | str in a hand-written pyi. And that this type is not available in the original implementation. One way to use this is string literal. value: "JsonScalar" = None. Is this recommended? Does anyone have any suggested alternatives?

[vivid ore]

hey why does mypy think that an int to the power of an int is Any‽

[soft matrix]

Because typeshed has it that way

It's a bit of a meme

But it's because of negative/fractional powers I think

[vivid ore]

couldn’t it return int | float | complex?

[soft matrix]

Write out the alias yourself or define it as an import if TYPE_CHECKING and in the else block assign it to None or object or something

[lunar dune]

That would cause a huge number of false positives, unfortunately. The issue is that it turns out that the vast majority of the time people are doing x ** y where both x and y are ints, they generally know that both x > 0 and y > 0. So it's really annoying for those users to have to manually narrow the type of x ** y after the operation to ensure that the result really is an int, when from their perspective, it's really obvious that it has to be an int

[vivid ore]

hm

I have mypy on strict mode so returning Any gets me a false positive as well

but I see the reasoning

[lunar dune]

Yeah, there's no way of making everybody happy on this one 😦

[vivid ore]

dependent types? 🙃/s

[lunar dune]

No way of making everybody happy at the moment 🙂

[heavy bloom]

Are there any efforts to included exceptions in some type of capacity in type hinting?

[lunar dune]

no. Other languages have tried it, and all had bad experiences with it

It seems like it's one of those things that sounds like it might a great idea but turns out to suck in practice :/

See https://discuss.python.org/t/extend-type-hints-to-cover-exceptions/23788 for a long discussion

[heavy bloom]

Currently we have the sop to just add whatever exceptions can be thrown in the docstring but that only helps for stuff that we write ourselves and doesnt use external libraries

Oh ima read that thanks

[royal lagoon]

Hey, I don't know if this is possible but maybe it is just my lack of knowledge for typing.
The object has two methods register and build.
The usage would look something like this:

tmp_folder.register(module_name="Test", dir=Path(__file__))
tmp_folder.register(module_name="TestTwo", dir=Path(__file__))
tmp_folder.build(module_names=["Test"])

Is it possible for register to build some kind of literal on the fly for build to use so that I get type hinting on it?
in my example it would be Literal['Test','TestTwo']

[heavy bloom]

Thanks for this link, it put some light on why this is very hard to do. Is there a standard way to indicate what exceptions can at least be expected in a function?

Maybe this is more of a design question i guess

[lunar dune]

There are various docstring conventions you can use

E.g. here's an example of Google-style docstrings with a Raises: section: https://google.github.io/styleguide/pyguide.html#244-decision

[void panther]

I disliked the discussion there as the comments against it were because of checked exceptions like Java's while others were talking about some dedicated syntax that'd just be easier to understand to 3rd party tools and use by users instead of forcing handling

[lunar dune]

It's hard to see what the advantage of having dedicated syntax is if the exceptions aren't checked, though. If your target audience for this information is human readers of the code rather than static type checkers, we already have conventions for how to include this information in docstrings. And I'd personally find it confusing, honestly, to make exceptions part of the type system, but then not have them checked by type checkers

If what you want is something that can easily be understood by non-type-checker external tools, you could probably write a library that uses typing.Annotated to integrate exception-related metatadata into your type hints

[void panther]

from my PoV, standardizing it could go a long way towards making exceptions nicer to work with. A type checker could check it on a simple level (only direct calls and not trying transitively?) with some flag if you wanted to check during development or something like that, not sure if there's some other way it could be done to not be annoying or a dump of way too many warnings/errors

[eager vessel]

There are alternatives to raising an exception, if you need to handle exceptions that your own library/app throws - maybe use a different approach like union types or a result type:

def can_error() -> int | SomeError:
    ...

# or

def can_error() -> Result[int, SomeError]:
    ...

[rare scarab]

Go uses a tuple for errors

[eager vessel]

I'd say unions are easier to type check

[rare scarab]

Yeah.

[trail kraken]

I tried creating another module for the extra types on a previous project. Got cumbersome quickly, adding it everywhere.

That said, assigning the types to object. I hadn't thought of that. This would remove the need for literals at least.

[eager vessel]

With tuples you either have to check both result and err values, or have an ability to ignore the error
e.g.

result: tuple[Ok | None, Err | None]  # Have to check both
result: tuple[Ok, Err | None]  # Can use `Ok` without checking `Err` value and run into issues at runtime

[heavy bloom]

I thought about this but i would rather not put all the errors in the return section

[eager vessel]

I think that's the only way to type check that atm

[heavy bloom]

After reading the thread i think i dont want it to be type checked

In rust there is a guarantee that the code inside the function cant throw unless its specifically allowed to

In python there is no such guarantee

[eager vessel]

Just don't use exceptions? 🤔

[heavy bloom]

Ye but code used in functions can also raise

Code that we dont control that is

[eager vessel]

You mean 3rd party/python lib code?

[heavy bloom]

Ye

[eager vessel]

Yeah, you'd have to catch that manually

That's why I said it's a potential solution if you want to ensure that kind of safety in your code specifically

[heavy bloom]

Ye thats the problem, I could wrap it in just a bare try: except: but thats not something i want to do

[eager vessel]

result lib also provides a decorator to wrap your exceptions in Err iirc

i.e. something like

@wrap_error(ValueError, KeyError)

@as_result(ValueError, IndexError)
def f(value: int) -> int:
    if value == 0:
        raise ValueError  # becomes Err
    elif value == 1:
        raise IndexError  # becomes Err
    elif value == 2:
        raise KeyError  # raises Exception
    else:
        return value  # becomes Ok

[heavy bloom]

Are the value/key errors the ones that get raised in 3rd party code here for instance?

Ah like that

[eager vessel]

You'd still need to figure out what errors 3rd party code raises

[heavy bloom]

Ye thats what i want to get around

[eager vessel]

mmm

import third_party

x = third_party.do_something(...)  # could raise; who knows?

safe_do_something = as_result(Exception)(third_party.do_something)

res = safe_do_something(...)  # Ok(...) or Err(...)
if isinstance(res, Ok):
    print(res.value)

But I don't really like this

[heavy bloom]

I ve had a couple of exceptions just blow up since I didnt know those error cases existed

[eager vessel]

What are you developing?

[heavy bloom]

Its a data pipeline

[eager vessel]

I see

[heavy bloom]

Im building some internal libraries around some processes that exist in the org

[trail kraken]

And then there is always sigint that can happen anywhere. Or the cancel exception in asyncio.

[heavy bloom]

So ppl that know the processes dont have to come up with their own structure

And its more reusable of others

[trail kraken]

I didn't know about the result package. Looks neat.

[heavy bloom]

I assume its just the rust result

[fossil sorrel]

Can anyone recommend me a runtime type checker that will not hinder the performance to the oblivion? I am already using Pydantic with FastAPI, just need something for anything that is not an FastAPI endpoint function.

Was planning to use this but not so sure. https://github.com/beartype/beartype

[soft matrix]

Runtime typechecking user inputs like that using bear type if it's meant to perform validation

Bear type prides itself on being constant time which it achieves by not checking all of a sequence/mapping etc

[fossil sorrel]

I don't want this for user inputs, i just need something that can warn my coworkers/friends when they change a function signature incorrectly and try to use it

so yeah, I sorta need something like you described

problem is, beartype is doing randomized checks

so if it checks a List[str], and gets a ["good", 50] there is %50 chance it will throw an error. typeguard does a full check, but it is unfathomably slow

[chrome hinge]

Would a static type checker not be enough for that?

[fossil sorrel]

Well, technically yeah. But, it is still optional for you to run the code if mypy throws error

I want to force it somehow, because might be a problem if project spirals into some monstrosity

Or wait, can I add mypy checking to CI/CD

[lunar dune]

yes

[fossil sorrel]

what would make more sense then? runtime checker or static checker like mypy in ci/cd?

[slender timber]

runtime checking is not free

[fossil sorrel]

how bad it can cripple the code tho

[soft matrix]

Depends but it could be bad

[slender timber]

It can be breaking (as in change) bad

[fossil sorrel]

o shit

[trim tangle]

beware that beartype, gets "O(1) checking" by only checking a random element of a collection instead of all of them

And in general it is impossible except for very simple checks
https://decorator-factory.github.io/typing-tips/faq/runtime-checking/

[fossil sorrel]

yeah that is why I hesitated to use it

[trim tangle]

ah

I can't read

[fossil sorrel]

So mypy in ci/cd is way to go then?

because runtime check is either slow, costy, or totally random lol

[trim tangle]

Yep. Or pyright 🙂

The whole point of static typing is that you can find errors without running the code

[fossil sorrel]

which one would you recommend?

[trim tangle]

pyright supports more typing features, provides editor integration, and is faster. So if you can, definitely try it

mypy is more popular and supports plugins

[fossil sorrel]

Mypys only plus is pycharm plugin for now but I can get used to CLI that is no problem

[trim tangle]

I think Pycharm is literally the only editor Pyright doesn't integrate with

because Pycharm doesn't support the LSP protocol

[fossil sorrel]

fkk

Pycharm's own type checking is literally so bad compared to any external type checkers

[trim tangle]

yup

I switched to VSCode and have no regrets 😛

Well, one regret I have is that PyCharm has very good refactorings

[fossil sorrel]

Yeah, optimizing imports etc etc, refactoring is godsend

[trim tangle]

optimizing imports?

[fossil sorrel]

except that, there is like 38385857 other features that I don't use just being memory hog lol

I mean, deleting unused imports, writing standard library imports to top and third parties to bottom in order

[trim tangle]

ah

There's isort for that. And I think I have a Sort Imports button in VSCode as well

[fossil sorrel]

that is nice. i may switch to vscode if pyright is far superior than mypy

like all i care is typing lol

[trim tangle]

Well, try both 🙂

one potential downside of pyright is that it requires node.js

(it's written in TypeScript)

Editor support in VSCode is provided by Pylance, which is based on pyright

[fossil sorrel]

and also this can go in ci/cd right? like just write pyright something and done?

[trim tangle]

Yep

[fossil sorrel]

will definitely check this, thanks a lot

[heavy bloom]

I personally recommend still getting pyright to run inside your editor

It just speeds up the cycle

Or any other type checker for that matter

[trim tangle]

sure, but putting it in CI/CD as well just makes sure someone else doesn't screw it up 😛

or if a new version of the type checker comes out and it is now unhappy with something

[heavy bloom]

Oh ye ci for sure

I just meant it as, also locally

[trim tangle]

https://github.com/marketplace/actions/run-pyright Haven't tested this, but... it exists

[heavy bloom]

Is pyright that much better than mypy?

[fossil sorrel]

Dunno, we will see if it is worth switching editors for

[trim tangle]

The deal breaker for me is the editor integration

[heavy bloom]

I have pyright running in nvim

Through mason

[trim tangle]

yeah, pyright can run in any editor that supports LSP

[trim tangle]

Stuff like completion, renaming, showing type on hover and such.

[heavy bloom]

Does pycharm not support that?

[trim tangle]

Pycharm doesn't integrate with LSP

[fossil sorrel]

yep, only mypy plugin exists atm

[trim tangle]

And if you mean this, it does, but this information is from its own world, and it might disagree with the type checker

[trim tangle]

and its world can be... misaligned

[slender timber]

Just use ruff and forget about pycharm already

[trim tangle]

that's a weird replacement but ok

[fossil sorrel]

yeah i didn't heard about it

[slender timber]

Ruff as a VScode extension

[slender timber]

https://beta.ruff.rs

(what it claims is true)

[heavy bloom]

Its built in rust so it must be perfect /s

[ancient river]

i miss pycharm's debugger, even though i only used it like twice

[fossil sorrel]

oh, this instead of pyright you say?

hmmmm

will check this too thanks.

[slender timber]

no

instead of pylint, flake8, isort, pycodestyle, pydocstyle, lot more

[heavy bloom]

Does it replace all of those?

[slender timber]

Not entirely in terms of the rules

but it does it fairly completely and does it quickly

[heavy bloom]

Mhh i might look into that then

[fossil sorrel]

Yeah that is definitely interesting

dude, thanks to all of you again, i learned so many cool shit lol

[fossil sorrel]

Do I need to setup a seperate config for pylance?

or just ```toml
[tool.pyright]
include = ["src"]

[heavy bloom]

Mine just works out of the box

Oh you are talking about pylance, sorry that i dont know

My pyright works out of the box

[fossil sorrel]

bummer

cant even set this to strict :/

[trim tangle]

I think you need a pyrightconfig.json in your project root

there's a description of options in the pyright github repo

[brazen jolt]

nah, you can use pyproject.toml

it's been supported for a while now

oh wait, you're talking about pylance, not pure pyright, then I'm not sure

but I'd imagine it should pick up pyright settings properly

[fossil sorrel]

[tool.pyright]
include = ["app"]
strict = ["app"]
``` files under the app folder still checked by basic

so it probably doesnt

[brazen jolt]

does it work with pyrightconfig.json?

if not, you might actually need something vscode specific in .vscode/settings.json

[echo knot]

What do you mean with this? PyCharm does not use LSP?

[fossil sorrel]

{
    "include": ["app"],
    "strict": ["app"]
}
``` pyrightconfig.json didnt work and settings.json also didnt work, it is still on basic ```json
  "python.languageServer": "Pylance",
  "python.analysis.typeCheckingMode": "strict"

[trim tangle]

https://en.wikipedia.org/wiki/Language_Server_Protocol

[echo knot]

I know what it is. I just did not understand your comment

[brazen jolt]

that's correct, pyright has it's own built in solution, it doesn't have LSP integration

[trim tangle]

An editor has to support LSP to integrate with a language server

as in, it has to know how to communicate with a language server

[echo knot]

Pyright is a type checker. But that at the same time is integrated into Pylance which offers LSP capabilities

[brazen jolt]

pyright itself is a language server with type checking abilities

[echo knot]

Wel, pyright is half I think

[echo knot]

Yes! I forgot

[brazen jolt]

you can run it as a typechecker on it's own, but you can also run it as a lang server

pylance is just a fancy wrapper around it, with some changes, made to be a vscode extension

[echo knot]

That was the root of my question. Pycharm does not support LSP?

[trim tangle]

It does not

[echo knot]

I remember it was a little bit limited. But maybe it's because I used it long time ago

[echo knot]

I had no idea. That limits it a lot in terms of integration with other plugins

[brazen jolt]

that's really weird, I don't use vscode so I can't really test it, but I remember when I did use it, setting the typechecking mode to strict in the local project settigns .vscode/settings.json worked just fine

[fossil sorrel]

yh, it still shows as basic but picks up weird as fuck errors like

[brazen jolt]

I think this error is actually what you'd see with strict mode, with basic unknown types don't cause errors

[fossil sorrel]

yeah somethings is wrong with ui probably

thanks a lot

bro i didnt even have issue in mypy what the fuck

HAHAHAHAH

[brazen jolt]

might be worth investigating if the pyproject.toml settings were enough then though, if they were, it's probably a better idea to stick to that, as it'll work anywhere, not just with vscode

[brazen jolt]

haha yeah, strict is, as the name implies, pretty strict

I prefer to toggle some of the options manually, and leave the general mode at basic

[fossil sorrel]

i will probably do that if i dont have fun with this

[echo knot]

Were you using mypy strict too? I cannot imagine so many differences

[fossil sorrel]

it was ```ini
[mypy]
warn_redundant_casts = True
warn_unused_ignores = True

[mypy]
mypy_path = ./stubs

[mypy-*]
disallow_untyped_calls = True
disallow_untyped_defs = True
check_untyped_defs = True
warn_return_any = True
no_implicit_optional = True
strict_optional = True
disallow_any_unimported = True

[brazen jolt]

as an example from one of my projects: ```py
[tool.pyright]
pythonVersion = "3.8"

reportUntypedFunctionDecorator = "error"
reportUntypedClassDecorator = "error"
reportUntypedNamedTuple = "error"
reportTypeCommentUsage = "error"
reportConstantRedefinition = "error"
reportDeprecated = "warning"
reportIncompatibleMethodOverride = "error"
reportOverlappingOverload = "error"
reportMissingParameterType = "error"
reportUnnecessaryIsInstance = "error"
reportUnnecessaryCast = "error"
reportUnnecessaryComparison = "error"
reportUnnecessaryContains = "error"
reportUnnecessaryTypeIgnoreComment = "error"
reportShadowedImports = "error"

ignore = ["examples/**"]

[fossil sorrel]

can i yoink this?

[brazen jolt]

sure, go ahead

[fossil sorrel]

thank you thank you

[brazen jolt]

https://github.com/microsoft/pyright/blob/main/docs/configuration.md here's the actual spec describing all the options

[echo knot]

Is it fully strict? Some of them might me missing if it is not set up with strict = true. Maybe that's why

[fossil sorrel]

yeah probably, pylance calmed down when i used itsdrike config

seems better

[echo knot]

What's your experience with pyright? I always use mypy with fully strict mode. I cannot think about any cool feature I might be missing that pyright has

[brazen jolt]

I would heavily suggest you to go over at least the options I have there, and read a bit on what they do from the spec ^^

[brazen jolt]

I find that mypy often doesn't support the newest typing features as quickly as pyright does, or at least that's what was my experience with it a while ago, since than, I mostly sticked with pyright, so I don't actually have that much mypy exp.

[echo knot]

That's so right hahaha. I forgot that for Self support on mypy I had to wait way longer. Pyright was much quicker

[brazen jolt]

oh yeah, that was the primary thing that made me use pyright instead of mypy actually, I just haven't really looked back since

I suppose I should give mypy more of a try now, see how it does

[echo knot]

Since they adapt all PEPs, I think that the main difference is about bugs + how quick they adapt the features

[brazen jolt]

even draft PEPs, like say 702 (@deprecated decorator)? there is typing_extensions.deprecated already

to be fair, pyright does have a fair share of bugs, I myself actually reported like 2 already, and found some more, which were already being worked on, but the nice thing is that these are usually fixed very quickly, like within 2 days of the report, there was a new version out

[echo knot]

We were using pyright in my previous company a few years ago and we switched to Pyright because of the bugs. But it is cool to see that it is evolving so quick then!

[fossil sorrel]

pyright doesnt support self-referencing types i guess

if i iterate something over ```py
Optional[List["Model"]] = None

wait this is not related to self-referencing.

[echo knot]

I was refering to typing.Self, sorry 🙂
But the example you wrote is so basic. I think this one was working when I was using pyright years ago

[fossil sorrel]

yeah this is related to optional may return none

[echo knot]

You mean doing like this?

class A:
    def func(self) -> "A":
        ...

[fossil sorrel]

class Model(BaseModel):
    children: Optional[List["Model"]] = []

[brazen jolt]

well, you just need to make sure it isn't none first, obviously pyright can't let you iterat over children if it's none

[fossil sorrel]

yeah but how do i set to optional then?

Pydantic model by the way

[brazen jolt]

you just need to check it's not None first

[echo knot]

Ah, ofc you cant iterate because that variable might be None at some point. You need to narrow down the type by doing if variable is not None:

[brazen jolt]

hm @trim tangle is something wrong with pyright playground SSL cert?

[echo knot]

Then, the type checker knows that whatever happens inside that if, the variable cannot be None. So you can iterate over it

[trim tangle]

yes, the cert is expired

am too lazy to fix

[brazen jolt]

understandable

lol

[trim tangle]

just press Accept the Risk

in the worst case... your code that embarassingly fails pyright will be MITMd

[fossil sorrel]

so i am just gonna do

def optional(*fields):
    def dec(_cls):
        for field in fields:
            _cls.__fields__[field].required = False
        return _cls

    if fields and inspect.isclass(fields[0]) and issubclass(fields[0], BaseModel):
        cls = fields[0]
        fields = cls.__fields__
        return dec(cls)
    return dec
@optional("children")
class Model(BaseModel):
    children: List["Model"] = []
``` for shutting up pyright?

[brazen jolt]

why did you even mark children as optional here? Optional[T] is an equivalent to Union[T, None], it means children can literally be set to None, it does not mean that it's optional in that it doesn't need to be passed during __init__, because it has a default value

[fossil sorrel]

wait that is correct too

thanks, my brain is half cooked i presume

[rare scarab]

Don't use [] as a default value

[echo knot]

Why not? Since it is a class variable and not an instance variable it should be safe

[rare scarab]

That value will be used for all instances

[brazen jolt]

it's the same like ```py
class Foo:
def init(self, x = []):
self.x = x

[rare scarab]

If this is pydantic, use Field(default_factory=list)

[echo knot]

But pydantic and dataclasses will create as instance attributes all variables that are defined in the class scope. That's why they have default factories. But, in that snippet, children will be a class variable. Meaning that its purpose is to be in common for all instances, no?

[brazen jolt]

!e ```py
from dataclasses import dataclass

@dataclass
class Foo:
x: list[str] = []

a = Foo()
a.x.append("hi")
b = Foo()
print(b.x)

[rough sluice]

@brazen jolt :x: Your 3.11 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 3, in <module>
003 |     @dataclass
004 |      ^^^^^^^^^
005 |   File "/usr/local/lib/python3.11/dataclasses.py", line 1230, in dataclass
006 |     return wrap(cls)
007 |            ^^^^^^^^^
008 |   File "/usr/local/lib/python3.11/dataclasses.py", line 1220, in wrap
009 |     return _process_class(cls, init, repr, eq, order, unsafe_hash,
010 |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
011 |   File "/usr/local/lib/python3.11/dataclasses.py", line 958, in _process_class
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/4J4GN626FULF4HIXHCUIB4PJYQ

[brazen jolt]

ends up with ```
ValueError: mutable default <class 'list'> for field x is not allowed: use default_factory

[echo knot]

Ofc, because you are using a dataclass. That default value is only calles once and shared among all instances. Which most of the times is not intended

[brazen jolt]

not sure if the same beahvior would happen on <3.11 though

[echo knot]

But in the snippet he wrote, he is not using a dataclass

[brazen jolt]

I think this enforcement was only added recently

oh wait, it's supposed to be a class var?

[echo knot]

I think I am getting confused because there are two scenarios

[brazen jolt]

the type hint should reflect this then, there's typing.ClassVar

[echo knot]

I am supposing it is supposed to be a classvar, while you think it eould be an instance bar

[echo knot]

Dataclasses work differently when typing.clasvar is used. But non-dataclasses will just assume the variable is a classvar if it is defined in the class scope (regardless the type hibt). That's why I said that before

[brazen jolt]

but they're using pydantic in the snippet they sent

it's not a "regular" class

[echo knot]

Ah shit

I always use dataclasses and I forgot that BaseModel is from pydantic. I thought it was user defined

Indeed, then the list should not be used as a default value if the factory is not used. Sorry 😅

[rare scarab]

Til pip doesn't work

[fossil sorrel]

I am quite sure I dont understand this

what is the difference between this and just plain []?

[brazen jolt]

!mutable-default-args

[rough sluice]

Mutable default arguments

Default arguments in python are evaluated once when the function is
defined, not each time the function is called. This means that if
you have a mutable default argument and mutate it, you will have
mutated that object for all future calls to the function as well.

For example, the following append_one function appends 1 to a list
and returns it. foo is set to an empty list by default.

>>> def append_one(foo=[]):
...     foo.append(1)
...     return foo
...

See what happens when we call it a few times:

>>> append_one()
[1]
>>> append_one()
[1, 1]
>>> append_one()
[1, 1, 1]

Each call appends an additional 1 to our list foo. It does not
receive a new empty list on each call, it is the same list everytime.

To avoid this problem, you have to create a new object every time the
function is called:

>>> def append_one(foo=None):
...     if foo is None:
...         foo = []
...     foo.append(1)
...     return foo
...
>>> append_one()
[1]
>>> append_one()
[1]

Note:

• This behavior can be used intentionally to maintain state between
calls of a function (eg. when writing a caching function).
• This behavior is not unique to mutable objects, all default
arguments are evaulated only once when the function is defined.

[brazen jolt]

!e ```py
class Foo:
def init(self, x: list[str] = []):
self.x = x

inst1 = Foo()
inst1.x.append("hi")

inst2 = Foo()
print(inst2.x)

[rough sluice]

@brazen jolt :white_check_mark: Your 3.11 eval job has completed with return code 0.

['hi']

[fossil sorrel]

So you say it re-uses the [] everytime I construct this model?

[brazen jolt]

consider the above example

it's essentially equivalent to what you're doing, it's just that pydantic masks this a bit, and acts more like dataclasses

[trim tangle]

I think pydantic actually makes a copy

No?

[brazen jolt]

why would there be default factories then

also, it can't possibly catch every mutable tye

[fossil sorrel]

Yeah that confused me too

[trim tangle]

Because you might have custom objects

[brazen jolt]

it'd be weird if it special-cased lists

[trim tangle]

Pydantic is werid

[brazen jolt]

oh it actually does special case them?

huh

[trim tangle]

!e

import pydantic

[rough sluice]

@trim tangle :x: Your 3.11 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "/home/main.py", line 1, in <module>
003 |     import pydantic
004 | ModuleNotFoundError: No module named 'pydantic'

[trim tangle]

Aw

[soft matrix]

it just checks for __hash__ i think

thats what dataclasses does

[fossil sorrel]

I will use default_factory nonetheless, I am confused but yea lol

[brazen jolt]

but how does it copy it then, copy.deepcopy? that's so weird, I'd never expect that

[trim tangle]

🙂

Don't ask what it does to convert things to datetime

[rare scarab]

If you don't need it to be mutable, you can make the type Sequence with a default empty tuple

[fossil sorrel]

Sequence??

[soft matrix]

empty tuples are goated as defaults

[brazen jolt]

!d collections.abc.Sequence

[rough sluice]

collections.abc.Sequence

class collections.abc.Sequence``````py

class collections.abc.MutableSequence``````py

class collections.abc.ByteString```
ABCs for read-only and mutable [sequences](https://docs.python.org/3/glossary.html#term-sequence).

Implementation note: Some of the mixin methods, such as `__iter__()`, `__reversed__()` and `index()`, make repeated calls to the underlying `__getitem__()` method. Consequently, if `__getitem__()` is implemented with constant access speed, the mixin methods will have linear performance; however, if the underlying method is linear (as it would be with a linked list), the mixins will have quadratic performance and will likely need to be overridden.

Changed in version 3.5: The index() method added support for *stop* and *start* arguments.

[soft matrix]

wtf is that doc

[brazen jolt]

yeah lol

[soft matrix]

not sure why those arent separated

[fossil sorrel]

Now I am beyond confused

Lmao

[rare scarab]

A sequence is an abstraction of list and tuple

[brazen jolt]

this is from official docs

so that's why

[rare scarab]

Click the link

[echo knot]

Most types in Python are implemented from collections.abc. There, you can find broader types (container, sequence, iterable...). Although you cannot instantiate them, you can use them for type hinting

[rare scarab]

Hm...

[brazen jolt]

!d sequence

[rough sluice]

sequence

An iterable which supports efficient element access using integer indices via the __getitem__() special method and defines a __len__() method that returns the length of the sequence. Some built-in sequence types are list, str, tuple, and bytes. Note that dict also supports __getitem__() and __len__(), but is considered a mapping rather than a sequence because the lookups use arbitrary immutable keys rather than integers.

The collections.abc.Sequence abstract base class defines a much richer interface that goes beyond just __getitem__() and __len__(), adding count(), index(), __contains__(), and __reversed__(). Types that implement this expanded interface can be registered explicitly using register().

Similar names: pyside2.sequence, label.sequence

[trim tangle]

well
some types

[fossil sorrel]

ayo yeah it clicked

so this will

class Model(BaseModel):
    children: Sequence["Model"] = Field(default_factory=list)
 ``` create an iterable Model sequence with default factory being list right?

[brazen jolt]

yeah, it's basically just an abstract base class, which specifies a bunch of interactions and things that any sequence type (array like type) in python should support

the advantage is that when you do this, you can pass in any kind of sequence, not just a list, a tuple, set, frozenset, ... would also work

[echo knot]

Any any case, I do not see the point of using a Sequence. If you initialize children as an empty list, I guess that each instance will have the intention to manipulate it eventually (append, pop...). You cannot do that with a Sequence

Unless you reassign children again with a new whole list

[brazen jolt]

yeah, if you need to mutate it, just type-hint it as a list

as long as you use the default factory, it's not gonna be an issue

and well, apparently, it wouldn't be an issue anyway, because pydantic is pure magic

it's still probably better to use the def. factory tho

[echo knot]

You will drive him crazy with so many changes lol

[fossil sorrel]

I am already going crazy with this information overload lol

thanks again guys

[fossil sorrel]

can i make a typed dict equivalent of a pydantic model?

e.g: ```py
class MyModel(BaseModel)
class MyModelButTypedDict TypedDict(MyModel)

my OCD would really like an answer for this lol

Model dump of BaseModel from pydantic returns dict[str,Any]. pyright complains about not knowing the any. hm.

[buoyant ridge]

"Subscripted generics cannot be used with class and instance checks" - why not? I assume there's some example (which?) with this just doesn't make sense to do but with most of them it seems fine

instance checks of List[str], Literals, etc

(This is a type error when using a parametrized generic from the typing module - a similar error is from builtins, e.g. list[str] - TypeError: isinstance() argument 2 cannot be a parameterized generic)

[trim tangle]

https://decorator-factory.github.io/typing-tips/faq/runtime-checking/ 🙂

TL;DR it's not really possible

For simple stuff like tuple[str, ...] it might be possible, but it doesn't work for even slightly more complex stuff

It will also not "know" how to work with your custom types

[buoyant ridge]

ah, tysm! this makes sense, you'd have to modify lists to do type checking for any new arguments and such because of mutability

[trim tangle]

yes

or consider this:

def f(x: object) -> None:
    if isinstance(x, list[int]):
        x.append(42)

y: list[str] = []
f(y)
print(y[0].upper())

[buoyant ridge]

right that's a great example ty

they're type annotations but python is still not a statically typed language

[fossil sorrel]

this is not possible right?

[trim tangle]

wdym by "not knowing the any"

[fossil sorrel]

I mean

[{
    "resource": "/home/caner/something/something",
    "owner": "_generated_diagnostic_collection_name_#2",
    "code": {
        "value": "reportUnknownParameterType",
        "target": {
            "$mid": 1,
            "path": "/microsoft/pyright/blob/main/docs/configuration.md",
            "scheme": "https",
            "authority": "github.com",
            "fragment": "reportUnknownParameterType"
        }
    },
    "severity": 4,
    "message": "Type of parameter \"set\" is partially unknown\n  Parameter type is \"dict[Unknown, Unknown]\"",
    "source": "Pylance",
    "startLineNumber": 91,
    "startColumn": 9,
    "endLineNumber": 91,
    "endColumn": 12
}]
``` @trim tangle

in a function ```py
async def findAll(
set: dict = {},
) -> list:

problem is, dict is generally a dictionary representation of model

[trim tangle]

I don't see how this is related to pydantic, but you should use dict[str, Any] instead of dict

There's no way to type "a dict representation of a pydantic model" if that's what you're asking

[fossil sorrel]

type of "findAll" is partially unknown
  Type of "findAll" is "set: Dict[str, Any] = {} -> Coroutine[Any, Any, list[Unknown]]"
``` isnt the problem with str,Any ?

or it is with coroutine?

[trim tangle]

You didn't specify a parameter for list

hence list[Unknown]

If you can't specify a parameter, write list[Any]

[fossil sorrel]

yaaay fixed

thanks dude

now I have even weirder problem, that I literally have no idea how to fix

[trim tangle]

?

[fossil sorrel]

Alright, buckle up. I get py db["collection"].find_one() Object of type "AsyncRead" is not callable when i try to access a find_one object in Motor librarys motor.motor_asyncio.AsyncIOMotorCollection. In my handcrafted stubs: py AsyncIOMotorCollection: Type[AgnosticCollection] and in AgnosticCollection py find_one = AsyncRead(doc=docstrings.find_one_doc) is equal to ```py
class AsyncRead(Async):
def init(self, attr_name=None, doc=None):
"""A descriptor that wraps a PyMongo read method like find_one() that
returns a Future.
"""
Async.init(self, attr_name=attr_name, doc=doc)

how the hell i type hint this

https://motor.readthedocs.io/en/stable/api-tornado/motor_collection.html#motor.motor_tornado.MotorCollection.find_one I wrote this in my stubs

So it is indeed callable, like this is original file:

[trim tangle]

ah, motor is a complicated mess

[fossil sorrel]

class AgnosticCollection(AgnosticBaseProperties):
    # omitted 
    find_one = AsyncRead(doc=docstrings.find_one_doc)
``` and this is my stub file ```py
class AgnosticCollection(AgnosticBaseProperties):
    # omitted
    find_one: AsyncRead

this is a literal mess indeed

https://jira.mongodb.org/browse/MOTOR-331 looks like they dont even care

[trim tangle]

I get a 403 forbidden

[fossil sorrel]

all fine for me, it is just a jira ticket

thrown into backlog in May 01 2022

[trim tangle]

Well, it's an open-source project. If you want something added faster, you contribute it 🙂

[fossil sorrel]

if i can figure this out, i definitely will lol

is there better way to do this: ```py
class FinderProtocol(Protocol):
def call(self, filter: Optional[Any], *args: Any, **kwargs: Any) -> Coroutine[Any, Any, Optional[Dict]]:
...

find_one: FinderProtocol

find_one: Callable[[Optional[Any], Any, Any], Coroutine[Any, Any, Optional[Dict]]]
``` doesnt show the parameter names, and I cant set them

Quick documentation when i hover over a variable just says FinderProtocol, that is why I ask.

[tranquil turtle]

you can type find_one( and IDE will show you all arguments

[tranquil turtle]

also i think this is customizable via pyright option

--show-aliases or something like that

[fossil sorrel]

thanks a lot!

[fossil sorrel]

and no, no such option as this exists

[fossil sorrel]

never mind, my type hints in stubs doesnt catch the documentation generated in the original file's AsyncRead, so it is completely useless. i am just making functions ```py
def find_one(
self,
filter: Optional[Any] = None,
projection: Optional[Any] = None,
...
)

[echo knot]

Separate question: is Optional going to be deprecated in favour of int | None in the future?

[echo knot]

Just FYI, Optional[Any] is the same as just Any. But I understand it can be helpful in terms of readability

[chrome hinge]

(one thing that tends to confuse people is that Optional has nothing to do with optional arguments)

[echo knot]

I got confused by it when I started 😆
And the real purpose of Any and how this one differs from object

[fossil sorrel]

Need to ask one more dumb question that may stray from the topic

can i edit a functions docstring manually like find_one.__doc__ = find_one_doc in the stub file? i did this but quick documentation didnt catch up

documentation lies in find_one_doc at motor.docstrings, and in my stub file py async def find_one( self, #omitted ) -> Mapping[str, Any]: ...) I literally have no clue about where to write it.

[slender timber]

no

[fossil sorrel]

urgh.

[digital peak]

Is there any way to get my_value to show up as typing.Optional[int] ? (using PyCharm here)

import typing

_T = typing.TypeVar("_T")


def return_type(_type: _T) -> _T:
    return _type


my_type = return_type(typing.Optional[int])

my_value: my_type

my_value resolved to Any. This works for simple direct types like int, str, etc... But not for SpecialForm types like Optional or Union

[trim tangle]

PyCharm has a mind of its own 🙂

I don't think calls as type annotations are supported though

For the identity function you don't need this, and for more complex cases type checkers won't be able to figure the answer out

why do you want this though?

[digital peak]

dependency injection with inferred typing. So instead of my_value: typing.Annotated[typing.Optional[int], depend(my_dependency)] it could just be my_value: depend(my_dependency), where the depedency function actually returns an Annotated object. Suprisingly this works fine for many cases, excpet Optional/Union ...

[trim tangle]

What is my_dependency?

Actually I'm not sure you can make it work well

[digital peak]

any callable (typed)

[trim tangle]

Well, the point of dependency injection is that you don't couple yourself to the way of creating a value. So specifying a concrete callable kinda defeats the purpose

You could do something like

if typing.TYPE_CHECKING:
    Depends = Union[_T]  # generic type alias
else:
    Depends = _Depends()

Then you can specify Depends[int] as an annotation

So e.g.

@inject
def do_stuff(auth: Depends[AuthService], photo: Photo) -> None:
    ...

To a type checker the auth parameter is of type AuthService, but at runtime it's whatever _Depends.__getitem__ returns

@digital peak ^

[soft matrix]

does this actually work with a one element union?

[trim tangle]

No idea

I am about to go to bed so...

(but I don't see a reason it shouldn't)

[soft matrix]

it probably depends where the union gets flattened

[digital peak]

It's actually a bit more complex, as I'm interested in the return type of the depedency, which is not always itself like a class. Signature currently looks like:

def depends(dependency: typing.Callable[
    ...,
    typing.AsyncGenerator[_T, typing.Any]
    | typing.Generator[_T, typing.Any, typing.Any]
    | typing.Awaitable[_T]
    | _T
]) -> type[_T]: ...

And for edge-cases or untyped depednecy I could add an extra param to manually indicate the type:

@typing.overload
def depends(
        dependency: typing.Callable[..., typing.Any],
        _type: _T,
) -> _T: ...

If i can get the latter working with Optional/Union types, I would be satisfied

[trim tangle]

I'm assuming you're copying this from FastAPI which kinda completely mis-explains it

[digital peak]

And yes I understand that the latter basically is a more shitty re implementation of with typing.Annotated provides, but I'm contrained by an existing framework which I do not control

yes fastapi

I'm just trying to make a less verbose wrapper of fastapi.Depends, which can infer types in 90% of the cases

[trim tangle]

In that case I think you're out of luck

Mypy and pyright will just complain that you can't use a call as an annotation. Maybe PyCharm tries to guess

[digital peak]

We'll 90% of the cases will not have Optional/Union as return type, which do work fine. Just looking for a convinient way of explicitly providing the type for the oth 10% ...

[digital peak]

Making mypy happy is a problem for later 🙂

[viscid spire]

ohmagoodness pls from typing import ...

[trim tangle]

btw, importing Generator, Callable and some other stuff from typing is deprecated since Python 3.9

[digital peak]

Controversial, I don't always have choice, conforming to a style guide out of my control

[trim tangle]

(You're supposed to use collections.abc instead, unless you need to support 3.8 and below)

[viscid spire]

it is standard to do from imports with typing basically

what is the style guide you are following?

[digital peak]

Employer 🙂

[trim tangle]

Pallets (flask, jinja, etc.) do import typing as t or something like that

[viscid spire]

ye olde "basically"

[trim tangle]

But yeah most people use memberwise imports, otherwise your eyes start escaping your skull on annotations more complicated than int

[viscid spire]

I'm assuming it's a blanket "import namespaces" sorta thing or whatever

[trim tangle]

It will be very fun writing out collections.abc.Awaitable[collections.abc.Sequence[typing.Union[int, str]]]

[viscid spire]

sobbing at that sight

[digital peak]

Still, I feel like the following can be shorter: value: Annotated[Optional[int], Depends(my_dependency)]. Annotated an Depends feel duplicate, would be nicer if fastapi would just regocnize any param annotated with a function automatically as dependency..

[viscid spire]

let's have some relief Awaitable[Sequence[int | str]] (phew!)

[trim tangle]

I would definitely bring up the issue of typing and PEP585 to people maintaining your style guide

[trim tangle]

Because this is madness

[digital peak]

We're getting sidetracked, lets ignore the styleguide 😂 (it's also not as bad as your example, exceptions are definetly made)

[digital peak]

agree with this?

[trim tangle]

I can't really comment because I think you shouldn't depend on the concrete "factory" at all. Though that's what the framework requires

But putting it as metadata does prevent you from finding a mismatch between the expected and actual type

[echo knot]

Quick question. Is Optional going to be deprecated in favour of using the union operator with None?

One set of rules from Ruff (FA) enforces removing Optional by union + None

[trim tangle]

I haven't seen any deprecation plans for it. Removing it will probably break a lot of libraries...

It's more of a there's
-a-better-way-now thing. Like %-formatting

At least if you have the piped or available

[echo knot]

I guess it is because usinf | None now is more handy than before, as you needed to inntroduce Union just to indicate None. Maybe now with the | some linters decided it would be better if you just weite | None 🤔

Ok thanks!

[viscid spire]

you didn't need to introduce Union

Optional

[echo knot]

Right, Optional was always there

[viscid spire]

it's not like the typevar boiler plate 😄

[echo knot]

I was just trying to understand that rule introduced in Ruff that made all my pieplines fail

[viscid spire]

3.13 importless Optional

type Optional[T = None] = T | None

[digital peak]

https://peps.python.org/pep-0695 🙂

[viscid spire]

I would assume that I know about it if I just wrote this... 😉

[digital peak]

I'm too slow 🤷‍♂️

[echo knot]

I got so lost with this lol

[viscid spire]

anyways, surprised that Pylance already has typevar defaults implemented lol

[echo knot]

I really need this feature. Mypy seems to be far from it

[viscid spire]

you can't use it btw...

like I said, 3.13

that's two versions away

[trim tangle]

Making plain Optional an alias to None 💀

[viscid spire]

(the = None part)

[viscid spire]

it already is

[echo knot]

Typevar defaults is only in 3.13? Cant you use typing_extensions to import typevar from there?

[trim tangle]

Hm?

[trim tangle]

I think Optional with no parameter is just invalid

[viscid spire]

the syntax I mean

[trim tangle]

Or implicitly Optional[Any]

[viscid spire]

oh wait...

it was messed up cause I wrote in that Optional type definition lol

so uh, nvm

[soft matrix]

its not in 3.13 yet but yes

[viscid spire]

okay 3.12 ready

type Optional[T] = T | None

[soft matrix]

i asked jelle about changing the def to that but alas itd break too much stuff

[echo knot]

Maybe I am missing something. But, for the signature, importing it from typing_extensions should provide backwards compatibility to include the default keyword. And about the syntax, with the postponed evaluation should not affect Python runtime 🤔

[soft matrix]

cant do postponed eval no but yeah you should get default=...

also auto_variance

[echo knot]

Never heard about auto_variance. Cant find anything about it

[soft matrix]

its a pep 695 thing

[echo knot]

I still need to read about this PEP. Kinda long but big improvement

It would be great if they implemented the lind of from __future__ import annotations for that PEP 🤔

[soft matrix]

they basically do hehe

its co_annotations but same sorta outcome

[echo knot]

What do you mean?

It seems that with this PEP I can see that they will be automatically covariant if I understood well

Oh, mevermind. It is auto-infered

[soft matrix]

no its nothing to do with variance what youre referring to here

im saying you dont need quotes for forward references with pep 695

[echo knot]

Yeah. I explained it in a really wrong way hahah. I was meaning that it could be great if this different way of working with typevars and generics could be ported to lower versions

[soft matrix]

just go to sleep for 5 years ;)

[echo knot]

I will, it is late here 😭😭😭

[viscid spire]

just gotta wait for 3.11 EOL 😂

[lunar dune]

Long answer for you here (short answer is "no"): https://discuss.python.org/t/clarification-for-pep-604-is-foo-int-none-to-replace-all-use-of-foo-optional-int/26945/6

[fierce ridge]

inheritance leads to the Liskov substitution check that i'm trying to avoid

[fierce ridge]

oh that's an interesting idea for the protocol. maybe i can do a test like _apply: ApplyMethodCallable = Algo3.apply where ApplyMethodCallable has the __call__ method with the signature that i want

[eager vessel]

You could make it generic?

And have a single dto with all parameters

class Algo(Protocol[T]):
    def apply(self, params: T) -> ...:
        ...

@dataclasses.dataclass
class Params1:
    a: int

class Algo1(Protocol[Params1]):
    def apply(self, params: Params1) -> ...
        ...

[fierce ridge]

i wanted to avoid this too but it is definitely a valid solution

[eager vessel]

If you have more than 3 parameters dto is definitely better in my opinion

[fierce ridge]

i don't expect to, most other parameters can be passed in init

this is for things like a previous algo output, in a recurrent algorithm

but it might be the best option anyway

[low escarp]

I have a class that look like this:

class EnumMapping(dict):
  # Implementation, not important...

which will act like a dictionary, however the keys are restricted to string or enum values, and the values will be some custom class.
I want to do some kind of generic type hinting such that I can do this:

collection: EnumMapping[SomeClass] = EnumMapping(MyEnum, SomeClass(), SomeClass())

I have tried to let the class subclass from Mapping[Union[str, Enum], T] where T = TypeVar("T"), but vscode didn't seem to like it.
I have tried to add

def __class_getitem__(cls, item):
  return Mapping[Union[str, Enum], item]

but vscode didn't like it.
The only thing I have got to work was to define
EnumMappingType = Mapping[Union[str, Enum], T] as a generic type alias and then use this as the type, but I'd really like to use the actual class instead of an alias like this if possible.
Btw by "vscode didn't like it" I mean that intellisense wasn't able to realise that when I did collection[key].method that the method belonged to the class I specified as the generic type.
Thanks!

ok. classic me... Figured it out after asking the question.
The solution was to define the class like this:

class EnumMapping(dict[Union[str, Enum], T]):
  # Implementation, not important...

Now vscode is happy 🙂

[echo knot]

Wouldnt it be better to inherit from collections.UserDict?

[low escarp]

I did look at that, but my class is very simple, and only overrides the __init__ method, so I figured it was simpler to just subclass for dict. Unless there is some better reason to use UserDict?

[soft matrix]

I thought UserX was basically pointless inheriting from at this point

[echo knot]

Is there an easy way to type hint a Callable with any number of arguments? Is it only with Protocol?

[echo knot]

Well, if in the future the class is extended and override some other methods, you might have uncertain behaviour, which might take some extra time to debug if you dont pay attention or if the developer does not know about it. But currently there should not be any problem

[low escarp]

Ok, thanks.
I think in this case it shouldn't be an issue as it's just meant to simplify doing some kind of switching between two different objects returned. So I think it won't ever be extended or anything.

[soft matrix]

All of the same type?

[echo knot]

No. Any types and any number of arguments. If I do typing.Callable I am forced to write input arguments and return types. I can do [[Any], Any] but this means a single input argument. With typing.Protocolandcall` I can define this by doing:

class AnyFunc(Protocol)
    def __call__(*args: Any, **kwargs: Any) -> Any:
        ...

[brisk hedge]

You can use ... in place of the parameter list: Callable[..., ReturnType]

[rare scarab]

A lot of people make this a typevar. Makes writing decorators simpler. ```py
AnyFunc = TypeVar("AnyFunc", bound=Callable[..., Any])

def decorator(func: AnyFunc) -> AnyFunc:
return func

[echo knot]

Now that you say it, I think I used it long ago 🤔🤔🤔

Thanks!

[echo knot]

Yeah. I was doing this exact thing for my decorator factory but with a Protocol

[turbid lotus]

can i do something like this in python

the example is in java

public <T> T foo(T x) {return x;}

public <T> int bar(T x) {return foo(1234);}

bar("abcd")

the generic T is localized to the specific function/method

instead of this

T = TypeVar("T")

def foo(x: T) -> T:
    return x

def bar(x: T) -> int:
    return foo(1234)

bar("abcd")

T have two different possible types in foo and bar because they have different scopes, if you were to erase the generics in the example it would look like this:

def foo(x: int) -> int:
    return x

def bar(x: str) -> int:
    #      ^^^ not the same as int in foo()
    return foo(1234)

bar("abcd")

[tranquil turtle]

!pep 695

[rough sluice]

**PEP 695 - Type Parameter Syntax**

Link

Status

Accepted

Python-Version

3.12

Created

15-Jun-2022

Type

Standards Track

[turbid lotus]

any release date for this?

[turbid lotus]

yes, when will python 3.12 be released?

[tranquil turtle]

!pep 693

[rough sluice]

**PEP 693 - Python 3.12 Release Schedule**

Link

Status

Active

Python-Version

3.12

Created

24-May-2022

Type

Informational

[turbid lotus]

ok thanks

[rare scarab]

You'll get a warning if you only use a type parameter once in a function signature.

So bar is essentially ```py
def bar(x: Any) -> int:
return foo(1234)

[trim tangle]

Yeah, that does not make sense

[rare scarab]

maybe there'd be a use if we had reified generics.

though I don't see why we'd need them when we can just use an argument.

[soft matrix]

For the type time staticness :(

[fossil sorrel]

This might be very off topic for this channel but don't know where to ask, does anyone know a security linter like this https://github.com/securego/gosec?

[rare scarab]

!pip bandit

[rough sluice]

bandit v1.7.5

Security oriented static analyser for python code.

[fossil sorrel]

hm

How strict is this?

[rare scarab]

!pip flake8-bandit with a flake8 plugin

[rough sluice]

flake8-bandit v4.1.1

Automated security testing with bandit and flake8.

[fossil sorrel]

I am using pyright, and i am comfortable with strict setting of it

I would appreciate something more stricter tho

[fossil sorrel]

but will check this, thanks a lot

[rare scarab]

Here's a list of all the checks: https://bandit.readthedocs.io/en/latest/plugins/index.html#complete-test-plugin-listing

[fossil sorrel]

List seems like we won't encounter any of the errors, but I will use it anyways just to be sure lmao

thanks again mate

[lunar dune]

@fossil sorrel Since you've asked this question in the type-hinting channel, you might be interested in pysa, which is a security-analysis tool built on top of the pyre type checker: https://pyre-check.org/docs/pysa-quickstart/

[fossil sorrel]

Looks like very complicated, but more comprehensive than bandit

thanks, not sure if i can use it

[lunar dune]

yes

[echo knot]

There is not much else you can do with a type checker in strict mode. Like, yeah, you can enforce 100% type hints and veryfying there is type coherency. And then some more extra things like unreachable code and basic studf

[spice locust]

is it acceptable to use an abstract base class as a type hint?

if the type hint is Type[MyABC], is there a way to specify that the user can pass in any subclass of MyABC but not the MyABC type itself?

[rare scarab]

If a class is abstract, you can't init it

[spice locust]

yes, that much i know, but one can still pass the class object (not instance) around without issue, e.g.

var: Type[MyABC] = MyABC

[worn rose]

Hello guys!
I've faced the challenge to map core errors onto gql.
My current solution is to use @overload but it's badly scaled.
Maybe someone has already faced this problem and can give me some advice

example
from typing import overload
from core.errors import NotFoundCoreError, PermissionCoreError
from gql.errors import NotFoundGqlError, PermissionGqlError

@overload
def map_core_error_to_gql(error: NotFoundCoreError) -> NotFoundGqlError: ...
@overload
def map_core_error_to_gql(error: PermissionCoreError) -> PermissionGqlError: ...

def map_core_error_to_gql(
    error: NotFoundCoreError | PermissionCoreError,
) -> NotFoundGqlError | PermissionGqlError:
    match error:
        case NotFoundCoreError():
            return NotFoundGqlError.from_core(error)
        case PermissionCoreError():
            return PermissionGqlError.from_core(error)

[rare scarab]

I can't think of a better way.

But do you have to have an exact conversion type? There's no BaseGqlError?

[worn rose]

There's BaseGqlError but i guess it's inappropriate for me to map all errors onto the one type. Thanks for your feedback, i'm gonna type monkey code 🙂

[trim tangle]

How are you calling this function?

[worn rose]

I use the result container in the core

[trim tangle]

could you give an example maybe?

Is it something like this?

def foo(bar: Bar) -> Result[int, NotFoundCoreError | SomeOtherSpecificError]:
    ...
``` or like this? ```py
def foo(bar: Bar) -> Result[int, BaseError]:
    ...

And if you use the first variant, do you really have to know that you map into NotFoundGqlError | SpecificGqlError? Why?

[worn rose]

You wrote first 😅 I use the first variant. To return an error to the client in graphql, you need to register it. I don't want the core to know graphql, that's why i don't want core errors to be immediately in the scheme

[errant tulip]

why does mypy throw error when I pass dict_keys[str, int] when Sequence[str] is expected. Isn't it a bit too much?

[trim tangle]

You can index a Sequence

If you want any iterable, you need Iterable

[errant tulip]

ah makes sense.. so I should just use Iterable[str] in my function?

[trim tangle]

yes

[errant tulip]

nice. thanks

[echo knot]

Just type hint like:

variable: MyABC

You can assign any instance that comes from the class MyABC. Since MyABC is abstract, in reality you will only able to pass instances of MyABC childs, as you cannot create direct instances from MyABC

[tranquil turtle]

Instances of MyABC childs*

[echo knot]

Yes sorry! Gonna edit it

I was wondering today. Is there a way to combine advantages of TypedDict with Mapping?

In the sense that no __setitem__ method is set in the typed dict at the level of type hints

[foggy snow]

wtf does this have to do with this topic? stop spamming

[verbal spoke]

Keep it on topic and don't harass other users. Treat others as you would want to be treated.

[spice locust]

is it necessary to type hint class variables with ClassVar[...]?

[soft matrix]

you dont need to put things inside the assignment but ClassVar can be useful yes

[fierce ridge]

not currently but i think this is an open feature request

a non-mutable typeddict would also allow subtyping with overrides as well as type covariance, which would be really nice features

[echo knot]

All type hints are most of the times optional. What do you mean by "is it necessary"?

[echo knot]

Ok thanks!

[spice locust]

in an abstract base class for example

class Foo(ABC):
  x: str

or

class Foo(ABC):
  x: ClassVar[str]

[echo knot]

There is no difference in runtime. So if we focus on the type hint itself, typing.ClassVar denotes a variable that can only be changed from the class itself, and not the instance. If you try to modify that x in an instance, type checkers will throw an error.

Usually, this is the expected behaviour for a class variable. If that makes sense to you, you can add it

Technically, it is not necessary. Not a single type hint (but in some cases like dataclasses or named tuples). But by adding this, you give more informations to the type checkers about how your code works. This will be helpful for type checkers to perfrom extra checks. Which might be handy for refactoring or spotting potential bugs

And the fact that you commented it is an abstract class, it does not affect either. Although the concept of instances does not make sense within abstract classes, any non-abstract class deriving from it will also inherit the class variable with that type hint

[spice locust]

the reason i ask is that both mypy and pyright have no difference in behavior with or without the ClassVar there

[echo knot]

There is:

[spice locust]

the only difference is that the type checkers error on assignment of a classvar, and dont when classvar is omitted. so i guess i'll leave it in

[echo knot]

In line 6, I edit the variable from the instance. Throwing an error. In line 7, editing it from the class is OK to Mypy

[echo knot]

Do not mix it with Final. This type hint means that the variable cannot be assigned. Not in instance, not in class.

ClassVar can be assigned from class, but not from instance

[spice locust]

i meant assignment on the instance, sorry should have been more specific

[echo knot]

Then yes. That can be seen as the only difference 🙂

Which to me is a big one

[pastel egret]

Another one might be with descriptors, it should only use __get__/__set__ if it's a ClassVar.

[spice locust]

i noticed pylint correctly gives warnings about subclasses of an abstract base class not overriding methods when the base class is created with metaclass=ABCMeta, but does not give any warnings at all if the base class inherits from ABC instead

[pulsar thunder]

import utime
from machine import Pin, PWM

Button = Pin(20,Pin.IN)
Butto = Pin(17,Pin.IN)
pwm = PWM(Pin(15))

MIN = 100000
MID = 150000
MAX = 200000

pwm.duty_ns(MID)
pwm.duty_ns(MAX)

left = Button(17)
right = Butto(20)

while True:
left = MIN
right = MAX

how could i add a code to that so when i press the button the servo moves?

like when i press right button servo rotates right

and i press left it rotates left

[fossil sorrel]

are we sure you asked in right channel

[soft matrix]

does anyone think turning off lsp checking bool would be problematic?

im thinking of trying to make object.bool -> Literal[True] so pyright can do better reachibilty analysis

[trim tangle]

Hmmmmmm

Everything inherits from object though

So that's just false

[dull lance]

from the type stubs, the signature is (Self) -> bool. that should be compatible with (Self) -> Literal[True] since the return type is more specific

[trim tangle]

No that's backwards

[dull lance]

return types are covariant, no?

[trim tangle]

def f(x: object) -> None:
    if not x:
        print("Foo")

There are cases where this function prints Foo

[trim tangle]

You can return a more specific thing from a child class. But not a more general thing

If your parent promises to return a string, a child cannot return an integer from that method

[dull lance]

Literal[True] is more specific than bool

[trim tangle]

Yes but object is the parent of all other classes

[dull lance]

oh wait, I misread Gobot's question

they want to change object's type hint

I see, then yeah

[soft matrix]

maybe it does make everything unless then

[trim tangle]

?

[soft matrix]

well if you cant then say anything other than final classes have a consistent bool then you break basically every check like this

[trim tangle]

[tranquil turtle]

That reminds me of object.__hash__ and Hashable LSP problems

[fierce ridge]

do tell

[buoyant ridge]

Mostly pydantic related, but I also might be unnecessarily subclassing str:

from pydantic import BaseModel
class Tag(str):
  def __new__(self, value: str):
    # throw error if value is not valid
    if len(value) < 3:
      raise TypeError("<3")

class Foo(BaseModel):
  tag: Tag
  x: int

Desired behaviour is to coerce a str into a Tag, but I get pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class Tag>

I'm kind of hoping there's a solution where I can just tell pydantic how to coerce into Tag - it feels like I shouldn't have to fundamentally change how Tag works?

[lunar dune]

>>> hash(object())
298204531
>>> class Foo:
...   def __eq__(self, other): return self is other
...   
>>> issubclass(Foo, object)
True
>>> isinstance(Foo(), object)
True
>>> hash(Foo())
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: unhashable type: 'Foo'
>>> object.__hash__
<slot wrapper '__hash__' of 'object' objects>
>>> Foo.__hash__ is None
True

[fierce ridge]

instead of overriding __new__ you might need/want to use a pydantic validator instead

that's how pydantic knows what to do

at least v1, i haven't switched to v2 yet

[fierce ridge]

oh, like Foo doesn't inherit object's "__hash__" method?

[buoyant ridge]

ah, ty

[fierce ridge]

it looks like in pydantic v2 the technique is slightly different

this is the pydantic v1 version that i know @buoyant ridge

from collections.abc import Callable, Iterable

from pydantic import BaseModel


class Tag(str):
    @classmethod
    def build(cls, s: str) -> Tag:
        if ...:  # check validity
            raise ValueError(f"Invalid input: {s}")
        else:
            return cls(s)

    @classmethod
    def __get_validators__(cls) -> Iterable[Callable[[str], Tag]]:
        yield cls.build


class Foo(BaseModel):
    tag: Tag
    x: int

where cls.build can be anything, and __get_validators__ returns an iterable of functions that would be valid to use with pydantic.validator

the v2 equivalent is here https://docs.pydantic.dev/latest/usage/types/custom/#customizing-validation-with-__get_pydantic_core_schema__

yikes this looks complicated. the api is starting to remind me of cattrs. this is going to be a hairy upgrade in our app 😬

[buoyant ridge]

Oh this is perfect, thank you. I got lost elsewhere in the documentation. Tysm haha

[fierce ridge]

i hate pydantic docs

[lunar dune]

Foo is a subtype of object (because all classes are subclasses of object), and object is a subtype of collections.abc.Hashable (since instances of object can be hashed). So therefore it should follow that Foo is also a subtype of Hashable. But unfortunately this is not the case. There are many Python classes whose instances are not hashable, so the normal LSP expectations are just completely violated when it comes to __hash__ methods. And indeed, Python will automatically set __hash__ to None if you override __eq__ on a class but don't explicitly also override __hash__

[fierce ridge]

i see. but you aren't really supposed to call __hash__ manually so it's kind of ok, right?

actually wait, no, it's an actual LSP violation

[lunar dune]

No, because the builtin hash() function calls __hash__

[fierce ridge]

yeah i realized

but you can still call hash() and it's valid statically from a type checker perspective

so i'm not sure it's an LSP violation. it's just that hash() raises an exception unconditionally in the child class which is a runtime thing (no checked exceptions)

[lunar dune]

And the builtin hash() function is called whenever you try to add an element to a set or use it as the key of a dictionary, so this is core to Python's design

[dull lance]

LSP isn't just about static types. It should also hold in terms of behaviour

[fierce ridge]

yeah but if behavior never changed in child classes then what would be the point?

[dull lance]

From wikipedia:

• Preconditions cannot be strengthened in the subtype.
• Postconditions cannot be weakened in the subtype.
• Invariants must be preserved in the subtype.

[fierce ridge]

so what are the preconditions, postconditions, and invariants? i'd argue that they're preserved here

the invariant is that the hash function behaves with such-and-such properties, or raises an exception if the object is unhashable. that's exactly what's happening

[lunar dune]

Subtype Requirement: Let φ(x) be a property provable about objects x of type T. Then φ(y) should be true for objects y of type S where S is a subtype of T.

It is a property provable about objects x of type object that any instance x can be safely passed to the builtin hash() function without any exception being raised, can be used as a dictionary key, or can be added as an element to a set. However, this is not true for instances of any arbitrary subtype of object. Thus, LSP is violated.

[dull lance]

If you put it that way, yes it would hold

[dull lance]

I guess now the problem is that different people might have different interpretations of the invariant

What is the invariant that we're trying to preserve with hash()?

[lunar dune]

Interpreting the concept of invariants in such a loose way would completely contradict the way type checkers have understood LSP as it applies to Python up to this point, and dramatically lessen the usefulness of the concept as it applies to Python, in my opinion

I disagree with the idea that dunder methods should be treated differently to any other methods in Python when it comes to LSP, just because you're generally not "supposed" to call them directly. They are very relevant to the ways in which objects can be used in Python code. Defining a dunder method on a class creates a set of invariants about that class, just the same as with any other method. The only difference is that Python has special syntactic support for many dunder methods.

Do we agree that, in general, this violates the Liskov Substitution Principle?

class Foo:
    def method(self) -> None: pass

class Bar(Foo):
    method = None

[dull lance]

Yeah I can see that, this is also why it is bad practice to inherit a base class that has a method that does not make sense, even if you "override" the method by raising an error - you are basically "lying" to the type checker

[lunar dune]

Yes -- it's still arguably an LSP violation, just not one that the type checker can detect

[dull lance]

the hash function behaves with such-and-such properties, or raises an exception if the object is unhashable
If we were to be strict, we can express this invariant statically as follows:

class object:
    # Use NoReturn for the case when the object is not hashable
    def __hash__(self) -> NoReturn | int: ...

[trim tangle]

That's the same as returning int tho

Though yeah, LSP only makes sense to talk about if you explicitly specify a property

Case in point: dict/Mapping vs defaultdict

[lunar dune]

int, when used as a type hint, is conceptually the same as "the set of types such that all members of the set are subtypes of int". NoReturn, when used as a type hint, is conceptually the same as "the empty set of types", since no type is a subtype of NoReturn. Thus, int | NoReturn -- the union of the two sets -- resolves down to the same type as int.

Short version: as @trim tangle says, int | NoReturn is identical to int from a type checker's perspective

[dull lance]

yes, so this shows that type checkers are simply unable to check for such invariants

[trim tangle]

A union with an empty set is an identity operation 🤓

I think you're supposed to set __hash__ to None if your object is not hashable

!e

print(list.__hash__)

[rough sluice]

@trim tangle :white_check_mark: Your 3.11 eval job has completed with return code 0.

None

[lunar dune]

yes, and as I demonstrated in #type-hinting message, Python even does this for you if you override __eq__ without explicitly overriding __hash__

[trim tangle]

Ye

Peak discord, adding an embed for a discord link

[dull lance]

if you define the invariant this way, the type checker may have false negatives in terms of detecting errors in LSP

for something that is a core Python feature, that may not be such a good idea

[lunar dune]

Small correction though: it is setting __hash__ to None that makes your object not hashable. You're supposed to set __hash__ to None if your object is not immutable, as mutable objects cannot be safely hashed; this would violate other invariants about Python objects in scary ways if you could hash them

[trim tangle]

What about just raising in __hash__?

Well, mutable objects can be hashable 🙂
(Classes for example. And any other type that compares by identity)

[lunar dune]

Also fine, but why bother, when hash() gives you such a nice error message when __hash__ is set to None? It's less characters to type to just set __hash__ to None 🙂

[lunar dune]

Yes. Functions as well. I believe the exception here is due to the fact that instances of type and types.FunctionType are all conceptually singletons and, as you say, are typically compared using identity rather than equality

[trim tangle]

In fact every class by default has hashable instances that compare by identity

Kinda useful for e.g. asyncio.wait_for

(which returns a set)

[lunar dune]

yeah you're right, it doesn't really make sense

sigh

[tranquil turtle]

Why mutables cannot be hashed?
You can set hash to some constanst, and it would satisfy all requirements, but it wouldn't be efficient

I can think of another example where you can evaluate hash to not constant value: datasteucture that consists of some name/key(which is a hashable string) and a list of values. You can define hash of this ds to be hash of its name. Yes, it wouldn't be efficient if you have a lot of objects with same name, but it would be good enough if there is not very much objects with same name

[tranquil turtle]

Oh, i forgor about classes and functions :(

[lunar dune]

In general it is desirable to have the invariant hold where if x == y, therefore hash(x) == hash(y)

Otherwise you end up with scary things like a dictionary that has two keys that compare equal

Perhaps it's generally okay to have mutable objects be hashable, as long as mutating the object does not affect the way in which equality operations evaluate when it comes to that object. Not sure though.

[trim tangle]

Yeah mutability is only bad in that it can violate that requirement

Also that the hash shouldn't change over the object lifetime

Actually mostly the second one

[fierce ridge]

hm, i see what you mean here. i guess the problem is conflation of a completely empty base type with zero behavior other than its own existence and the actual object type, which has a small amount of individual behavior like instances being hashable

fwiw i don't think i'll lose sleep over it, especially if mypy treats object as non-hashable and requires you to use Hashable instead or similar

i don't know if there's a way to say x: object, but invariantly which would loosen that restriction and make the lsp violation a non-issue

[fossil sorrel]

I dont know if this is appropiate place, but does anyone knows a library like this but for pydantic v2? https://github.com/mansenfranzen/autodoc_pydantic

[rare scarab]

mansenfranzen/autodoc_pydantic#146

[mighty linden]

GitHub

[autodoc_pydantic] #146 Support pydantic v2 & test on pre-releases?

[fossil sorrel]

yeah it doesnt support yet

buuuuuut, there is no replacement too in v2.

[rare scarab]

then none that I'm aware of.

[primal raptor]

What am I missing here?

@dataclass(eq=False)
class Turner(Creature):

    turns: int = field(init=False, repr=False, default=0)

    def apply_turn_effects(self, tick_delta: int) -> None:
        super().apply_turn_effects(tick_delta)
        self.turns += 1

[soft matrix]

that looks like a bug

can you get it in https://mypy-play.net?

[primal raptor]

hmm now i'm hitting errors out of code that was already removed, so I'm hitting some kind of mypy caching issue

well, removed the cache and the old code error was removed but the one I showed is still showing, I'll look to make a commit that I can share, not sure how easy I'll have in trying to isolate the issue, the codebase and these classes aren't small

[tranquil turtle]

not a bug
after creation of lambda creature var can become a Creature again, so this typeckecker behaviour is correct

tldr: narrowing works only in current scope, not in child scopes

[rare scarab]

type checking is wonky when it comes to lambdas

[tranquil turtle]

class X: ...
class Y(X): a: int

x: X = [X(), Y()][0 or 1]

if isinstance(x, Y):
    f = lambda: x.a
else:
    f = lambda: ...

x = X()
f() # BOOM!

[primal raptor]

oh yeah closures work on variables not values

that always trips me up

[rare scarab]

You can do this to fix it. (lambda creature: lambda: creature.turns)(creature)

It "binds" the variable to the lambda

[tranquil turtle]

i dont think this is correct

[rare scarab]

I missed a lambda

[tranquil turtle]

lambda creature=creature: creature.turns should also work

[rare scarab]

If python ever decides to add arrow functions, they need to make it auto-bind

yay for more namespaces.

[primal raptor]

I hacked it register("Turns", lambda: creature.turns if isinstance(creature, Turner) else 0)

Conceivably I'm never having input that doesn't fulfil the Turner requirement, maybe in some debug scenario so it doesn't matter if it shows 0 then

lambda creature=creature: complained about Cannot infer type of lambda

[rare scarab]

Did you try doing an inline lambda?

(lambda foo: lambda: foo.bar)(foo)

Partial may also work

partial(lambda foo: foo.bar, foo)

[slender timber]

Why don't IDEs expand TypeAliases?

[fossil sorrel]

Anyone knows is it safe to just set python/typing.py TYPE_CHECKING flag to True?

[trim tangle]

It is supposed to be False at runtime

[fossil sorrel]

yeah, and sphinx is trying to document the code by running code

so any guarded types cannot be imported from pydantic with sphinx_autodoc_typehints

like ```py
WARNING: Failed guarded type import with ImportError("cannot import name 'AbstractSetIntStr' from 'pydantic._internal._utils'

dunno wtf i should do

[trim tangle]

Well, the code in a type_checking is often supposed to not work at runtime

Can you show an example maybe?

[fossil sorrel]

i mean, dunno what example i can show. here is pydantics definition: ```py
if typing.TYPE_CHECKING:
MappingIntStrAny: TypeAlias = 'typing.Mapping[int, Any] | typing.Mapping[str, Any]'
AbstractSetIntStr: TypeAlias = 'typing.AbstractSet[int] | typing.AbstractSet[str]'
from ..main import BaseModel

[trim tangle]

You could do something like

if TYPE_CHECKING:
    from foo import Bar
else:
    Bar = "Bar"

Why guard the alias definitions?

[fossil sorrel]

dunno, ask pydantic.

[trim tangle]

Oh it's from pydantic

Cursed

[fossil sorrel]

https://github.com/pydantic/pydantic/blob/main/pydantic/_internal/_utils.py#L20-L23

[rough sluice]

pydantic/_internal/_utils.py lines 20 to 23

if typing.TYPE_CHECKING:
    MappingIntStrAny: TypeAlias = 'typing.Mapping[int, Any] | typing.Mapping[str, Any]'
    AbstractSetIntStr: TypeAlias = 'typing.AbstractSet[int] | typing.AbstractSet[str]'
    from ..main import BaseModel```

[fossil sorrel]

so need to enable this flag somehow before running the sphinx.

https://github.com/tox-dev/sphinx-autodoc-typehints this library causes error by the way

[fossil sorrel]

it is all fine if i dont include type hints in sphinx, but else, i hit this flag barrier

[trim tangle]

I don't think there's a solution that doesn't modify library code

Like not putting random stuff under if TYPE_CHECKING

[trim tangle]

Or this

[fossil sorrel]

i tried ```py
from typing import TYPE_CHECKING

if TYPE_CHECKING:
from pydantic._internal._utils import AbstractSetIntStr
else:
AbstractSetIntStr = 'typing.AbstractSet[int] | typing.AbstractSet[str]'

seems like it doesnt care

[trim tangle]

Strange

[fossil sorrel]

https://github.com/tox-dev/sphinx-autodoc-typehints/issues/22 open since 2017 lmao

[soft matrix]

this is why i wrote an ext to use a type checker to resolve them

luckily for you though it doesnt work

https://github.com/microsoft/pylance-release/discussions/3822

[fossil sorrel]

wait i can mock pydantic

import mock
MOCK_MODULES = ['pydantic']
for mod_name in MOCK_MODULES:
    sys.modules[mod_name] = mock.Mock()
``` how bad is this idea

[rustic gull]

Hello, how can I add string literals as hints for a function? And how can I allow the user to create as many string literals hints he/she wants?
I have this function called thing.create_event("name", ...); and I want that when the user (on vscode) write thing.call_event(); a list of strings inserted from create_event show up.
Is this possible?

[soft matrix]

!d typing.LiteralString

[rough sluice]

typing.LiteralString

typing.LiteralString```
Special type that includes only literal strings.

Any string literal is compatible with `LiteralString`, as is another `LiteralString`. However, an object typed as just `str` is not. A string created by composing `LiteralString`-typed objects is also acceptable as a `LiteralString`.

Example...

[soft matrix]

this?

[rustic gull]

How can I bind that to a function argoument of create_event and then show up as a list to call_event ?@soft matrix

[tranquil turtle]

That's not typesafe

[rustic gull]

That allow me to specify the argoument has to be a string literal, but it does nothing else. I can't figure out how to bind it ìto show up as an hint.

[rustic gull]

I'm not talking about type checking, I'm talking about making a function that require a string and then another function show up (on vscode) that string as an hint to use.

I have this function called call_event(<name>), I want that vscode show hints about all avaiable event names that has been added from add_event(<name>).

[soft matrix]

you can't because you cant mutate generics internal args

[rustic gull]

Damn...
I found out that what I want to achieve is called dynamic typings because it add type hints across multiple function definitions from the same object.
Do python support dynamic typings or only static typings exists?

[tranquil turtle]

Dynamic typing happens at runtime, so it is not your case

[rustic gull]

*Dynamic hints
I typed it wrong. 😆

[tranquil turtle]

Type system currently doesn't support that :(

[rustic gull]

Yep, the first answer of this (https://stackoverflow.com/questions/49018605/python-dynamic-type-hinting) also confirm that. 😦

[tranquil turtle]

If you are hsing mypy, you can write plugin to correctly typecheck it.

But you are talking sbout IDE, so there is some LSP running inside, probably it is pyright. I dont think pyright supports plugins

[tranquil turtle]

I think thus wuestion is not related to your problem.
IMO, this answer is not absolutely correct. Good solution would requre usage of typevartuples and type-level Map, so it is almost solvable

[soft matrix]

i mean mutation cant really be tracked at type time unless youre aga(???) so i dont think theres every gonna be a way to do this

[trim tangle]

Agda doesn't have mutation

[rustic gull]

👍

[soft matrix]

whats the language with typed lists

[trim tangle]

Rust, Haskell, Idris, Agda

TypeScript

If you mean type-level lists

I don't know any language where mutation is tracked like that. Usually you create a new list based on the old one

[soft matrix]

oh maybe that was just a crazy assumption that they could do that if they had type-level lists

[gray bronze]

how would I type hint a list of types? like x = [str, int]

[viscid spire]

x: list[type] = [str, int]

and for reference

x: type[str] = str

[acoustic thicket]

how are you going to use the list

[gray bronze]

definitions for commands
CommandDef(kind: CommandKind, required_args: list[type])

[acoustic thicket]

[robust tangle]

Hi guys I made this site from WordPress please give feedback https://blogospace.com

[viscid spire]

no

[robust tangle]

?

[viscid spire]

this is a python server, and also this a specific channel for an aspect of python

so you are wildly off-topic

[robust tangle]

Oh

Sry my bad

[fossil nest]

Vscode generates the following type hint for my function automatically: "Any | None". Isn't the or none part redundant if the function returns anything? Also, should I use the actual "Any" typehint from the typing library or is a lack of a typehint a suitable typehint itself?

[trail kraken]

I was told Any serves as a placeholder to be replaced later. It can be used to silence warnings about missing type hints, depending on your configurations.

Until you get around to figuring a suitable type to use.

[tranquil turtle]

x: Any = ...
x.foo() # this is ok

x: Any | None = ...
x.foo() # shouldn't this be an error? Because None has no .foo method

Any is top and bottom type at the same time, so it is a bit confusing

[rare scarab]

!d typing.Never is the bottom type

[rough sluice]

typing.Never

typing.Never```
The [bottom type](https://en.wikipedia.org/wiki/Bottom_type), a type that has no members.

This can be used to define a function that should never be called, or a function that never returns...