internals-and-peps

[static bluff]

Passing in dictionaries like I am isn't very pythonic I don't think. But, the launch script is more of a configuration file than anything. I have the suspicion that a lot of application frameworks outside of Python would be using this JSON-like setup. All the actual stuff one would do, beyond configuring the program, takes arguments as usual

[native flame]

passing dictionaries as args does seem very js-y
i think its more idiomatic to pass them as kwargs

then again, thats not the only js-y thing in that code, heh

[static bluff]

And thats intentional

The framework is a pythonic implementation of Javascript's DOM API

Among other things. Its a graphics library to control an HTML/WebGraphics UI rendered in a (python powered) browser, by rebuilding the DOM API in Python as closely as possible, with any "actions" taken on the DOM or its elements transmitted to Javascript

I want it to feel like front-end web dev. Hence the camel case. The dictionary arguments are a point of consternation for me though. The only reason to do it at all is that I need to send in two potentially massive dictionaries of arguments (must be dictionaries, by nature) to serve as the config args for one of the underlying frameworks

But, since I've got those args in place already, my instincts tell me I should be consistent. The result of that consistency is what you see. But its unnatural-ness is not lost on me, which is why I'm hoping for you guys' suggestions

The other thing is, after a certain point I've just got to make a call. To me this looks quite attractive. But its mondo-important that I collect people's opinions

[native flame]

which audience is this library targetting - js devs, or python devs?

[static bluff]

Absolute beginners to programming (kids/teens who just want to get playing with graphics, fast), JS devs who's application demands go beyond what JS can do easily, and Python devs unsatisfied with other python graphics libraries

The first one is the most important

[undone hare]

https://mail.python.org/archives/list/[e-mail]/thread/D5U6MTIXMBR6BZRVMCLZAXSICWZ4VOWT/

Gosh, the suggested implementation is such bad design

tl;dr: have a str.isfloat function that returns True if it a float, False if it is an int, None otherwise

Why would you combine both in one and have an Optional[bool] as your return type

Doesn't make any sense

[spice pecan]

I can't express how heavily I dislike that idea

[native flame]

from me too

[spice pecan]

If performing that check is a common enough operation, it can easily be extracted into a free helper function, I don't see a reason to have an isfloat method at all, let alone one that also checks for ints and returns None in the general case

[undone hare]

isfloat could be useful in case of very large floating points numbers I'd be assuming?

I don't know how costly it is to perform a float conversion of, say, a 10^6 digits number

[spice pecan]

Whether or not the performance overhead is significant is a good question

[undone hare]

I guess the verification could be done through a regex

[spice pecan]

But the suggested implementation actually just runs the conversion, which kind of undermines the whole endeavor

[undone hare]

You can probably find some for IEEE standard

[native flame]

i dont think performance overhead would be significant enough to be taken into consideration

[flat gazelle]

what's wrong with just catching the exception?

[spice pecan]

Exactly

[elder blade]

The thing is.. the implementation does that 😂

[undone hare]

Well I mean this is obviously pseudo code since the string type is in C lmao

[candid pelican]

!e

print(foo := 2 + 2)
print(foo)

[fallen slate]

@candid pelican :white_check_mark: Your eval job has completed with return code 0.

001 | 4
002 | 4

[candid pelican]

it works

but it's useless

[charred pilot]

this isn't on topic for this channel, try #python-discussion

[formal warren]

oh k sry

[verbal escarp]

that strongly depends on the context and what the alternatives would be

i'm finding passing **kwargs is more useful in more contexts than positional args, my go-to pattern these days is a single positional-only and the rest kwargs

[static bluff]

Same

[verbal escarp]

there's one alternative that you could try

[static bluff]

open is a great example — taking a path or filename is a given, of course open is going to do that. But naming the arguments beyond that is just a kindness we do for ourselves and our linters 😄

[verbal escarp]

it's an actual (data-)class specific to passing messages around like in a conveyor belt

django uses that iirc

i've worked with a message-class before myself, but coming from my experience with the buffet table, i'd prefer named kwargs these days, i think

there are a few benefits to using named kwargs over a message class - less typing for one

also, you can nicely annotate your args and it's easier for an IDE to give useful assistance

also it's less work to change things if you need to refactor stuff

[paper echo]

A Config or CurrentContext class isn't unusual

[static bluff]

So do you have any thoughts?

[verbal escarp]

well, as i said, it depends on the context which pattern fits best, but i wouldn't say passing around dicts as kwargs isn't per se non-pythonic

[static bluff]

🙂

[verbal escarp]

just keep in mind there are alternatives that might fit better

[static bluff]

Such as using kwargs

And/or breaking it into smaller classes

[verbal escarp]

apropos signatures, i invented another pattern for justuse i quite like for simplifying boolean parameters

i thought of using enum.Flags but they work quite well enough for this

basically, you define enums like

class Use(ModuleType):
    # MODES to reduce signature complexity
    # enum.Flag wasn't viable, but this alternative is actually pretty cool
    auto_install = 2 ** 0
    fatal_exceptions = 2 ** 1
    reloading = 2 ** 2
    aspectize_dunders = 2 ** 3
    no_public_installation = 2 ** 4

and wherever we need boolean flags, we have a single mode parameter

which is called like mode=use.auto_install | use.fatal_exceptions etc

and to extract, we do ```python
fatal_exceptions = bool(Use.fatal_exceptions & modes)
auto_install = bool(Use.auto_install & modes)

which is much cleaner than having 5 or more named parameters and less typing when calling it

[prime estuary]

Enum flags are a little better, because it stops you passing in random numbers, and you can use auto() instead of having to manually pick powers of two.

Though for boolean parameters, another thing you could think about doing is instead splitting it into multiple functions - it could be a sign this one is doing too much.

[verbal escarp]

not sure what the reason was but it didn't work as well as i hoped with enum Flags, so i fell back on global constants

something with the boolean operators i think was the problem

[prime estuary]

Well, there's both Flags and IntFlags, which act a little differently - the latter is an int subclass and allows all int operations, the former only implements more sensible operations - you can't or in arbitrary bits , only the defined ones, it doesn't compare equal to regular ints, etc.

[swift imp]

Can you add descriptors to ModuleType? That way you can have variables with docstrings?

[elder blade]

Wait why not 1 << 0 instead of 2 ** 0?

[verbal escarp]

because 2** 0; 2** 1, 2** 2, .. looks nice ^^

[paper echo]

bit fields are cool, but i think you're at least half a century late to "invent" them 🙃

[verbal escarp]

i only invented them as pattern to simplify boolean parameters, not in general 😉

[paper echo]

that's what they are though

[verbal escarp]

of course i've known about bitfields

[paper echo]

a byte is effectively an array of 8 bools

[verbal escarp]

but i never used them to simplify signatures

also never seen them used for that purpose

apropos, a while ago i tried to use actual bitfields for something else.. it's much harder to use them elegantly in python than i thought

[paper echo]

fwiw i'm not sure how this is simpler than

@dataclass
class UseModes:
    auto_install: bool = False
    fatal_exceptions: bool = False
    reloading: bool = False
    aspectize_dunders: bool = False
    no_public_installation: bool = False

        fatal_exceptions = modes.fatal_exceptions
        auto_install = modes.auto_install

(or whatever the defaults are)

[verbal escarp]

enum flags suck for that purpose, bitarray sucks.. i think i had to resort to numpy to accomplish what i wanted without jumping through hoops

[paper echo]

you could use an enum of mode names, and implement your own __and__, __lshift__, etc. methods if you really wanted to write modes = modes & Mode.auto_install

[verbal escarp]

yeah, that's what i call jumping through hoops ^^

[deft pagoda]

i 've used intflag enums with | stuff already

[paper echo]

yeah fair it's hoop-ey

[deft pagoda]

enum actually makes a nice repr for it

[paper echo]

https://docs.python.org/3/library/enum.html#enum.IntFlag yeah this seems like exactly what you'd need

[deft pagoda]

In [95]: from enum import IntFlag
    ...: 
    ...: class MyFlag(IntFlag):
    ...:     foo = 1
    ...:     bar = 2
    ...:     baz = 4
    ...: 

In [96]: MyFlag.foo | MyFlag.bar
Out[96]: <MyFlag.bar|foo: 3>

cool repr!

[marsh tiger]

any ideas?

[raven ridge]

That's off topic for this channel. We don't have a channel that that's a good fit for, though #user-interfaces or #game-development might come closest (at least those both relate to images)

[marsh tiger]

Oh ok gotcha, thanks

[quasi hound]

you know what would be nice? *args comprehensions

def example(*args):
  pass

# old boring way
example(*[x for x in iter])

# cool way
example(x for x in iter)

and **kwargs why not
example(**k: v for ... in ...)

that one looks a tiny bit more confusing (looks like **k) but what do you guys think

but the first way is definitely clear and clean

[gleaming rover]

how would you distinguish this from cases where you actually want the generator expression itself

[quasi hound]

extra layer of parens

example((x for x in args)):

you mean what if you want to pass in a genexp? ^

because i don't think a function call will be confused with a genexp

it's easy, just check if there's an identifier in front with no spaces separating them

still think this is a fine idea

[gleaming rover]

yup

[quasi hound]

then just use more parens

[gleaming rover]

would be backwards incompatible, too

[quasi hound]

wdym

oh

[gleaming rover]

shrugs no opinion on this tbh because I haven’t had this problem before

[quasi hound]

i could've used that thing to clean up my code a tiny bit a few minutes ago

but yeah it's nbd

edited Grammar/python.gram
bad idea

just redownload it

it's definitely this part

    | a=(await_primary | prefix_crement) '**' b=factor { _PyAST_BinOp(a, Pow, b, EXTRA) }
    | prefix_crement
    | await_primary```

what are you even trying to do

unless you edit the code and not just the grammar you prob aren't gonna do much

i don't even know how grammar files work sorry

[raven ridge]

I don't think it's got anything to do with that.

Unless... Should prefix_crement end with await_primary instead of primary?

What case did you break, defining a function that accepts **kwargs or passing unpacking **kwargs to a function in a call?

The power expression shouldn't be used for either of those, so if something you're showing broke **kwargs, it seems like it must be because something is incorrectly being detected as a power expression when it used to not be

[raven ridge]

Hm. I'm not able to spot the bug, though.

[terse orchid]

app/
├── .gitignore
├── .venv
├── packageA/
│   ├── __init__.py
│   └── moduleA  # imports moduleB
├── packageB/
│   ├── __init__.py
│   └── moduleB
└── tests

Hello. Is it a bad practice a module imports something from another package in the same level? Or even an upper level?

Here, moduleB is being imported in moduleA. Whenever I do this bad things happen: is hard to test, hard to run the moduleA standalone etc. Does this mean "if a module needs something from another package, they probably should be in the same level. Don't make them packages, just restructure them as modules"?

Or should I duplicate the code I need in moduleB to moduleA, but it's very open to mistakes. Duplicate code is a bad thing as I know.

[regal tide]

Hey all - thoughts on using a callable variable alias (eg for composition purposes) so in init you link a class method to a attribute object method ?

[verbal escarp]

example?

[regal tide]

On my mobile so can’t so can’t add code example but similar question to this https://stackoverflow.com/questions/60584948/pythonic-way-of-doing-composition-aliases

As functions/methods that get wrapped by composition can lead to a hundred line implementation with type annos in repos

I’m struggling to see the disadvantages of using the composition alias

[verbal escarp]

i think that depends on what you actually want to achieve

pattern have a purpose, they don't exist in the void

that SO example reminds me of a visitor pattern, but also facade, delegation or proxy

in python you could go as far as implementing it as a direct proxy, "augmenting" the arguments

here's an example how to build a transparent proxy class that we have in justuse: ```python
class ProxyModule(ModuleType):
def init(self, mod):
self.__implementation = mod
self.__condition = threading.RLock()

def __getattribute__(self, name):
    if name in (
        "_ProxyModule__implementation",
        "_ProxyModule__condition",
        "",
        "__class__",
        "__metaclass__",
        "__instancecheck__",
    ):
        return object.__getattribute__(self, name)
    with self.__condition:
        return getattr(self.__implementation, name)

def __setattr__(self, name, value):
    if name in (
        "_ProxyModule__implementation",
        "_ProxyModule__condition",
    ):
        object.__setattr__(self, name, value)
        return
    with self.__condition:
        setattr(self.__implementation, name, value)

you could do something similar and simply forward calls to matching methods or (rather) functions

[bronze briar]

Hi i need some urgent help please

[gleaming rover]

#❓｜how-to-get-help

[regal tide]

What exactly does this add for the additional complexity ?

[verbal escarp]

it's pretty much constant, depending on the scenario

independent from the size of the class you're shadowing

the more special cases you have, the more you need to add to the proxy of course

[regal tide]

The place I’m thinking of is in fastapi, there’s a fair few places that just proxy other calls eg https://github.com/tiangolo/fastapi/blob/master/fastapi/applications.py line 212

[verbal escarp]

yeah, proxies are one of the most useful patterns imo

[regal tide]

I just can’t see the disadvantages (other than being explicit which is the pythonic approach)

Explicit is better than implicit

[verbal escarp]

there are very few disadvantages

it might make serialisation harder (it would require to "unwrap")

[regal tide]

You get all the auto completion etc for free

[verbal escarp]

yep

of course, there's a performance penalty

[regal tide]

Is it that much different than proxying explicitly though?

I haven’t looked into the assembly code using dis

[verbal escarp]

depending on your application and how much time the machine needs to spend to go throw the layers

[regal tide]

I think by that point you shouldn’t be using python anyway 🤣

[verbal escarp]

not really, but it's tempting to wrap the wrapper and go on wrapping 😄

if you have a complex underlying framework like a game engine or database, you might get the idea that "just one more layer of abstraction" might make your whole thing agnostic to whatever framework is used

then you end up sacrificing functionality and performance for a perceived option in the future to be able to switch stuff

proxies make that idea very tempting

[regal tide]

Yeah, I feel like the only downside to implicit proxies vs explicit is that you don’t know that it’s a callable that you’re assigning - though I guess with type hinting you could give some indication. I feel like that’s better than 100 line explicit proxy though

Surely performance-wise it’s the same

[verbal escarp]

well, there is one difference though

with explicit forwarding you've got the option to turn things into properties and cache things, for instance

caching can have a huge performance impact

much bigger than the penalty of the proxy

[regal tide]

Can you not achieve that with your proxy wrapper though?

[verbal escarp]

yes and no. i could filter out selected calls and reroute them

[regal tide]

Just add lru_cache decorator ?

[verbal escarp]

basically a hybrid

[verbal escarp]

"just" 😉

you might be dealing with a c-extension

or some other code you don't want to directly mess around with

[regal tide]

Depends on usecase I guess - having a fully generalised solution would be difficult and would be more useful in the code python library

[verbal escarp]

yeah

but never underestimate properties in python, their usefulness is much bigger than what their uglyness suggest

i'd say they're in the same category of ugly as lambda

[regal tide]

I love a good lambda

[verbal escarp]

don't we all - but the syntax is ugly 😉

[regal tide]

Yeah I do prefer JavaScript’s anonymous functions

[verbal escarp]

one thing about properties i found really cool some time ago was that i was able to store all attribute information globally in numpy arrays and have extremely lightweight classes - only an id and the rest properties, pointing to their respective dict/np.array

it was a bit of a hassle at first to set everything up, but it's so nice to be able to do simple attribute lookups for basic cases AND numpy vectorization for complex stuff, all with a single source of truth

on the other hand it can be a bit dangerous if you think "it's just a simple attribute lookup" but behind the scene there's a db access with JOINs etc

[regal tide]

Yeah, abstraction of any kind can hide a lot of magic. Pros and cons of it

[verbal escarp]

(but that's what lru_caches are for, right?)

😉

[regal tide]

You working on anything interesting at the moment?

[verbal escarp]

a public thing i've been working on is https://github.com/amogorkon/justuse

[regal tide]

I’m still in lockdown in Melbourne and between jobs so looking for some cool things to work on. Might take a look around the issues see if I can contribute anything

[verbal escarp]

ahh 🙂 well, @broken furnace and i would love another helping hand

[regal tide]

I’ll take a look around 👌🏼 I’ll join your slack channel to chat on there?

[verbal escarp]

you might need a guide through the code, though.. it's quite advanced - i wouldn't say it's a mess at this point, but it might be unwieldy for the uninitiated

[verbal escarp]

sure

[true ridge]

What do you mean by how it can be displayed? AST identifier type corresponds to str in Python, you should be able to print it.

If you need to do it in the C level, you can use PyObject_Print

[elder blade]

Isn't that the point of your changes?

I thought you changed the Python source to disallow usage of **kwargs?

[surreal sun]

This might not even be a reason at all, it’s just a wild guess, but maybe it interprets it as a power because you messed around with the functions that are part of the recursive descent parsing?

[surreal sun]

Ah

It probably thinks you’re doing a double unary op?

Oh wait

Double multiplication

Because ** isn’t even parser

parsed

It is probably double multiplication come to think of it

I wonder what’ll happen if you take out the prefix cremenr

[paper echo]

Is it possible to introspect on and manipulate the currently executing bytecode? I'm wondering if you can implement something like resumable exceptions with some deep cpython bytecode fuckery

https://pycon.org.il/2016/static/sessions/tom-zickel.pdf

My motivation was to insert debug code to live running process without putting pre-existing code to debug them.

Nice

[sharp flume]

yes ive done it before but i honestly cant remember how

i was in a deep alcohol state of cpython fuckery

[verbal escarp]

lmao

we should rename this channel

[surreal sun]

I've messed up bytecode before to the point where I got a long hexadecimal number as an exit code

It was not live running though

I monkey-patched a function

[verbal escarp]

or rather #balmer-peak

[paper echo]

chilaxan at one point had helped me write a tracer for imports, which I do still intend to use for a hot reloader that can follow rebound names

So I know it's possible to inspect the bike code as it's running, but I don't know how to do something like "save the entire call stack and resume instead of unwinding to throw an exception"

[verbal escarp]

we've solved that issue in justuse with the ProxyModule 😉

way less hacky

[verbal escarp]

reminds me of the "omniscient debugger" in thonny, sort of

but i don't know if thonny could rewind after an exception is thrown.. that would be pretty awesome if that worked

[livid dove]

check hellscape im pretty sure you sent it there

[sharp flume]

i didnt

[livid dove]

hmm

[sharp flume]

ive never sent it anywhere

[livid dove]

pretty sure you sent me smt with bytecode fuckery somewhere

could be wrong tho

[sharp flume]

oh yeah ive sent mild bytecode fuckery

but not modifying running bytecode

[livid dove]

¯_(ツ)_/¯

[paper echo]

Time travel debugging is definitely a thing, I'm talking about actually being able to install some kind of resuming handler

Like adding to limited continuations to python

[paper echo]

Yeah it's a little easier when you don't have to trace the bytecode generated by import

But even then, you do still need to figure out when an attribute of your module is assigned to some other name in the currently executing module

[white oxide]

can i ask u a help?

is it legal?

[paper echo]

You should open a help channel, see #❓｜how-to-get-help

[verbal escarp]

is it an advanced topic?

[white oxide]

idk

[verbal escarp]

just ask your question and we'll see

[white oxide]

its like importing codes from scratch

so

idk

but

idk its legal

is it?

im sorry if its illegal

[verbal escarp]

asking questions isn't illegal

[white oxide]

likr

[verbal escarp]

depends on the topic

[white oxide]

can u translate a scratch project I made

[verbal escarp]

no

[white oxide]

ok

just poppin to see if u let me

so

if u can

DM me

[verbal escarp]

you can ask in one of the help channels 🙂

[white oxide]

ok....

:3

[peak spoke]

In what situations can sys.excepthook receive None as the traceback? typeshed doesn't have it hinted as optional and I can' find much online

[pliant tusk]

you could in theory get a reference to the frame that raised the exception, reset its f_lineno, and set some flags to put it in a running state again, then jam it into a generator object. Then calling next(fake_gen) should continue running that frame

[paper echo]

That's surprisingly elegant

[pliant tusk]

I'll try to write a demo later tonight

You would also probably have to recursively restart all of the upper frames

[peak spoke]

Managed to reproduce it with this, I guess it happens when the exception is created out of python?

import sys
from functools import partial
from PySide6 import QtCore, QtWidgets

def gen():
    raise ValueError
    yield
app = QtWidgets.QApplication([])
g = gen()
try: next(g)
except ValueError: pass
QtCore.QTimer.singleShot(1, partial(next, g))

sys.exit(app.exec())

[paper echo]

You're a hero

Could you do stuff like rewrite the bytecode of a function to perform tail recursion elimination

[pliant tusk]

Yea you could

[verbal escarp]

yesss

do it! dooo ittt!

[paper echo]

Hah cool

I wouldn't know how tbh

[unkempt rock]

How would you create or what is it called when you do something like

with open(params) as f:
  f.foo

what is that called? how would i create my own custom class in which i can do that

I can't really google it because I don't know the name or how its done or what its called

[charred pilot]

it's called a context manager, your class needs to implement __enter__ and __exit__

[unkempt rock]

Thanks, much appreciated

[spice pecan]

You could also use the contextlib.contextmanager decorator for similar effect if a class is a bit overkill for your desired functionality

!d contextlib.contextmanager

[fallen slate]

contextlib.contextmanager

@contextlib.contextmanager```
This function is a [decorator](https://docs.python.org/3.10/glossary.html#term-decorator) that can be used to define a factory function for [`with`](https://docs.python.org/3.10/reference/compound_stmts.html#with) statement context managers, without needing to create a class or separate `__enter__()` and `__exit__()` methods.

While many objects natively support use in with statements, sometimes a resource needs to be managed that isn’t a context manager in its own right, and doesn’t implement a `close()` method for use with `contextlib.closing`

An abstract example would be the following to ensure correct resource management:

[verbal cipher]

Guys any recommendations on a final year Project

[lusty scroll]

can you write to f_lineno?

[pliant tusk]

you can write to anything with ctypes

[lusty scroll]

it will actually move the execution?

i thought that's possible when tracing but didn't know it works during regular execution

[deep bramble]

What would be the proper way to type-hint something like this?
I've tried using type as a type-hint for both, however pyright doesn't like that and ends with an error: Expected class as first argument to "super" call but received "type". Problem is, I wasn't able to find any type hint for a class in the typing module. Does it actually contain something that would fit this and solve the error?

def foo(owner_cls, instance):
    ...
    super_instance = super(owner_cls, instance)
    ...

[raven ridge]

you're looking for typing.Type[object] or something like that. object is the base class you require, so if you know you want only subclasses of a particular base class, you would do typing.Type[SuperClass]

[spice pecan]

Depending on the version, type[object] should also work I suppose

[deep bramble]

yeah, I'm on 3.9

[raven ridge]

basically, whatever type you're annotating instance as, owner_cls should be annotated as Type[that_type]

[deep bramble]

that makes sense, I didn't know type could be used for type hinting like this, this is very interesting

[prime estuary]

You probably should use a TypeVar actually, so multiple types can be passed and the checker knows they must match.

[raven ridge]

what do you mean by "multiple types"? There's only 2 arguments to this function, a class and an instance of that class (or a subclass of that class), right?

[spice pecan]

Depends a lot on what sort of types are going to be passed, but making it generic could be worth it

def foo(owner_cls: type[T], instance: T): ...

If I understood correctly

[raven ridge]

oh, if the instance really can be anything, then yeah, that makes sense.

[deep bramble]

yeah, that's probably better, I'm not very used to using typevars, is it just T = typing.TypeVar() for this?

[spice pecan]

Roughly, T = typing.TypeVar('T') I believe

!d typing.TypeVar

[fallen slate]

typing.TypeVar

class typing.TypeVar```
Type variable.

Usage:

```py
T = TypeVar('T')  # Can be anything
A = TypeVar('A', str, bytes)  # Must be str or bytes
```  Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well as for generic function definitions. See [`Generic`](https://docs.python.org/3.10/library/typing.html#typing.Generic "typing.Generic") for more information on generic types. Generic functions work as follows...

[spice pecan]

yeah

[deep bramble]

why does it need the 'T' as a first argument?

[spice pecan]

Probably for reprs

[prime estuary]

Yep, just for the repr.

[raven ridge]

yeah, so that the type knows its name.

[spice pecan]

Because it can't know that you assigned it to the name T

Well it can, but not without black magic

[deep bramble]

that makes sense then

thanks for your help!

[raven ridge]

which, interestingly... if this were a class variable or instance variable it could, through the descriptor protocol. Seems like there would be a use for generalizing the descriptor protocol to module globals...

[deep bramble]

yeah, I was thinking about this too after your response to that, but I suppose it may sometimes be useful to have it named differently then the actual variable

[spice pecan]

Yeah, that's also a good point

But at the same time, I guess it could be an overloaded form

By default, with no argument, it could infer the name it's assigned to, and if you do specify a name for it, it could use that name

Though I don't think it's that big of a deal

[deep bramble]

I agree, that would make it a bit less annoying, this is probably a similar case with initializations of namedtuples

[raven ridge]

there's plenty of other places in the stdlib where you need to tell an object the name you've assigned it to - namedtuple, the alternative constructor for dataclass, and TypeVar are the 3 I can think of off the top of my head, but I know there's more than that.

oh, the 3-argument form of type for another

[lusty scroll]

and corollary to that is the weirdness of the namedtuple package name for the types

I've had weird names show up

i think if you reassign __name__ it will change next time you create a namedtuple type

would be nice if it could just use the unqualified name you pass in

like builtins don't have 'builtins.list' etc

[verbal escarp]

hey @lusty scroll i wonder whether we should put "isolation of packages via subprocesses" on our list of goals as discussed here the other day..

[lusty scroll]

possibly?

[verbal escarp]

i'm not going to do it, though 😉

[pliant tusk]

generators are very funky to hack arbitrary frames into

this may not work until i do more research

[unkempt rock]

@lusty scroll hi grey

[boreal umbra]

how would you feel about a dunder method that receives the name of an object after each assignment expression? (it being ones job to decide if something should be done each time its called)

[lusty scroll]

hey, Andy, that's you?

howdy

[boreal umbra]

def __assign__(self, name: str):
    self.__name__ = name if hasattr(self, '__name__') else self.__name__

something like that

[lusty scroll]

i think there could be some limited value

[verbal escarp]

reloading of packages, clean separation..

possibly using optimized python variants for different packages..

[spice pecan]

I think that would be a good idea

It would remove a bunch of repetition from code

We can use dirty hacks to get similar effects currently, but they're just that, dirty hacks

[boreal umbra]

what would this be but a dirty hack with dunder methods?

[spice pecan]

An officially supported and reliable hack that doesn't depend on reflection

[surreal sun]

CC: @unkempt rock

I think you wanted ideas for what you want to mess with in CPython code -- maybe you could add this

[raven ridge]

that's what __set_name__ is, it's just not called for every assignment, only ones where the object is assigned to a class dictionary

[raven ridge]

Have you seen the work that's happening in CPython to allow running multiple subinterpreters in a single Python process?

[lusty scroll]

I have seen some bits and pieces of it

seems like it would fit better than subprocess for isolation

the only question would be communication

and fixing extension issues

[elder blade]

Didn't you guys talk about subinterpreters before?

[verbal escarp]

we did

[elder blade]

Oh lmao, too slow again ;-;

[verbal escarp]

^^

@paper echo man.. the idea of having explicit package and module names was so nice

use(package_name="matplotlib", module_name="matplotlib.pyplot")

bah 😐

would it make sense to check if package_name exists as directory and then put it before module_name?

in the attempt to make things simpler for users, everything got so convoluted 😦

sadly, i'm to be blamed myself for that same thing - for justuse the call would be something like use(package_name="justuse", module_name="justuse.use") i guess

[paper echo]

Wdym?

What's the API currently?

[verbal escarp]

you suggested use((foo, bar)), use("foo/bar") and use(package_name="foo", module_name="bar")

i implemented those, but noticed that python regards the root directory as part of the module name

which often, but not always, corresponds to the package name

[paper echo]

Huh... what do you mean by "root"?

[verbal escarp]

the directory where the module lives

[paper echo]

Implicit namespace packages strike again?

[verbal escarp]

matplotlib/pyplot.py

would be the path

[paper echo]

Oh hm

Wait, I suggested the / for the distribution name?

And not something like ; that isn't already taken?

[verbal escarp]

i did a little survey here about the best separator

i suggested : or / instead of .

[paper echo]

Ah right

Well i realize now that is kinda difficult to distinguish from a file path...

[verbal escarp]

settled on / because it matches the path semantics

[paper echo]

But use works on filenames too though right? Did i make a really bad suggestion?

[verbal escarp]

no, no.. not your fault

the problem is not with / or any other sep

it's that python the "package" also can refer to the directory where the module lives.. and if you ask python what the name of "pyplot" is

it says "matplotlib/pyplot"

so if you want to be explicit, you need to type "matplotlib" twice

basically corresponding to what you tell pip to install and the second time what to import

[paper echo]

Make it so that matplotlib.pyplot implies matplotlib/matplotlib.pyplot unless package= is also provided explicitly

[verbal escarp]

might be a solution, yeah

[paper echo]

Is it too late to change it to : or , or something else that doesn't clash with a filesystem path separator?

I regret voting for / if that is how I voted

[verbal escarp]

no, but there's not really a point in changing the sep

the problem is that implicit and explicit clash 😐

[paper echo]

You can't write use("foo/bar.py") then

Unless you require file= for using files by path

[paper echo]

Yeah double typing the name sucks

[verbal escarp]

i think using a hybrid approach like you suggested makes the most sense, but then people will complain that it's not consistent :p

can't really make everyone happy

[paper echo]

I think it's consistent! If you omit the distribution name, It assumes that the distribution has the same name as the top level module/package

[verbal escarp]

hm

[paper echo]

The issue is if you want a module that's on your local file system

So you don't always wanted to make that inference, but maybe then you need local=True

[verbal escarp]

no, no, we got that covered

[paper echo]

Or set the package to . for local

[verbal escarp]

if you only want a single module, you always could use Path directly

[paper echo]

Ah

[verbal escarp]

that was the first thing i implemented, ever in justuse

[paper echo]

I'm not a fan but it's also not my library 😛

[verbal escarp]

dispatch to Path, URL and str

well, it makes things nice and simple if you want to quickly import some local module from another project without having to do the package dance

[paper echo]

Could have abused URI syntax! pypi:some-package.name/some.module

[verbal escarp]

or having to copy code

[verbal escarp]

wellll... let's come back to that debate with version 2 😉

apropos URI, is there maybe an elegant syntax that might fit with package name/module name?

something that takes care of potential redundancy

[verbal escarp]

i really like that syntax, although i'm not sure what the benefit would be to have a "protocol" specifier for that purpose

[paper echo]

Maybe just omit the "authority" pypi:/matplotlib.pyplot

[verbal escarp]

hm 🙂

[paper echo]

Although this raises the question of identifier vs locator

That's really a locator

It doesn't have to be pypi

[verbal escarp]

github?

[paper echo]

Any git remote, any http package index

git: is fine because it's a "scheme" i.e. a mechanism for identifying the thing

[verbal escarp]

i'm a bit afraid of security leaks that way, though

[paper echo]

But even then it's more like an abused url

[paper echo]

I mean in the sense of --index-url and --extra-index-url

[verbal escarp]

i'll have to meditate over that

there was an internal discussion on supporting private repos earlier, maybe a protocol specifier like "local:" might work for that purpose

on the other hand, i'd prefer not to hard-code those points of reference if possible

[lusty scroll]

pip also supports a local tag for artifacts but it seems messy

[paper echo]

what if you had this:

# Get a module from PyPI with Pip
use(distribution='any-valid-pip-specifier', module='foo')

# Get a module from PyPI with Pip, with extra options like --index-url
use(
    distribution='any-valid-pip-specifier', module='foo',
    pip_options=[...],
)

# Get a module only if it's already in sys.path, no remote fetching
use(distribution=None, module='foo')

# Get a module by verbatim filename, converting the filename tail to
# a module name and bypassing or adjusting sys.path as needed
use(file='foo.py')

and the kwarg-less ones would only cover the most common use cases, which is pretty much what you have now

[quasi hound]

what does @ mean in this context

AnyStr that was passed into the compile method (this is re.compile documentation)? similar concept to a decorator?

[surreal sun]

That looks like the matrix multiplication infix operator @quasi hound

!d object.matmul

[fallen slate]

object.\_\_matmul\_\_

object.__matmul__(self, other)``````py

object.__truediv__(self, other)``````py

object.__floordiv__(self, other)```
These methods are called to implement the binary arithmetic operations (`+`, `-`, `*`, `@`, `/`, `//`, `%`, [`divmod()`](https://docs.python.org/3.10/library/functions.html#divmod "divmod"), [`pow()`](https://docs.python.org/3.10/library/functions.html#pow "pow"), `**`, `<<`, `>>`, `&`, `^`, `|`). For instance, to evaluate the expression `x + y`, where *x* is an instance of a class that has an [`__add__()`](https://docs.python.org/3.10/reference/datamodel.html#object.__add__ "object.__add__") method, `x.__add__(y)` is called. The [`__divmod__()`](https://docs.python.org/3.10/reference/datamodel.html#object.__divmod__ "object.__divmod__") method should be the equivalent to using [`__floordiv__()`](https://docs.python.org/3.10/reference/datamodel.html#object.__floordiv__ "object.__floordiv__") and [`__mod__()`](https://docs.python.org/3.10/reference/datamodel.html#object.__mod__ "object.__mod__"); it should not be related to [`__truediv__()`](https://docs.python.org/3.10/reference/datamodel.html#object.__truediv__ "object.__truediv__"). Note that [`__pow__()`](https://docs.python.org/3.10/reference/datamodel.html#object.__pow__ "object.__pow__") should be defined to accept an optional third argument if the ternary version of the built-in [`pow()`](https://docs.python.org/3.10/library/functions.html#pow "pow") function is to be supported.

If one of those methods does not support the operation with the supplied arguments, it should return `NotImplemented`.

[quasi hound]

yeah but that doesn't make sense here does it

[surreal sun]

I don’t see any other way it could be applied

Without a syntax errro

Besides the infix matmul op

[quasi hound]

here it is again

[paper echo]

pyright uses this for some shorthand notation

[surreal sun]

Ohh, Nevermind not what I was thinking

Sorry for the confusion

[quasi hound]

for what?

[paper echo]

i forget what exactly it means, i think it's telling you the function where the type was refined

i don't use pyright much, let me try to find a more precise definition

[quasi hound]

k thx

[paper echo]

it's pyright's syntax for specifying in which context to take the type var from
according to fix error (one of our moderators)

so Pattern[AnyStr@compile] is saying that the AnyStr type variable is the one listed in the signature for compile()

[quasi hound]

makes sense

ty

[sharp chasm]

Can we use argparse in discord.py bot?
i have asked detailed q here, (plz let me know if i shall post all of it here, too big)
https://stackoverflow.com/questions/69421071/how-to-use-argparse-with-discord-py-bot

[paper echo]

answer is "yes", but this belongs in a help channel. see #❓｜how-to-get-help

[lusty scroll]

I was checking import times with -X importtime and see that a apport_python_hook module gets imported. Searching my entire /usr doesn't turn up anything though.

And encidings took 107+ ms to import on cold startup..

Too bad LC_ALL=C doesn't help.. 😅

anyone else want to report their slowest module?

Found the rogue import in '~/.local/lib/python3.9/sitecustomize.py`

[quasi hound]

so i'm making a text adventure and i have this method which i pass an item name and a message type into, and it returns the proper message for the item

should i use an enum?

or is this fine

just a general thing

there will be many more items when i'm done but only like 6-7 message types

throughout

[surreal sun]

Pattern matching might be able to make it look nice and there’s only ~2 days left till 3.10 releases

[quasi hound]

maybe

it won't be in beta?

actually yeah i wanna use that

[surreal sun]

I think you could either use that or mapping to a dictionary

[quasi hound]

i did that here - this is the similar method i have for the rooms within the game

similar but a lot longer

[main lynx]

i would do that with dictionaries absolutely, no need for pattern matching

[quasi hound]

kk

[main lynx]

if it's just string -> value

[quasi hound]

it's a little more complicated than that

notice the +=s

im trying to update the message based on the item's attributes (like whether a match has been lit or not)

so i might use pattern matching

ty

the previous version of this game i made, this one method was 1000+ lines long

so anything that can cut that down will be nice

[main lynx]

e.g. s += items["Dull Rock"]["onInspect"]

[surreal sun]

I’m not sure what an example would be, just saw a new Dunder method and thought you’d be interested, you might wanna ask sterlecus on that @unkempt rock

[gleaming rover]

wrong data model IMO

also wrong channel

but if you wanna discuss this ping me in a help channel or #software-architecture

[main lynx]

true ECS is pretty solid for that

[jagged portal]

So I am learning Deep learning

https://www.youtube.com/watch?v=omz_NdFgWyU&list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3&index=6 by Sendex but I still don't know what will I be able to create after this session

[devout locust]

hey i am going to build the video chat website using django WEB-RTC with django channels please give some docs or any suggestion is it possible to do this ??

[paper echo]

You can ask questions like this in #web-development

[surreal sun]

Does defaultdict allow custom classes? If not, what is the reason behind it? I'm surprised there hasn't been much discussion in the Python world regarding allowing custom classes

[surreal sun]

It seems like it allows insertion of a custom class but it would be cool if you could set some sort of attribute for the defaultdict to read from for custom classes

[boreal umbra]

you don't pass a class to defaultdict per se, you pass a callable.

[surreal sun]

Ah

[boreal umbra]

!e

called_list = list()
print(called_list)

[fallen slate]

@boreal umbra :white_check_mark: Your eval job has completed with return code 0.

[]

[boreal umbra]

calling the list class like a function returns an empty list.

[surreal sun]

Ah, so you could implement some sort of default with a custom class using that

[boreal umbra]

yes. if the __init__ method for the custom class takes no arguments, you can simply give the name of the class. Otherwise you can use a lambda and fill the arguments accordingly.

!e

from collections import defaultdict

class A: pass

dd = defaultdict(A)
print(dd['foo'])
print(dd)

[fallen slate]

@boreal umbra :white_check_mark: Your eval job has completed with return code 0.

001 | <__main__.A object at 0x7f82040eef10>
002 | defaultdict(<class '__main__.A'>, {'foo': <__main__.A object at 0x7f82040eef10>})

[surreal sun]

!e

from collections import defaultdict

class Foo:
  def __init__(self, *args, **kwargs):
    if not args and not kwargs:
      return "testing"

test = defaultdict(Foo)
print(test["foo"])

[fallen slate]

@surreal sun :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 9, in <module>
003 | TypeError: __init__() should return None, not 'str'

[boreal umbra]

you don't return from an init method.

[surreal sun]

Ah yea, nvm

[boreal umbra]

There's also my superior version of defaultdict: https://github.com/swfarnsworth/dynamicdict

[astral gazelle]

Even if you didnt have a return in the init there it would still error

When defaultdict tries to create default values it would give an error because its not passing anything into the init

You could pass in a lambda however

defaultdict(lambda: Foo(*args, **kwargs))

[static bluff]

Would you guys mind checking my logic on this?

https://hastebin.com/fufevukigo.properties

The part with the file-structure is what I'm concerned about

[lusty scroll]

nothing to do with correctness, but Path.mkdir has an exist_ok kwarg you could use

oh nvm you use that

[static bluff]

🙂 Thanks! I feel like catching the error specifically is a bit more explicit to the eye than passing the exists_ok flag. Its just aethstetics, but I hear you

[brave badger]

Not necessarily logic, but structure; if you have multiple global variables like that and you're setting them through some function, I suggest having some local singleton object that holds all of those constants instead.

Globals are difficult to reason with in terms of state especially once you start unit testing code that relies on them, especially seemingly volatile ones like these.

[lusty scroll]

maybe check they didn't provide the current directory to get wiped out

[static bluff]

I, as a general rule, I endorse the arguments against using global variables. That said, global state does exist and has to go somewhere, that somewhere being this module (at this point). Can you explain the logic behind storing it all in an object as opposed to in a 'globals' module?

[static bluff]

Sorry, what do you mean by this?

[lusty scroll]

just that the logDirectory could be supplied by the user and they could have given "" or something that would mean shutil.rmtree(logsDirectory, ignore_errors=True) would try to remove the working directory

[static bluff]

Oh shit

They could supply any number of dangerous locations, now that you mention it

[lusty scroll]

that's bitten me before

[brave badger]

Two (slightly related) reasons, imports are weird and globals are hard to mock in tests. Suppose that in one file, you do an unqualified import:

# unqualified.py
from environment import APPLICATION_SUPPORT
```and in another, you do a qualified one:
```py
# qualified.py
import environment
```If for some reason, the `configureFileStructure` function is called again, only the qualified import will be able to read the new value

[static bluff]

Thats a good gatch

I mean, that's something I feel is more a responsibility of the user to know about, but, its still a potential screw up

All these values are meant to be internal, but you never know 😐

Those adventurous pythonistas

God I wish Python had privacy

[spice pecan]

Python's version of privacy is a leading underscore

[brave badger]

If everything lives under some implicit singleton object:

from pathlib import Path

@dataclass
class _Environment:
    APPLICATION_SUPPORT: Path
    EXTERNAL_ROOT: Path

    @classmethod
    def initialize(cls):
        return cls(..., ...)

Environment = _Environment.initialize()
```You can import `Environment` everywhere and any sort of updates will stick

[static bluff]

Geee, I wish I'd known that

[spice pecan]

Sure, they can technically access that, but then it's their fault if something breaks. You've removed liability

[brave badger]

and you can also do fancy stuff like freezing or serialization/deserialization if you ever have a need to save/load configs

[static bluff]

Nice. Okay, so, an environment variable it is then

[static bluff]

@lusty scroll you had mentioned coming up against a potential-overwrite-important-directory issue in the past. How'd you solve it?

[lusty scroll]

not that great... added a bunch of checks like "is the path under /usr" and "is the path empty or '.' or '/'" etc

[static bluff]

And aside from the overwrite issue, does the logic for checking and setting the directories have any other potential problems you all can see?

Yeah, a blacklist of locations was my thought, which is similar to your solution

[elder blade]

I generally dislike global state of a module. What's wrong with making your own class GlobalState: and essentially putting it in that?

[static bluff]

Nothing! PureFunctor mentioned this already — I agree with you both, I'll work it in

[warm junco]

You can also invoke something called name-mangling with a double leading underscore, which changes the way you are able to access an object's attribute externally

[raven ridge]

how is that different from making your global state the a separate module, moving it into _global_state.py instead?

[elder blade]

Because it is easy to reset

[static bluff]

I've fully embraced the notion, with much consternation, that there is nothing I can or should do to limit people's access. Eventually I'm going to write this framework on the C level and I'll be able to implement privacy that say

[raven ridge]

how so? You'd add a reset() method to the class? Couldn't you add the same reset() method to the module?

[static bluff]

using a custom object lets you define its behaviour, such as properties

[raven ridge]

modules have __getattr__ and __setattr__ as well these days

hm, no, that's wrong - they have only __getattr__, not __setattr__

[static bluff]

Just yesterday actually, I played around with importing a module, and in that module creating a new object inheriting from ModuleType, injecting all the enclosing modules data, and replacing it in sys.modules. You can override the module object with your custom one and do whatever you want with it. Its not even that hackey — though I could hear all your guys in the back of my mind telling me it wasn't suitable for production code except in dire circumstances

This Envrionment object I'm creating — it'd be a dataclass, correct?

[warm junco]

!e Then why does this work?

import sys
setattr(sys.modules[__name__], "apple", "pie")
print(apple)

[fallen slate]

@warm junco :white_check_mark: Your eval job has completed with return code 0.

pie

[raven ridge]

I've never understood why people do things like class GlobalState:. We already have modules, and modules are already containers for global variables. Creating a global container for global variables inside of a module just feels like an unnecessary layer of indirection to me, at least in the typical case. Though maybe there are cases where it's useful, if you need descriptors or whatever.

[raven ridge]

modules allow the author to override what happens when an attribute is retrieved from them, but not when an attribute is set on them.

https://docs.python.org/3/reference/datamodel.html#customizing-module-attribute-access

[static bluff]

Personally, I like the dataclass method simply as a matter of organization. I think we're splitting hairs

Global state has to live somewhere, and its never easy to decide where exactly it should go

[warm junco]

I'm not sure I follow. Doesn't the link give an example of how to override module __setattr__?

[static bluff]

@elder blade @warm junco while you're here

The code I provided automatically attempts to overwrite the logsDirectory and templatesDirectory values supplied by the user. @lusty scroll brought up the possibility of supplying a "dangerous" location, such as the CWD

[raven ridge]

it does, but only by subclassing ModuleType and creating a module object in a strange and hacky way. To override __getattr__, you only need to define a global def __getattr__(): in the module. To override __setattr__, you need to change the class of your module to something different.

[warm junco]

Ah ok that makes sense. I've never used ModuleType before and didn't realise that there was a great distinction between that and a regular module

Hang on—what actually is the distinction between ModuleType and the default module type?

[static bluff]

Personally, I think the design impact of this is negligible. The only reason not to do it is confusion on the part of the reader

[static bluff]

Nothing. Modules are of type ModuleType

You can check out how module objects are created in importlib, or by doing some checking on writing your own importer. importlib is bogged down by a lot of version checking but the basics of the import machinery are fairly simple

[raven ridge]

ModuleType is the type of a module, but this example that shows how to define __setattr__ for a module is taking an object of type ModuleType, and telling it to start behaving as though it's an instance of a user-defined subclass of ModuleType, where you define that subclass to have a custom __setattr__.

it's changing the class of the current module from ModuleType to a subclass of ModuleType, to allow overriding things from ModuleType.

[static bluff]

Before I move on to writing the logic

https://hastebin.com/anicetusal.properties

Any objections?

[warm junco]

But functionally what does that affect, other than the displayed name of the module's type? Let's say that I create a subclass class A(ModuleType):... and set that as the module dunder class . What impact does that have?

[static bluff]

The main reason I can think of would be properties — either validating a type, synthesizing a value returned from a getter, or modifying how something is printed when its accessed

[raven ridge]

if the subclass doesn't override anything from the base class, then the only thing it changes is what type() returns

[static bluff]

And there is a case for module level properties

Properties are properties, we all love and use them regularly. Having them on modules would be nice. Not exactly a deal breaker, but could be useful

[raven ridge]

well, the recipe I linked above shows how to accomplish that, if you want it.

[warm junco]

Then why do you consider them to be strange and hacky, if the only change in functionality is exactly what you change the functionality to be? I'm not sure I see the difference between changing module __getattr__ and changing __setattr__ other than the slightly more roundabout way you have to do the latter.

[raven ridge]

the fact that changing the type of an existing object is a crazy thing to do, I suppose. It's weird to me that the Python language even allows you to do so. I don't think I've ever used another language that does.

[static bluff]

What do you all think about only accepting the one directory from the user, the external root directory, and havings the two dependant directorys (logs and templates) always calculated automatically from it

[warm junco]

Yeah, looking at this in the broader picture of the differences between languages, Python does seem to be quite unusual there

[visual shadow]

those directories are being generated from this user entered dir?

or rather, being generated inside?

[static bluff]

Yeah. The "external root" is where the framework will write its logs and write some automatically generated HTML, segregated into the Logs and Templates directories respectively

[visual shadow]

then yes, good.

[raven ridge]

that's a relatively new feature, too. The ability to change the class of an existing object was added somewhere in the Python 3 series... And also, it isn't possible to do for all objects, which is also weird and hacky. Only certain types of objects allow their class to be changed.

[visual shadow]

it's better to let user give 1 single path, top level, and let everything be generated accordingly. less headache for them

[static bluff]

Those two directories need to be emptied and refilled with each launch

And for me

Okay good! 😄 Thanks!

https://hastebin.com/ofobidiyus.py

K, think I'm done. Lemme know what you think

[warm junco]

One tiny thing about the type hinting—the externalRootDirectory hint could possibly be wrapped in a typing.Optional[...] since you explicitly handle the case in which it is None

[static bluff]

If you're talking about its definition in the signature, linters automatically incorporate the type of the default value in with the otherwise stated typehint

At least, PyCharm's linter does in cases when the default is None

[lusty scroll]

seems mypy wasn't that smart, required explicit Optional[...] to appease it

[warm junco]

Yeah, the docs for typing.Optional recommend that you use that hint even if it’s implicit like that

[paper echo]

i am actually in favor of encapsulating things this way, to a certain extent. it allows for dependency injection and also keeps things tidy/grouped, documented, and type-annotated

however overwriting the module import to be a single class instance is just weird to me

at that point yes just make it a module

[static bluff]

Noted!

Any of you cefPython users?

I'm just wondering in which form I'm supposed to pass settings and switches to cef.Initialize()

[lusty scroll]

and the wording in the docs makes it sound like eventually you won't be able to overwrite your module with something that's not a ModuleType

[paper echo]

what are the performance characteristics of large dicts? does dictA | dictB have complexity like O(len(a)+len(b))?

[lusty scroll]

there's three sizes of dict, right?

[paper echo]

sizes?

[lusty scroll]

yeah, I think different code based on number of keys

roughly

[paper echo]

class GlobalConfig:
    def __new__(cls, *args, **kwargs):
        newmod = ModuleType()
        ...
        return newmod

[lusty scroll]

lol

<module '?'>

[paper echo]

🙄 ModuleType(__name__)

[lusty scroll]

not sure it needs to even be a module ?

[paper echo]

well my point is that you can still do dirty stuff like that and inject it into the modules list

[lusty scroll]

if one is into that sort of thing

i vould see config.Config() and config.get_config()

or make GlobalConfig a singleton module 😜

ConfigModule seems like a more typical name

# might as well make the module also a contextmanager..
with __import__("config") as config_mod:
    # do something reading config_mod```

i ask you, have you ever seen anything more pythonic

jokes aside, I see what you're saying :p

[elder blade]

I finally kind of have a good argument to this conversation. Let's take discord.py as a good example.

There should only ever be one bot running.. so why not have all of these methods directly on the discord module?

@raven ridge Would you prefer this? Instead of having a Client object that you use .get_channel() just do discord.get_channel().

[raven ridge]

for things that there's usually only one of, no. For things that there must be only one of, yes.

I don't know anything about discord.py, but given that you said "should" instead of "must", I think there should be a class.

[elder blade]

I feel like an asyncio event loop is a good example of something that there must only be one of. That is pretty much implemented as a class.

Except that you argument sort of holds because you can get an event loop using asyncio.get_running_loop() and surely that uses global variables to some extent.

[raven ridge]

I feel like an asyncio event loop is a good example of something that there must only be one of.
That's definitely not true. There's (up to) one per thread, not one.

[paper echo]

that's my point too

enforcing singleton-ness is such a weird pattern

[raven ridge]

it's rarely the right thing to do, but there are definitely cases where it is. Your application's configuration that you read from a config file, for instance, is singleton global state. A single run of your application will only ever be configured in a single way.

singleton resources provided to you by the operating system are another - stdout/stdin/stderr are global singleton resources.

likewise for signal handlers, for instance - there's at most one handler per signal, can never be more.

[main lynx]

it would just be obnoxious if you couldn't create and close loops wherever and whenever you want

[paper echo]

sure, but enforcing that you must have a global singleton object representing that singleton resource is the odd part

again because it just takes away perfectly good opportunities for dependency injection (i.e. testability)

[main lynx]

singleton in python library is just rude

[raven ridge]

I don't think it does - you can still inject a separate configuration for your tests. What you can't do is inject two different configurations simultaneously, and run tests that assume two different configurations are in place simultaneously, because you will have written your code to assume that at any given time only a single configuration is in place.

[paper echo]

right

all the apps at my current job use a global db: AsyncIOMotorDatabase = None that gets filled in when the app starts

it sucks so much and i have been begging my tech lead to let me spend a sprint fixing it

testing is a nightmare

so much patching

[raven ridge]

isn't it roughly the same amount of work as if you could use dependency injection? Instead of the tests calling py fake_db = make_a_fake() my_module.some_method(fake_db) They call ```py
fake_db = make_a_fake()
monkeypatch.setattr(my_module, "db", fake_db)
my_module.some_method()

[elder blade]

Huh? I thought it was like one per process. So what's run_threadsafe() then?

[raven ridge]

it's a way to enqueue a coroutine onto an event loop that isn't running in the current thread.

!d asyncio.loop.call_soon_threadsafe

[fallen slate]

asyncio.loop.call\_soon\_threadsafe

loop.call_soon_threadsafe(callback, *args, context=None)```
A thread-safe variant of [`call_soon()`](https://docs.python.org/3.10/library/asyncio-eventloop.html#asyncio.loop.call_soon "asyncio.loop.call_soon"). Must be used to schedule callbacks *from another thread*.

Raises [`RuntimeError`](https://docs.python.org/3.10/library/exceptions.html#RuntimeError "RuntimeError") if called on a loop that’s been closed. This can happen on a secondary thread when the main application is shutting down.

See the [concurrency and multithreading](https://docs.python.org/3.10/library/asyncio-dev.html#asyncio-multithreading) section of the documentation.

[raven ridge]

It's an instance method that needs to be called on a loop - if there were only ever one loop, it wouldn't need to take a loop as an argument.

[main lynx]

right, running event loops in separate threads is easy and sometimes you need to do it

as it should be

the "easy" version of call_soon_threadsafe: https://docs.python.org/3/library/asyncio-task.html#asyncio.run_coroutine_threadsafe

[paper echo]

More nesting, more indirection, more opaque code, more verbosity

[raven ridge]

it's one extra line, in my example. No extra nesting, no extra indirection. Slightly more verbose.

[paper echo]

It's also a matter of knowing which routines require which resources

The logic tends to be just complicated enough that generically patching out every external resource with a mock won't really work

Whereas if the function signature specifically lists the required resources, it's a lot easier to know what you have to mock

[raven ridge]

that much is true I guess - the dependency injection approach makes it obvious which dependencies a routine has, since they're parameters to the function. In the case of using globals, you don't know.

[paper echo]

And when you have like 5 things to mock it just gets really visually ugly, with a exit stack and really long lines of code that don't really format cleanly and visually break up the flow

Having linebreaks in with is nice but we are on 3.8 still and I doubt we will upgrade until it goes EOL

[raven ridge]

if you use pytest, there's no need for contextlib.ExitStack, nor for with

[paper echo]

We use unittest of course

[raven ridge]

well, that's what you should spend a sprint or two fixing 😄

[paper echo]

unittest has better tornado integration i think

at least that's why we started using it

and the backend tech lead wasn't comfortable with adding new 3rd party tools at the time

[raven ridge]

sad trombone noise

I do not enjoy working in unittest in the least. It makes everything so verbose and so difficult.

[paper echo]

you're saying with fixtures right? fwiw i mostly do fine with defining these at the unittest class level, lots of mixins

more verbose but I think pytest fixtures are too magical and it's not easy to "goto fixture definition"

[raven ridge]

yeah. you can use the monkeypatch fixture, and it records the patches it makes and unwinds them all when the test case ends.

[paper echo]

Ok that's a neat trick

Could implement that for unittest but idk if it plays well with hypothesis. Also i tend to try to keep with patch blocks as small as possible

Regardless, the big issue is the "non obvious resource dependency" thing

It hugely increases the mental overhead of writing a new test case

To the point where I've been more or less unable to onboard anyone else to my test suite for lack of time to explain all the moving parts, bus factor of ~1

Fixing that is a top priority, and instead of building a big janky pile of test infrastructure to paper over it, I'd rather refactor the code to not require said infrastructure

[raven ridge]

come to think of it - that's an odd take when you're already depending on tornado.

[raven ridge]

that's fair enough, except that the refactoring comes with risk to the production code, whereas janky test infrastructure is insulated to just the tests.

which means there's a cost/benefit analysis to be performed.

[paper echo]

Indeed

In my case the refactoring is kind of needed anyway + there is buy in from the upper heads

And now that i have a huge goddamn test suite i feel comfortable at least refactoring the parts that I personally worked on.

But very good point, always tradeoffs

[surreal sun]

I want to mess around by setting f_lineno in a frame to another line number kind of like a goto, but I'm not sure how to implement it since i need some "trace" thing enabled with sys._getframe

[craggy plank]

Hi colleagues, Hope you all doing good. I need your suggestion in Data Science domain. So if someone who have been working or already in this field would be suggest in better way.
I want to become a data analyst. I haven't any masters degree or phD in a particular field for data analysis. So I afraid about it weather i would become or not.
I think if i mention here my progress in this domain then you would be guide me in better way.
I cleared my python intermediate after completing 12th.
Then i turned to automate things in python using selenium and had been worked 1 year.
Now i started to learn about Numpy and pandas but in meanwhile i stuck. Everything is white. No any guidance what i have to do.

Any Suggestions please. Your suggestion would be valuable for me.

[tranquil flax]

Traceback (most recent call last):
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/http.py", line 300, in static_login
data = await self.request(Route('GET', '/users/@me'))
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/http.py", line 254, in request
raise HTTPException(r, data)
discord.errors.HTTPException: 401 Unauthorized (error code: 0): 401: Unauthorized

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "main.py", line 513, in <module>
bot.run('Token')
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 723, in run
return future.result()
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 702, in runner
await self.start(*args, **kwargs)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 665, in start
await self.login(*args, bot=bot)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 511, in login
await self.http.static_login(token.strip(), bot=bot)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/http.py", line 304, in static_login
raise LoginFailure('Improper token has been passed.') from exc
discord.errors.LoginFailure: Improper token has been passed.


Can somebody help me with this error.

[unkempt rock]

hi

where is chat

[sacred yew]

@tranquil flax #❓｜how-to-get-help

@craggy plank #career-advice #data-science-and-ml

@unkempt rock #python-discussion

[tranquil flax]

  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/http.py", line 300, in static_login
    data = await self.request(Route('GET', '/users/@me'))
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/http.py", line 254, in request
    raise HTTPException(r, data)
discord.errors.HTTPException: 401 Unauthorized (error code: 0): 401: Unauthorized

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "main.py", line 513, in <module>
    bot.run('Token')
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 723, in run
    return future.result()
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 702, in runner
    await self.start(*args, **kwargs)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 665, in start
    await self.login(*args, bot=bot)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 511, in login
    await self.http.static_login(token.strip(), bot=bot)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/http.py", line 304, in static_login
    raise LoginFailure('Improper token has been passed.') from exc
discord.errors.LoginFailure: Improper token has been passed.```



**Can somebody help me with this error.**

[sacred yew]

#❓｜how-to-get-help please open a help channel @tranquil flax

[pliant tusk]

Look into sys.settrace

[lusty scroll]

you know what the sign of insanity is?

trying the same thing more than once and expecting a different result...

that's only true for idempotent systems though

[elder blade]

random.randint() moment

[lusty scroll]

you calling me random

it's in reference to a doubly posted traceback 😅

[verbal escarp]

i just had the idea in reference to the example-py-package that it might be useful to have other such minimal example packages on pypi, for instance one with a c-extension

problem is, i've never built a c-extension - any pointers?

could also cover weird-ass examples like R-compiled-extensions and other languages, variants of python like cython etc., as long as it's minimal and fully contained so that it should work

[verbal escarp]

einstein clearly had no idea of programming 😉

sideeffects, caching..

[gloomy rain]

Probably not, but he also never said the thing about the definition of insanity.

[verbal escarp]

TIL

the quote apparently was from Narcotics Anonymous

[gleaming rover]

imagine depending on mutable global state

🤢

[elder blade]

Isn't it from FarCry though?

[verbal escarp]

ehh.. https://www.professorbuzzkill.com/einstein-insanity-qnq/

[verbal escarp]

you mean like .. python? 😉

sys.path

comes to mind

[lusty scroll]

aha I could never imagine such a thing..

professor buzzkill.. aptly named

[lusty scroll]

it's not global-it's "module local" thank you very much

to the sys module which everybody can access :)

I believe the word is "singleton" ? 😉

"sys.path is subprocess and module local"

now it doesn't sound so bad anymore

[verbal escarp]

right 😉

why subprocess?

[lusty scroll]

hey it's called backwards compatibility

[verbal escarp]

hmmm.. that comment just gave me an idea

[lusty scroll]

people would have a heart attack if sys.path became inaccessible

[verbal escarp]

i worked with approaches to singletons in the past, uuid was a fairly nice go-to solution for that

if we give the modules-packages we load a __uuid__, it might simplify how to address things

then it wouldn't matter how the package is isolated, what its name or version is - just call it by its uuid

sounds like sci-fi.. ^^

[lusty scroll]

a uuid is the same thing as a GUID, right?

[verbal escarp]

there are different implementations, but the idea is the same, yeah

[spice pecan]

Different names for the same concept, yeah

GUID is what they call UUID at microsoft

[sacred yew]

microsoft has to be Special ™️

[lunar trail]

This should go in an off-topic channel, not here.

Ot channels are for anything non-python

!ot

[fallen slate]

Off-topic channels

There are three off-topic channels:
• #ot0-psvm’s-eternal-disapproval
• #ot1-perplexing-regexing
• #ot2-never-nester’s-nightmare

Their names change randomly every 24 hours, but you can always find them under the OFF-TOPIC/GENERAL category in the channel list.

Please read our off-topic etiquette before participating in conversations.

[woeful idol]

ok chill i didnt see

[lusty scroll]

thanks!

they're a bit cumbersome to work with, but i guess you pay that price if you want "universally (probably) unique"-ness

can't help noticing they're the same size as IPv6 addresses

[verbal escarp]

cumbersome how?

[lusty scroll]

if you don't have it to copy-and-paste, plus it's a bit unweildy to visually scan through them

same issue with ipv6 :p

[verbal escarp]

well, you could shorten it much like git hashes..

ipv6 is different in that regard

uuid is a hash, ipv6 is a structured number

[lusty scroll]

true, but I think Windows registry would balk at the shortened uuids

true, ipv6 is better

[verbal escarp]

it's not "better", it's a different beast all together 😉

very different goal

[lusty scroll]

and they have notations for collapsing down 0s etc

replace better --> more wetware-friendly

[verbal escarp]

well, with uuids you could shorten it arbitrarily, as long as things are inambiguous

depends on the context

[deep bramble]

What's the difference between typing.Generic and typing_extensions.Protocol?
I know that I can do something like this: ```py
from typing import Generic, TypeVar, Hashable
from typing_extension import Protocol

KT = TypeVar("KT", bound=Hashable)
VT = TypeVar("VT")

class Mapping(Generic[KT, VT]):
def getitem(self, key: KT) -> VT:
...
but I seem to be able to do exactly the same thing with protocols:py
class Mapping(Protocol[KT, VT]):
def getitem(self, key: KT) -> VT:
...

I understand what they're supposed to be doing here, but i don't understand the actual difference between them, what do they do differently?

[paper echo]

Generic means "this type has one or more parameters"

Protocol means "a type is a subtype of this type if it has matching method names and signatures, it doesn't have to actually be a subclass"

Mapping(Generic[KT, VT]) defines a concrete Mapping class with parameters KT and VT

Mapping(Protocol[KT, VT]) defines an abstract Protocol class that isn't really meant to be instantiated; it's meant to be used for type checking only, or also for isinstance checks if you decorate it with typing.runtime_checkable

i don't know if there are actual problems that can arise from using Protocol as a concrete class, but that's not really how they're meant to be used

https://github.com/python/cpython/blob/v3.10.0/Lib/typing.py#L1449-L1581 the source code suggests that there are some limitations/problems with using a protocol as a concrete class, e.g. they can only be used to subclass a specific whitelist of other classes (_PROTO_ALLOWLIST)

tldr: a Protocol is a description of a class that is used for type checking. Generic is how you create a parameterized class. they are kind of independent concepts, although Protocols can be parameterized, like Generic

[deep bramble]

thanks!

[lusty scroll]

why would there be code in cpython to check if sum() results in Union[str, bytes, bytearray] and throw away the result if so raising a TypeError

[prime estuary]

It's because adding a string/bytes like that is a bad idea, you make a huge amount of temporary strings which is very slow.

[lusty scroll]

I see. that does make sense, then.

in the case of a bytearray, if the buffer is large enough, could it be done efficiently?

[raven ridge]

It could be done efficiently for all of these types. str.join and bytes.join do it the efficient way. It's a deliberate design choice to not make sum() special case these types to do the efficient thing, because then it would just be inefficient for user-defined string-like or bytes-like types, which just pushes the problem down the lane.

it was a design decision to keep sum() dumb and make it always function as iterated addition, and to refuse to do that in a few cases where it knows that's worse than using str.join or bytes.join.

[lusty scroll]

thanks for the explanation, it's very clear

[mystic yew]

Is there a website or program that can linkup to terms? I am applying to college very soon and my school has told me about various colleges that would look me up and I shouldn’t have anything bad on any account online, so I would like to put my name on one term and my username that I use everywhere online on another term and see if they connect to each other at all in any way

sry if this isn't the type of server for this kind of question, idk what other server to ask this on

[rugged harbor]

what are "terms" in this sense? not sure what you're asking. but since it's not related to python it can go in #ot2-never-nester’s-nightmare

[lusty scroll]

yea not this server

[unkempt rock]

hi

A socket connection using port 443 (basically https), would it be allowed to have the connection open forever?

[raven ridge]

what do you mean by "allowed"?

[static bluff]

Would you guys mind taking a look?

https://hastebin.com/vofobasavi.py

This is my attempt at creating a pythonic analogue of javascript's Dataset object

Which is basically a dictionary that works with both dict and dot notation

When set with dict notation, the key is automatically converted to kebab-case

I can't say I'm over the moon with it 😐

As for why I chose to make it a mapping at all, heres my logic — and I'm happy to have anyone contribute to this stream of thought. The dataset is a container with which to segregate certain attributes on Element objects. Data attributes are different from normal instance attributes.

So:
— The dataset is a collection/container
— The dataset is either a namespace or a dictionary, as what it contains is organized by key
— The dataset should be iterable
— to be typed as a mapping, the object must inherit from mapping, and, must implement a __getitem__
— traditional usage of the Dataset object, in Javascript, uses both dot and dict notation

I'm a bit tired, so, maybe not thinking super clearly

[spice pecan]

it seems mostly fine, though some things could be simplified

You could avoid a lot of proxying by inheriting from dict

Also, __getattr__ is only called if an AttributeError was raised IIRC, so I'm pretty sure the first return is always going to be suppressed

The loop in format could be replaced with a comprehension, I think that would look a bit nicer, but it's fine as is

[spice pecan]

class Dataset(dict[str, 'Primitive']):
    def __init__(self, element: 'Element'):
        self.element = element
    
    def __getattr__(self, attribute: str) -> 'Undefined | Primitive | Element | dict[str, Primitive]':
        return super().get(self.toKebab(attribute), undefined)
        
    get = __getitem__ = __getattr__
    
    def __setattr__(self, attribute: str, value: 'Undefined | Primitive | Element | dict[str, Primitive]') -> None:
        if attribute == 'element':
            return super().__setattr__(attribute, value)
        self[attribute] = value
        
    def __setitem__(self, attribute: str, value: 'Undefined | Primitive | Element | dict[str, Primitive]') -> None:
        super().__setitem__(self.toKebab(attribute), value)
        
    set = __setitem__
        
    @contextlib_ext.suppress(KeyError)
    def __delattr__(self, attribute: str) -> None:
        super().__delitem__(self.toKebab(attribute))
        
    delete = __delitem__ = __delattr__
    
    
    def format(self) -> str:
        return ' '.join([
            f'{attribute}="{value}"'
            for attribute, value
            in self.items()
        ])
    
    def __repr__(self) -> str:
        return super().__repr__().strip('{}')
    
    @staticmethod
    def toKebab(string: str) -> str:
        return stringcase.spinalcase(string)
    
    @staticmethod
    def toCamel(string: str) -> str:
        return stringcase.camelcase(stringcase.snakecase(string))

This should be functionally equivalent

[safe wedge]

hi
how can i send code like your code?

[spice pecan]

!code

[fallen slate]

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

[spice pecan]

There's also pycon for standard repl output:

>>> 2 + 2
4

[safe wedge]

thanks

[paper echo]

i was wondering if we had that! i knew about ipython but never guessed pycon

[spice pecan]

I found it on accident a while ago, it's pretty useful

Standard py highlighting still highlights >>>, but pycon also removes highlights from output lines so it doesn't mess up:

>>> print("a class is a thing")
a class is a thing

vs

>>> print("a class is a thing")
a class is a thing

[hoary sky]

Hello people of much knowledge 🙂 About structural pattern matching... Is my understanding correct that I have to type the case-patterns explicitly? Rather than say keep a pattern stowed away as an object under a clear variable name. What if I have a complicated pattern that I want to reuse in different match statements, is there a way to keep the code clean or do I have to copy-paste it into all places?

[elder blade]

Yeah if you try to use a variable as a case it will just become "assign the object to this name" rather than what you want.

[hoary sky]

Exactly, but I was hoping there would be some special object type that I could store a pattern in, and then have case recognize it and treat it properly. Doesn't look like it though

Seems a bit bulky to me but might not be a big deal, haven't really used 3.10 yet

[static bluff]

"No real objections" is not what I was expecting to hear. You guys normal rip my code to shreds (and I appreciate it) XD

I was kinda iffy about the whole thing

Thanks!

[spice pecan]

I mean, now that we're used to the camelCase, it's okay 😄

[static bluff]

XD

❤️

Trust me, when all is said and done and you can see it in action, you'll feel a lot better about it

As for inheriting directly from dict, well. For a while this was a a mapping-in-name-only type object and I had to rig it a bit differently. That said, I don't want to inherit update/clear/pop

This object has a pretty specific use — whenever a change is made it has to send a signal through to Javascript, where its "live" counterpart lives. I don't think the auxiliary dict methods are appropriate

While I've got you — I've got a bit of a tricky one 😐

Element objects have a special dictionary inside them to store the values of their 'markup' attributes. Each markup attribute has a property-ish descriptor bearing that attributes name attached to the class. Its a good, very explicit, system

But the question is, what should happen if you reach for an attribute with no value. In javascript you'd get an undefined object returned

If the attribute isn't there are you reach for it then, in the internals, you'd get a KeyError since the value is trying to be pulled from the markupattrs dictionary. KeyError isn't right, so its a matter of either returning a None/Undefined, or, raising an attribute error

I'm leaning towards the undefined option, since the attribute itself exists on the element, just without a value

[flat gazelle]

wouldnt JS also error out with "use strict";, so I say you would say raise an AttributeError

[paper echo]

this was one of the objections to pattern matching in python. the argument is that "ocaml and haskell don't let you do that" so it's ok to not have it in python

the main problem with it imo is that you can't really match on literals in a nontrivial codebase

[static bluff]

I'm stoked for PM

[paper echo]

it's one thing to match on types - that makes sense

but you don't really want to match on 'magicString123', you want to match on myapp.constants.MAGIC_STRING

oops, sorry, you can't

if you could match on Literal[] maybe that'd be good

(hey, a runtime use for Literal!)

but you can't put non-literals in Literal anyway!

so idk

there's no solution, pattern matching is a toy that doesn't belong in the stdlib

that's my take

[static bluff]

I feel a bit differently about it

Assuming I understand PM correctly, I think its another step in Python's journey to developing an effective typing system

[hoary sky]

Interesting @paper echo 🙂 I've only seen hype around this feature before. But then I haven't been digging.

[flat gazelle]

the main merit of PM IMO is to allow easier handling of different python types representing the same logical type

[static bluff]

These days, properly annotated code is mandated, and linters just a part of life. Once beartype hits v1.0 in 22, hopefully, it'll be gamechanging in my mind because Python will then be an optionally staticly typed language. The amount of brain power people have put into typing in the last five years is astounding, especially in the face of the notion that "duck-typing-is-fine-quack-quack"

PM is just part of that, and its a good thing

[flat gazelle]

when you are writing a library, PosixPath('/mnt/vol') '/mnt/vol' b'/mnt/vol' PurePath('/mnt/vol') all represent roughly the same value in 4 different ways

and patma makes it easy to pick one and force all 4 into it

[elder blade]

I mean, if it comes to you repeating a pattern then maybe you want a function?

You'd then re-use this function instead

[hoary sky]

Yes, dementati suggested the same when I asked in #python-discussion . I think that solution covers some aspects of what I want but not all. For example, I like to use variable names to clear up if-statements:

VALID_DIRECTIONS = ['N', 'North', 'W', ...]

if direction in VALID_DIRECTIONS:``` it just makes it pretty to read 🙂 Same with regexes, I hide away the messy details under a clear variable name. But that kind of approach just doesn't seem to work with pattern matching as far as I can tell

[elder blade]

Just use a comment for that purpose😅

[hoary sky]

Well.. 😄

[verbal escarp]

well, i liked the proposed refactoring of my buffet pattern using pattern matching, that's the first actually useful application of it i've seen so far

although i haven't put it to use because i don't want to break backwards compatibility just yet..

if someone can convince me how patma (i'd prefer that abbreviation over PM, that's reserved for private message in my mind) could be used to realise multiple dispatch in python nice and simple i'm sold 😉

[paper echo]

pattern matching doesn't compose

that or you build up a tuple of args and dispatch on their types?

def foo(x, y, z):
    match (x, y, z):
        case (int, int, int): ...
        case (float, float, float): ...
        case (Decimal, Decimal, Decimal): ...

idk if that's valid syntax

[verbal escarp]

could you use it to dispatch on enum members?

because singledispatch only dispatches on types, not individual members

[elder blade]

Nah it's not an if-statement

Just use one for that

[verbal escarp]

could you programmatically build cases?

like add a case via decorator?

[spice pecan]

it is as long as you put parens after the class names, otherwise it treats them as names to bind to

[verbal escarp]

i'm guessing that would involve AST sorcery, right?

[spice pecan]

probably way too much of it

[paper echo]

entirely new syntax afaik

[paper echo]

ah, right. so if you wanted to name the values you'd write

def foo(x, y, z):
    args = (x, y, z)
    match args:
        case (int(val1), int(val2), int(val3)): ...
        case (float(val1), float(val2), float(val3)): ...
        case (Decimal(val1), Decimal(val2), Decimal(val3)): ...

(you wouldn't actually write it like that, but just demonstrating the syntax)

[spice pecan]

yeah

[paper echo]

heh the python console doesn't recognize match either

[spice pecan]

I think that's how the current syntax is

[paper echo]

>>> import dis
>>> def f():
...     match 1:
  File "<stdin>", line 2
    match 1:
          ^

[near coral]

probably too new

[paper echo]

this is in the 3.10 release 🙃

[near coral]

yup, I know

[spice pecan]

maybe you can only match on variables?

[paper echo]

but yeah i guess they didn't have time and didn't think it was important

[verbal escarp]

that would be my guess, too

[near coral]

apparently error reporting has gotten better as well with 3.10

[paper echo]

yeah that ^ is nice

[verbal escarp]

what version is the bot?

[spice pecan]

Good question actually

!e import sys; print(sys.version)

[fallen slate]

@spice pecan :white_check_mark: Your eval job has completed with return code 0.

001 | 3.9.6 (default, Jun 29 2021, 19:27:32) 
002 | [GCC 8.3.0]

[spice pecan]

Well that answers that

[native flame]

we have a bot running 3.10 too, listening to ]e
not sure if it works in this channel

[paper echo]

]e print(123)

[native flame]

...oh wow

[paper echo]

@ Moderators !!!

[verbal escarp]

grins and waits

funky channel names there

[elder blade]

Wdym?

[verbal escarp]

#deleted-channel

for instance

i admit, i was hyped too about the whole patma thing, but i'm also failing a bit to see its practicality in general..

i guess it's one of the features that screamed "but, but, but.. ALL THE OTHER LANGUAGES SUPPORT IT WHY DOESNT PYTHON?!!?"

shrug

[x] done - let's move on 😉

[peak spoke]

Most people seem to look at it as a switch case, which really wasn't necessary

as a match it's nice but there aren't that many general uses so I'm not sure if it warranted adding a statement that complex

[verbal escarp]

but under the motto "python can do anything other languages can, but better" i really wonder why tail call optimization hasn't been added, years ago :p

[split chasm]

I've already had a few places I can use it (pattern matching), if you need it, it's great, if you don't, you'll struggle to find a use

It's one of those things like the walrus operator that kinda hits you with an "aha" moment eventually

[verbal escarp]

can you share some examples?

[split chasm]

I think most of my usage is in a proprietary code base unfortunately, I'll see if I have any meaningful uses in my open source stuff

[verbal escarp]

just minimal examples to get an idea, not your actual code

apropos tail recursion, i see it might clash with the use of closures as decorators etc. so it might require its own syntax

but since GvR isn't a fan of recursion because it involves brain gymnastics that was out of the question

[elder blade]

Can I get a quick Tl;Dr about that?

[verbal escarp]

tail call optimization or why it isn't in python?

[elder blade]

The former

[verbal escarp]

you know recursion?

a function calling itself or function a calling b and b calling a back and forth etc

it's useful to traverse recursive structures like trees

traverse(root): traverse(root.left_child); traverse(root.right_child)

for example

[elder blade]

Yeah

[brave badger]

GvR mentioned something about how TCO doesn't really retain stack information and that's too magical

I'm paraphrasing of course

[verbal escarp]

tail calls are such recursive calls that end with a function call as the last statement - in a purely functional context, you can discard the information of containing above since everything relevant is in the call contained

[brave badger]

http://neopythonic.blogspot.com/2009/04/tail-recursion-elimination.html

[verbal escarp]

in python you might need that context later - for instance as a decorator etc

but if you can be sure that you don't need the context as a closure, you could throw away the stack

[elder blade]

What benefit does this give?!

[native flame]

it becomes as good as iterative

[brave badger]

Constant stack size because you're essentially turning recursion into a fancy loop

[verbal escarp]

you could have basically infinite recursive calls without stack overflows

[elder blade]

Huh..

[paper echo]

i just get annoyed because really basic small changes like json.load_file get bikeshedded into oblivion but then we get this half-baked ocaml-ism that requires major changes to the parser and AST and introduces entirely new syntax and semantics

like ok, i'm not a core dev and i'm probably not a core dev for a reason. but i really don't get it

[paper echo]

in addition to avoiding the ever-growing call stack, you can update the return value in-place instead of allocating more and more new memory

and you can optimize away the function call overhead

[verbal escarp]

i'm guessing, but it feels like a politically motivated thing.. like those mega projects politicians push for as personal monuments

[paper echo]

i don't know. but i believe pattern matching will remain a mostly-unused feature

[pliant tusk]

i actually implemented a python bytecode tail call optimizer yesterday and immediately some issues jumped out at me. Pythons exception system really doesnt work with it as is

[paper echo]

it might also be availability bias of the core devs: they would use it all the time in the stdlib, ergo it's useful

[verbal escarp]

ah, very cool! -v 🙂

(that means --verbose = tell me more)

[pliant tusk]

>>> @tco
... def fact(n, acc=1):
...     if (n < 2):
...         return acc
...     return fact(n - 1, n * acc)
... 
>>> dis.dis(fact)
  3     >>    0 LOAD_FAST                0 (n)
              2 LOAD_CONST               1 (2)
              4 COMPARE_OP               0 (<)
              6 POP_JUMP_IF_FALSE        6 (to 12)

  4           8 LOAD_FAST                1 (acc)
             10 RETURN_VALUE

  5     >>   12 NOP
             14 LOAD_FAST                0 (n)
             16 LOAD_CONST               2 (1)
             18 BINARY_SUBTRACT
             20 LOAD_FAST                0 (n)
             22 LOAD_FAST                1 (acc)
             24 BINARY_MULTIPLY
             26 STORE_FAST               1 (acc)
             28 STORE_FAST               0 (n)
             30 JUMP_ABSOLUTE            0 (to 0)
>>> ``` it optimizes like this

cut the function runtime in half, but exception handling broke in most cases

[paper echo]

huh, how did it break?

[elder blade]

I don't think it's a bad feature, and I do think it should be in Python.

But it was of course slightly over-hyped to be "the new if-statement" but that's isn't the case.

[pliant tusk]

any return inside an except block cant be optimized

and the jumps mean that you lose the recursive stack trace

[paper echo]

yeah i guess i should be clear: it's a niche feature, and it's nice enough in its place, but it's a big feature and a brand new one and an unnecessary one, and its presence bothers me because there is a lot of other "quality of life" stuff that isn't happening, or is happening very slowly

[pliant tusk]

https://paste.pythondiscord.com/loluhipefi.py @paper echo @verbal escarp incase yall want to mess with it

[paper echo]

honestly i'm 100% ok with that. maybe there should be some kind of global flag that enables or disables @tco?

[verbal escarp]

@pliant tusk as i said earlier, i suspect we'd need an alternative return statement like yield that signals that we want to throw away the stack

not sure

[paper echo]

if not os.environ.get('DEBUG'):
    tco.enable()

[verbal escarp]

hehe

[pliant tusk]

there also arent many situations where tail call optimization is useful (unless a further logic analyzer is implemented to transform non-tco functions into tco functions)

[paper echo]

it is kinda cool to be able to write foldl in python though

[verbal escarp]

in my code i would suspect it's more useful than patma, tbh ^^

[paper echo]

😆 i think it, like patma, belongs in a "language that compiles to python" (hy, coconut) and not python itself

heck someone is working on a python backend for idris2 now

[flat gazelle]

I often find myself writing code like

if isinstance (a, tuple):
  return board[a]
elif isinstance (a, int):
  return side[a]
else:
  ...

And patma does make it a lot easier

[paper echo]

yeah that would be the use case for me too

basically doing what singledispatch/multipledispatch does

[safe wedge]

who one work with pydrive?

[flat gazelle]

I buy the rationale in the original pep. There is this kind of code all over the public APIs of popular libraries and patma makes it cleaner and less error prone

[paper echo]

that is true, too. but those libraries also need to be very backward compatible. so i guess the thinking is that in 2025 this is just normal, and until then we don't see it in the wild much

[flat gazelle]

That's the case with every feature

[paper echo]

this is how i normally write stuff like that

def _load_stdin():
    return sys.stdin.read()

def _load_path(p):
    with open(p) as fp:
        return fp.read()

def _load_buf(b):
    return b.read()

def load_file(path_or_buf: str | PathLike[str] | TextIO):
    if path_or_buf == "-":
        return _load_stdin()
    elif isinstance(path_or_buf, (str, PathLike)):
        return _load_path(path_or_buf)
    else:
        return _load_buf(path_or_buf)

i personally don't see a compelling reason why that if needs to be match, nor can i envision it making e.g. pandas' internals any cleaner

they're messy because the logic is messy, not because they need pattern matching syntax

a more-generalized destructuring assignment with dicts and attributes would imo be much more useful than this

it looks pretty in ml and haskell because the language is designed so fundamentally differently

[lusty scroll]

are there serious benefits from specialization of the sort recently added to python? like adaptive opcodes

or is it more of a distraction than a performance boost

[peak spoke]

they did describe the boosts as being considerable

[paper echo]

i think measurable performance boosts are very worthwhile

[lusty scroll]

what if it's like 3% gain in instruction throughput only with certain workloads

I guess it's better than nothing

[paper echo]

if it's a common workload then maybe, or if it also paves the way for future optimizations

[verbal escarp]

code like that would also benefit from TCO

just saying

[verbal escarp]

that sounds like massive potential general performance boost, btw

isn't performance improvement what GvR is aiming for these days? maybe he should be told about these findings..

[pliant tusk]

Only in really specific places tho

[verbal escarp]

how specific? is it common enough to make a difference?

[pliant tusk]

It can be if code is written to benefit from it but most code isn't done like that by default

[verbal escarp]

i mean, dispatching is quite common and it's TCO-able, so i wonder how much impact would signaling the interpreter to throw away the stack at this point could make

[flat gazelle]

honestly, I dont really see a reason for TCO. For recursive functions, the transformation to a while loop is trivial, and for most other cases, I would rather have a stack trace than one less stack frame

[verbal escarp]

i have quite a bit of dispatch code that i don't care in a stack trace

maybe you could flag certain functions as "don't bother keeping track of this"

[lusty scroll]

i'm not sure thats possible, python does calls using actual function calls doesn't it?

[flat gazelle]

hmm, being able to omit functions from a stack trace is an interesting feature

[lusty scroll]

how would you do that? you need to keep track of nesyed execution contexts, right

[verbal escarp]

not all, no

[flat gazelle]

I mean, for convenience and keeping the stack trace useful you could just omit them from the repr and still keep the real frames

[verbal escarp]

if you pass all necessary context by arguments, you don't need to keep the current stack

[flat gazelle]

but well, with TCO you cant do

try:
    return tail_optimised_call()
except:
    ...
```since you have no stack left to return

which is far too janky for python IMO

[lusty scroll]

assuming rewriting the eval loop isn't likely

[verbal escarp]

actually you could, but it wouldn't give a useful stack trace, just a "something happened, try to disable TCO to check for details"