#internals-and-peps

1 messages ยท Page 152 of 1

halcyon trail
#

there's lots of preferences

#

the problem is many of these are based on fairly subjective things (at least, in cases of comparable library availability), whereas performance isn't very subjective.
And in practice things are not usually purely IO bound, especially as you scale up, you start seeing the effects of python's overhead, in many cases (lots of big companies have encountered this issue)

grave jolt
#

nice, easy, GC languages that are way faster than python
๐Ÿ˜‰ typescript

#

/hj

halcyon trail
#

lol

#

speaking of about to do advent of code in kotlin, second year in a row ๐Ÿ™‚

grave jolt
#

I'm about-to-do-advent-of-code-but-never-actually third year in a row ๐Ÿ™‚ (๐Ÿ˜”)

elder blade
#

That's because it's JIT-compiled to literal machine code like C is

#

There's also fundemental issues with the CPython interpreter, to get it faster you want to look into JIT-compiling it like PyPy (which.. PS: is much much faster at loops and similar).

swift imp
#

Yeah but PyPy doesn't and cannot support everything right?

#

Isn't there fundamental stuff that PyPy can't handle, making it less than ideal

peak spoke
#

It should be able to handle anything cpython can

magic nova
#

pypy's main struggle against adoption is support for all C-extensions

#

from what I remember

peak spoke
#

I believe they should work fine with it, there's just not much of a point as the jit won't do anything

white nexus
#

I mean

magic nova
#

compatibility still wasn't complete last I looked and you took a performance hit for C extensions

white nexus
#

you can use a lib that has c-extensions in a project written in pure python

magic nova
#

if it can't be a full drop-in, it won't take off

white nexus
#

so not providing a speedup on c-extensions shouldn't be a dealbreaker

elder blade
#

CPython works on any system that can compile it

magic nova
#

People looking for high performance generally don't care too much about exotic architectures

raven ridge
#

It's the other way around, though: most C extensions (though not all) will work in pypy, but pypy has to emulate a big chunk of the C API in order to make them work, and that emulation is much slower than CPython's C API implementation. Which means pypy has the paradoxical effect of making library code that no one ever bothered to optimize run much faster, but the heavily optimized parts that the library authors needed to be as fast as possible run slower

white nexus
#

that never happened

white nexus
raven ridge
magic nova
# white nexus wait what

Generally the advice in the Python community for a long time has been: if it's speed critical, write it in C. But on PyPy C extensions are slower because the C API relies so much on CPython's internals, meaning you take a big performance hit in parts of code that are likely pretty hot

white nexus
#

that's ironic

#

since pypy is pretty fast, imo, but you loose the speed gain in the already fast parts....

#

;-;

swift imp
#

So tldr JIT Compiling is monumental of a task to make it work with current Cpython plus C extensions?

raven ridge
#

CPython could add a JIT without killing performance. That may happen over the next several years. Possibly.

magic nova
swift imp
#

Is machine code generation LLVM or JIT?

#

Guido says Machine Code Generation is the future post 3.11

#

Also there's working being done for 3.11 that is "Zero overhead exception handling". The way I am understanding it, they're trying to make it so if an exception is not raise there is no overhead but I thought that was already case...tell me if I am wrong

peak spoke
#

It's fairly low already but there is some cost to it afaik

spice pecan
#

Yeah, it's not as big as it could've been, as Python uses exceptions for flow control a bunch, but it's not zero-cost either

#

there's considerable overhead

prime estuary
#

Currently, what happens when you have a try or with block is that they have bytecode instructions that push/pop a "block" to an internal stack. If exception occurs, the interpreter consults the topmost value there to know where to jump to to start evaluating except blocks. The PR moves all that to the compiler, instead the try: block does nothing at all. There's an additional table generated and added to the code object, that indicates for every bytecode where you should jump in case of an exception.

#

So there's then zero cost to try blocks, it's only if an exception occurs that you need to check the table.

#

The issue is that it breaks the PyCode_New() C-API call, because you really have to add the extra table arg. In 3.8 positional only args also caused an issue, that was solved by adding PyCode_NewWithPosOnlyArgs() that has the extra field, and defaulting to zero otherwise. But it's kinda clear that code objects are likely to change structure, something needs to change here.

raven ridge
#

PyCode_New is already not part of the stable API, and they've always been allowed to change it with each new Python version - right?

#

(not that it doesn't still cause extension authors trouble when they do so, of course)

lusty scroll
lusty scroll
#

since they're used fairly often

raven ridge
misty oxide
#

I'm embedding python in my application. It would be compiled against system python, on Linux. The non-python parts of my application are extremely performance sensitive numerical linear algebra computations. Do I actually need -fwrapv? It seems to be in python3-config --cflags on most systems I've tried.

prime estuary
magic nova
#

and then just link it all together at the end

misty oxide
#

Fair enough, thank ya.

halcyon trail
#

just be really really careful once you are linking together bits of C++ compiled with different flags

#

ODR violations can go boom

red solar
#

namespace support?

#

we have namespaces in python?

#

it seems to be just __buiiltins__, globals(), locals(), and enclosing scope

#

am i missing something?

sand python
#

those are namespaces, though?

raven ridge
#

and there are many different globals() - one per module, at least. And there's things like types.SimpleNamespace

spice pecan
#

Usually you just use a module as a namespace

raven ridge
red solar
#

yeah

#

namespaces in PyPI is different to normal namespaces?

raven ridge
#

the idea is that companies want the ability to register that they own the "google" namespace or the "facebook" namespace, and make it so that the company can centrally manage who's allowed to publish a "google.foo" or "facebook.bar" package to PyPI

red solar
#

ahhh ok

raven ridge
#

they can already force takedowns of packages that use their name through trademark enforcement, but there's no way for them to centrally manage permissions, or say which people are allowed to represent their company to PyPI, or things like that. No way for one member of a team to publish a package and another member to yank it, or whatever.

#

actually, that last one shouldn't be true - multiple people can have publish perms for a package. But if both of them quit the company before permission can be handed off to someone else...

#

It doesn't come up just for companies, either. Tools like Sphinx might want to control the entire "sphinx.*" namespace, or whatever.

verbal escarp
#

so dots in package names will have semantical meaning.. sigh

#

as if package/module naming isn't already confusing enough

elder blade
#

What's already confusing about package naming?

verbal escarp
#

let's say you want to import foo.bar.baz there is absolutely no hint what is what

#

foo.bar could be a package and subpackage, foo could be a package, bar a module..

#

with namespaces, foo could be a namespace

spice pecan
#

I'd argue you don't need to know which is which

verbal escarp
#

until you do

#

there were a number of occurances where i wondered why i'd need foo.foo.Foo

spice pecan
#

That can and should be fixed with reexporting names

verbal escarp
#

then you forget one foo and end up with an import error that doesn't make sense on first glance

spice pecan
#

Whether a module is a package or a file shouldn't matter

#

Although having some way to group separate pypi packages (if there isn't one yet) would definitely be nice

#

to have the org.package pattern reflected in code

verbal escarp
#

and then facebook eh.. meta asks to remove anything with meta in the name..

elder blade
#

Why can I set any attribute on object() while at the same time I can define __slots__().. magic?

spice pecan
#

If you add __dict__ to slots you can still set arbitrary attributes

#

though those will not get the benefits of slots

elder blade
#

Yeah true, but is there anywhere in the Python interpreter where there's both a __dict__ and availability for __slots__?

#

That has to be special-cased y'know

#

!e ```python
class MyObject:
...

With this object I can set any attributes I want to, exactly like `object()`:
```python
obj = MyObject()
obj.a = 'a'

Now take this subclass, it can't use __slots__ because there's still a __dict__:

class MySubclass(MyObject):
    __slots__ = ('abc',)

Let's see:

sub = MySubclass()
sub.xyz = 'abc'
fallen slateBOT
#

@elder blade :warning: Your eval job has completed with return code 0.

[No output]
elder blade
#

That should've errored out, this means that there's special behaviour around object()

spice pecan
#

well it can use slots

#

it has a type with a dict in its hierarchy

#

if you didn't have a dict on MyObject it would error

#

Oh, and IIRC object can't have arbitrary attributes, can it?

#

!e object().attr = 1

fallen slateBOT
#

@spice pecan :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | AttributeError: 'object' object has no attribute 'attr'
prime estuary
#

It's not really special behaviour there no, it's the other way around - __slots__ is special for Python classes.

spice pecan
#

My assumption would be that if you don't specify slots, it defaults to ("__dict__",)

#

And inherited slots are additive, which means that if you don't have a type with a dict in your hierarchy, arbitrary attributes will fail

prime estuary
#

The way it works is that in the type object, there's a field specifying the offset to the __dict__ pointer. When you define a Python class with slots, it adds that many PyObject pointers to the object size, so they're at the end (after the object data if it's inheriting something else), then adds descriptors to the class that gets/sets those.

spice pecan
#

Yeah

elder blade
#

I always thought you could???

#

I've instantiated object() and put attributes on it sooooooooooo many times... I think?!

prime estuary
#

Nope, you cannot.

#

Which sorta makes sense, since it would imply all subclasses also can.

spice pecan
#

So there's no special casing going on, object has no dict, inherited slots are additive and if you don't specify slots, it defaults to only include dict (effectively)

surreal sun
#

!e
Hmm, so say I do something like:

def foo():
    print(1)

def bar():
    print(2)

foo.__call__ = bar

foo()
fallen slateBOT
#

@surreal sun :white_check_mark: Your eval job has completed with return code 0.

1
surreal sun
#

it still prints 1 rather than 2 - why is that?

#

Is that because it uses the co_code attribute under code for running the function?

#

But then if func() does func.__call__, why is its behavior not changed? or does call in function dispatch to something else

peak spoke
#

You're creating a new instance attribute

surreal sun
#

Ohh

#

Because call is in the tp_slots? Or whatever it's called

#

I don't think it's called that but yea

#

no wait I see the issue

peak spoke
#

It's called from the type, not really important where it is

surreal sun
#

ahh

peak spoke
#

so you'd have to change function type's call which you can't really do without breaking everything

surreal sun
#

then what does it do when it's called from the type?

#

does it just run the bytecode in func.code.co_code?

spice pecan
#

sort of, yeah

#

it creates a new stack frame for the call and executes the bytecode stored in the function

surreal sun
#

Ohh

#

I see

surreal sun
#

what's the difference between all of them?

magic nova
#

tp_vectorcall is not actually used directly, it's just a field name convention that types implementing the vector call protocol use to store their vectorcallfunc pointers, and then they point their tp_vectorcall_offset to it.

#

e.g _typeobject, the object for types like set in typestruct.h and funcobject.h both have a vectorcall in their object structures. The tp_vectorcall_offset for their types then does offsetof(PyFunctionObject, vectorcall) to point the vectorcall offset to the vectorcall field inside the object struct

surreal sun
#

What's the difference between the vectorcall slot and just a call slot?

#

is one on a class level and the other on an instance level?

magic nova
#

tp_call lives in the type, tp_vectorcall_offset in the type points to a field inside the object/instance to allow overloading of the calling convention based on the specific object

#

vector calls are mostly an optimization to avoid making args tuple and kwargs dictionaries, they're not guaranteed to be called on all paths. It's your responsibility to make sure tp_call and the vector call behave identically

grave jolt
lusty scroll
#

@magic nova good info, thanks

#

what makes up the majority of calls?

#

it does seem like the vectorcall functions are quite common (at least looking at 3.11 via gdb)

unkempt rock
#

@lusty scroll

boreal umbra
# 
In [1]: def a():
   ...:     stuff = list(range(1000))
   ...:     for i in stuff:
   ...:         for j in stuff:
   ...:             pass
   ...:

In [2]: def b():
   ...:     stuff = list(range(1000))
   ...:     for i in stuff:
   ...:         pass
   ...:

In [3]: %timeit a
9.23 ns ยฑ 0.00796 ns per loop (mean ยฑ std. dev. of 7 runs, 100000000 loops each)

In [4]: %timeit b
10 ns ยฑ 0.0981 ns per loop (mean ยฑ std. dev. of 7 runs, 100000000 loops each)

Is this just timing how long it takes to look up a and b from the symbol table?

#

I was mistaken about what %timeit does as compared to the timeit.timeit function, and during a help session earlier, saw similar %timeit calls have more variance than these two: #help-honey message

paper echo
#

also the %timeit source code is worth skimming. it runs the code a couple times to estimate overhead, then figures out how many runs to perform, and maybe even tries to subtract off the overhead if i remember right

#

it's actually kind of frustrating that it's all implemented as an ipython magic and not as a standalone library

#

ripping out that code and publishing it would probably be useful to people out there in the world

#

ipython itself is wonderful but quite a big dependency

surreal sun
feral cedar
#

%timeit uses timeit.timeit, but it auto determines runs, loops, etc for you. and the output is pretty

surreal sun
#

ah

cloud crypt
#

regarding ParamSpec, is the following technically valid (according to PEP) or not?

P = ParamSpec("P")
T = TypeVar("T")


def call(function: Callable[P, T], *args: P.args, **kwargs: P.kwargs) -> T:
    return function(*args, **kwargs)```
# 
MaybeAwaitable = Awaitable[T] | T

async def maybe_await(maybe_awaitable: MaybeAwaitable[T]) -> T:
    if is_awaitable(maybe_awaitable):  # from `inspect`
        return await cast(Awaitable[T], maybe_awaitable)

    return cast(T, maybe_awaitable)

def maybe_coroutine(
    function: Callable[P, MaybeAwaitable[T]], *args: P.args, **kwargs: P.kwargs
) -> Awaitable[T]:
    return maybe_await(function(*args, **kwargs))

my actual use-case is something like this

grave jolt
cloud crypt
#

that's kinda fair, but I'm more asking about ParamSpec

lusty scroll
lusty scroll
#

!d typing.ParamSpec

fallen slateBOT
#
typing.ParamSpec 

class typing.ParamSpec(name, *, bound=None, covariant=False, contravariant=False)```
Parameter specification variable. A specialized version of [`type variables`](https://docs.python.org/3/library/typing.html#typing.TypeVar "typing.TypeVar").

Usage:

```py
P = ParamSpec('P')
```  Parameter specification variables exist primarily for the benefit of static type checkers. They are used to forward the parameter types of one callable to another callable โ€“ a pattern commonly found in higher order functions and decorators. They are only valid when used in `Concatenate`, or as the first argument to `Callable`, or as parameters for user-defined Generics. See [`Generic`](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") for more information on generic types.
lusty scroll
cloud crypt
#

I see, thanks

#

just wanted to make sure, haha

grave jolt
cloud crypt
#

yeah, that's fair too

grave charm
#

How do I know if a chain access will work?

Owo()
.get_owo()
.get_uwu()

Also how does python determines this?

quick snow
#

There's no magic. Owo() either returns an object that has a callable get_owo attribute, or not. Same with get_owo() and get_uwu

grave charm
#

Sometime I see they have a \ in the end but some doesn't thinkmon

quick snow
#

Oh, that's what you mean. When it's parenthesized, it works without the continuation-backslash

grave charm
#

Parenthesized?

#

Wdym

quick snow
#

Works:

(
    Owo()
    .get_owo()
    .get_uwu()
)

Doesn't work:

Owo()
.get_owo()
.get_uwu()
grave charm
#

Ahh

lusty scroll
#

ohhh

quick snow
#

Basically, if the call could end at that line, Python will assume it does

lusty scroll
#

wonder if that works in javascript

#

I think it does

spice pecan
#

IIRC JS is one of (if not the only) languages that does semicolon insertion backwards

#

Most languages try to insert it at the end of a line and skip if it doesn't make sense, while JS tries to skip it by default and only inserts if the next line won't make sense as a continuation

lusty scroll
sharp plover
#

!eval ```py
class A:
def foo(self):
1+1

B = A

for _ in range(1000):
class B(B):
...

from timeit import timeit
print(timeit('x.foo()', 'x = A()', globals=globals()))
print(timeit('x.foo()', 'x = B()', globals=globals()))

fallen slateBOT
#

@sharp plover :white_check_mark: Your eval job has completed with return code 0.

001 | 0.0640873140655458
002 | 0.06316586304455996
sharp plover
#

Can someone explain why this is the case?

peak spoke
#

why is what the case?

sharp plover
#

Erm, so the call in the second timeit is of a method which belongs to the 1000th class in the mro, right? Why doesn't this seem to affect the time it takes to run?

spice pecan
#

!e I wonder what the mro looks like ```py
class A: pass
B = A

for _ in range(1000):
class B(B): pass

print(B.mro)

fallen slateBOT
#

@spice pecan :white_check_mark: Your eval job has completed with return code 0.

(<class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '__main__.B'>, <class '_
... (truncated - too long)

Full output: too long to upload

spice pecan
#

It does actually add it all, hmm

peak spoke
#

Just not that big of a cost I'd guess as 1000 isn't that much, the call is the most expensive thing here

spice pecan
#

That is true, I think it would be way worse if each of those had an overload call super().foo()

sharp plover
#

However, as I increase that number 1000, the time it takes to run this part: ```py
for _ in range(n):
class B(B): pass

#

Sorry, actually n^2.3 ish.

#

Basically doubling n results in 5x runtime.

lusty scroll
quick snow
# 
for B in (type("B", (B,), {}) for _ in range(1000)):
    pass
lusty scroll
#

love how you put the pass on its own line following the crazy loop header :p

grave jolt
hidden root
#

hello

grave charm
#

Is it possible to make a string a code?
like

owo = '228283848   28282872'
for x in ['strip', 'split', ...]:
  owo.x()```
#

I have been wanting this functionality for python for a long time can't find a solution

astral gazelle
#

!e

print(getattr(str, "split")("hello world"))
fallen slateBOT
#

@astral gazelle :white_check_mark: Your eval job has completed with return code 0.

['hello', 'world']
astral gazelle
#

could also just use the method reference itself

#

!e

print(str.split("hello world"))
fallen slateBOT
#

@astral gazelle :white_check_mark: Your eval job has completed with return code 0.

['hello', 'world']
dusk comet
#

obj.__dict__[x]
Works only for objects with __dict__

spice pecan
#

Just use getattr

#

Or operator.attrgetter for repeated access to the same attribute on different objects

#

__dict__ is a very unorthodox way to achieve that

verbal escarp
#

dir(obj) also works for slotted classes

#

you can do for attr in dir(obj): getattr(obj, attr)

#

!e ```py
class A: slots = []
a = A()
for attr in dir(a): print(getattr(a, attr))

fallen slateBOT
#

@verbal escarp :white_check_mark: Your eval job has completed with return code 0.

001 | <class '__main__.A'>
002 | <method-wrapper '__delattr__' of A object at 0x7fc4c3a910a0>
003 | <built-in method __dir__ of A object at 0x7fc4c3a910a0>
004 | None
005 | <method-wrapper '__eq__' of A object at 0x7fc4c3a910a0>
006 | <built-in method __format__ of A object at 0x7fc4c3a910a0>
007 | <method-wrapper '__ge__' of A object at 0x7fc4c3a910a0>
008 | <method-wrapper '__getattribute__' of A object at 0x7fc4c3a910a0>
009 | <method-wrapper '__gt__' of A object at 0x7fc4c3a910a0>
010 | <method-wrapper '__hash__' of A object at 0x7fc4c3a910a0>
011 | <method-wrapper '__init__' of A object at 0x7fc4c3a910a0>
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/palawayegu.txt?noredirect

lusty scroll
#

then you can pass the object or call __get__(obj) to bind them

static bluff
#

Is python really falling out of favor?

#

I've seen tonnes of articles lately (more than usual) talking about how python is not the way of the future

#

And I've detected a sentiment similar to that among some other more advanced programmers

#

So I thought I'd ask you guys, since you're pretty well informed

flat gazelle
#

the age is starting to show, and the library ecosystem will only carry python so far.

#

but I think it is a matter for the far far future

static bluff
#

Old but spry ๐Ÿ˜Ž

#

Good languages are notoriously hard to kill after all

flat gazelle
#

it's not really a matter of dying, it's more a matter of "is there really a reason to use python when you could be using kotlin"

static bluff
#

So what's next? Julia? Some sort of Python derivative?

elder blade
#

Python's biggest downsides is its speed, and concurrency (more specifically, the GIL is a big pain-point for a lot of software even though there's better ways to speed up Python programs such as processes and event loops - which covers everything really).

flat gazelle
#

no one really knows what's next, my bets are on kotlin, C#, and rust

#

speed isn't even the issue IMO

#

it doesn't help

elder blade
#

But all of those are being worked on

flat gazelle
#

but the real issue is that it turns out you want closed by default, statically typed languages with local error handling which compose expressions readably

#

which is most certainly not python

#

look at go, rust, kotlin, even modern C#

elder blade
# flat gazelle speed isn't even the issue IMO

In a real world test, not really no. If you need the speed it would've already been written in C.

But it's still a big "rumour" (there's a better word for it, I just can't find it) that Python is slow.

flat gazelle
#

lots of expressions, exceptions only for the truly exceptional cases, expression are at least partially read from left to right, static typing

flat gazelle
#

language design is IMO the biggest reason

static bluff
#

yeah, I don't really mind the speed

#

What I do mind actually is the ducktyping โ€” but not from a memory point of view

#

Rather, from a usability point of view, I feel an API is easier to program when you don't have to worry about getting who-knows-what as input

#

Though, I guess I'm not really in any place to speak. Lemme finish my BSc first

flat gazelle
#

the secret to being a happy python programmer is to just assume your input is correct

#

but well, that's part of what I mean with local error handling

elder blade
#

Hmm, I see ducktyping (when it is strongly typed that is - not like JavaScript) as a big benefit

flat gazelle
#

it just turns that it doesn't work super well in practice

static bluff
elder blade
flat gazelle
#

the idea is that you write one function which can then be used in ways you didn't even consider later

#

turns out, it's not a great idea

static bluff
elder blade
#

Also isn't JavaScript technically ducktyped?

#

Technically any dynamically typed language is?

static bluff
#

My ideal language would be one with optional static type enforcement, privacy, and method overloading, in combination with python's clean syntax and OO nature

flat gazelle
#

I don't think you want python syntax

static bluff
#

Something about how privacy and type enforcement lets you explicitly specify the domain, and even the usage, of an API really rings true to me. It feels a lot more stable than Python's go-with-the-flow attitude

#

But

#

I'm coming around to the notion that I have no idea what I'm talking about. In another ten years after a bunch more education, then I'll be in position to have an opinion

elder blade
flat gazelle
#

yeah, while I find the python design of just assuming everything works and dealing with the consequences later fun, I can see why it would feel unstable

static bluff
#

They should, and they try to

flat gazelle
#

an issue with duck typing is that you lose overloading

#

since you are assuming things work, you don't really have a way to check whether X will work

#

which is why python has things like types and abcs

elder blade
static bluff
#

โค๏ธ That's sweet Blu

#

Thanks :3

elder blade
#

Sure- some people just don't care about things like that until they don't work anymore, but I dislike when the language has too much to say about that.

flat gazelle
#

every useful language has ways to bypass its type system and privacy separations

static bluff
flat gazelle
#

in python, it's just a lot less obnoxious

elder blade
flat gazelle
#

reflection

#

sth like SomeClass.class.getMethod("methodname").invoke(instance, arguments);

#

in java you also have to call .setAccessible(true)

elder blade
#

So runtime stuff?

flat gazelle
#

yeah

static bluff
#

One thing is sure โ€” there is one thing that drives me up the wall about decorators

#

And its that if you want to decorate something, but call it first with some set of priming arguments

#

You have to double-nest your functions

#

I'd do it from a class perspective, where the syntax invoked a __decorate__ method on the object

elder blade
#

That makes sense though with how the decorator feature works? Though I guess other languages have decorators be a bit special?

static bluff
#

Its functional, sure

flat gazelle
#

protip: __decorate__ is __call__

#

class decorators are quite common

#

specifically to avoid double nesting functions

static bluff
#

So you initialize the object, and then its __call__ consumes the function and sends back another function. Not bad

flat gazelle
#

though IG you still kind of have to if you want to preserve signature

static bluff
#

I'd much perfer something like this

elder blade
# flat gazelle yeah

I want to say though, you mentioned "every useful language", is Java not useful or did you not mean that a useful way to bypass?

flat gazelle
#

java has a way to bypass

static bluff
# 
class Decorator():
  def __init__(self, *args, **kwargs):
    ...
  def __decorate__(self, method):
    self._internal = method
  def __call__(self, *args, **kwargs):
    ...
flat gazelle
#

so it is useful

#

elm doesn't have a way to bypass, so it isn't useful

elder blade
flat gazelle
#

so what, it doesn't have to be practical

#

it just has to exist

#

an insane amount of java libraries are built on reflection

elder blade
#

Or hmmm, I kind of take that back. I feel like a lot of people would use that code "since it works"

static bluff
#

God I'm tired ๐Ÿ˜ Had my calculus final yesterday

#

I lack the strength to even talk about Python. Which is like, my favorite thing to do

elder blade
#

!e ```python
from typing import *

P = ParamSpec('P')
RT = TypeVar('RT')

class printer:
def init(self, before: Any, after: Any) -> None:
self.before = before
self.after = after

def __call__(self, func: Callable[P, RT]) -> Callable[P, RT]:
    def inner(*args, **kwargs):
        print(self.before)
        func(*args, **kwargs)
        print(self.after)
    return inner

@printer('Before call...', 'Executed successfully')
def sleep(duration):
return import('time').sleep(duration)

sleep(1)

fallen slateBOT
#

@elder blade :white_check_mark: Your eval job has completed with return code 0.

001 | Before call...
002 | Executed successfully
elder blade
#

@static bluff like that kind of

#

Otherwise you need two levels yeah, the first is the one that takes the configuration - then the one that gets called by Python as the decorator with the function, and lastly the one that replaces the function

#

This kind of is the same thing though, first you call the type (that's printer) which returns another callable (an instance of itself) that lastly when called returns the one that replaces the function (inner())

static bluff
#

Yeah more of less

#

I just subscribe to the notion that nesting should be avoided whenever possible

#

That

#

And

#

The idea of returning a function that can see the scope in which it was defined is (for some reason) really hard for people to wrap their minds around at first

#

Lots of people get gotcha'd with decorators, and it takes a while to come around to them

elder blade
#

You think so? I thought it made so much sense and I had a lot of issues learning about closures because there's no difference in Python

static bluff
#

Shit, no, decorators are one of the hardest things for people to figure out

#

Esspecially double-nested decorators

#

Its not that idea that a function is being returned โ€” its wrapping your mind around the sequence of events

elder blade
static bluff
#

And the idea that the function you wrote is being called, but, not directly

#

Blu โ€” my brain is full of mucus, and my soul is devoid of everything

#

I've got the itis

#

The calcul-itis

#

Ima go pretend to study for my english final

elder blade
#

From a beginner there is nothing weird with a function being able to access variables in a function it's define in. I agree that returning a function is weird to most though

verbal escarp
#

i'd say python's rise is only just beginning

#

not necessarily cpython, but python the language has a lot more to go

#

it's not like there's a lack of ideas to improve

#

and it's not like there's a lack of fresh blood to feed to the machine

#

it's definitely an interesting time with quite a couple of interesting languages and everyone is borrowing ideas from the other

#

and with active python dev in so many different areas its momentum is pretty strong

boreal umbra
surreal sun
#

Have any of you ever found yourself wanting something like str.find but instead of stopping at the first instance of the substring you pass in, it'll get every instance's index? Like

>>> "1231".find_all("1")
[0, 3]
```?
#

I'm sure there's probably an alternative to this though

radiant garden
#

find_all = lambda x: [i for i, c in enumerate(s) if c == x]

surreal sun
#

Yeah that's what I've used quite a bit in most of my code where I need it

#

I could see it being useful as a string method tbh, but then again there are already more than enough of that

white nexus
#

/docs str.

#

..

#

one second

#

!d str.removeprefix

fallen slateBOT
#
str.removeprefix 

str.removeprefix(prefix, /)```
If the string starts with the *prefix* string, return `string[len(prefix):]`. Otherwise, return a copy of the original string:

```py
>>> 'TestHook'.removeprefix('Test')
'Hook'
>>> 'BaseTestCase'.removeprefix('Test')
'BaseTestCase'
```   New in version 3.9.
white nexus
#

someone added this so like

#

find_all seems more useful than a two line if

surreal sun
#

I think that was mostly because of the confusion regarding strip, rstrip, and lstrip

#

But fair point nonetheless

white nexus
#

i know that str.strip uses this:

#

!d string.whitespace

fallen slateBOT
#
string.whitespace 

string.whitespace```
A string containing all ASCII characters that are considered whitespace. This includes the characters space, tab, linefeed, return, formfeed, and vertical tab.
white nexus
#

so its possible to use that to strip all whitespace and other characters at once

surreal sun
#

Basically they just interpret the argument into a set of characters, so "\n\nhi" as the argument would be {"\n", "h", "i"}

#

ohh you're talking about that

white nexus
#

ye

boreal umbra
# 
In [8]: tup = tuple(range(10_000))

In [9]: %timeit re.sub(r'\s+', '', str(tup))
2.85 ms ยฑ 237 ยตs per loop (mean ยฑ std. dev. of 7 runs, 100 loops each)

In [10]: %timeit '(' + ','.join(str(i) for i in list(tup)) + ')'
1.59 ms ยฑ 83.9 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

In [11]: %timeit '(' + ','.join(str(i) for i in tup) + ')'
1.6 ms ยฑ 84.3 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

The disappointing performance of the regex aside, I'm surprised that 10 and 11 (the only difference being one has a redundant call to list( )) are the same. Does cpython optimize around this?

grave jolt
# fallen slate

tfw the docstring is a few times longer than the implementation

#

...and string.whitespace doesn't include " " for reasons

white nexus
#

lance stop that!

grave jolt
#

oh wait, it does

#

I just can't read

white nexus
#

!e import string ; print(repr(string.whitespace))

#

i- i don't know what i was expecting

grave jolt
#

lmao

fallen slateBOT
#

Lance might stop, but I never will.

#

quack

#

@white nexus :white_check_mark: Your eval job has completed with return code 0.

' \t\n\r\x0b\x0c'
white nexus
#

there we go

#

\x0b?

fallen slateBOT
#

You are not allowed to use that command here. Please use the #bot-commands channel instead.

radiant garden
white nexus
#

i have no idea what I'm expecting from this bot ;-;

white nexus
radiant garden
#

And the builtins are implemented in C, so

boreal umbra
# radiant garden And the builtins are implemented in C, so 
In [15]: a, b = tuple(range(1_000)), tuple(range(1_000_000))

In [16]: %timeit list(a)
2.06 ยตs ยฑ 155 ns per loop (mean ยฑ std. dev. of 7 runs, 100000 loops each)

In [17]: %timeit list(b)
9.64 ms ยฑ 382 ยตs per loop (mean ยฑ std. dev. of 7 runs, 100 loops each)

I would still expect an entire extra pass to be noticeable, but I suppose the length of the tuple was still too low in the original example.

magic nova
#

yeah 2.06 microseconds is gonna disappear in benchmarking noise in the millisecond scale

grave jolt
# boreal umbra ```py In [8]: tup = tuple(range(10_000)) In [9]: %timeit re.sub(r'\s+', '', str...

not an answer, but there's this

In [4]: %timeit re.sub(r'\s+', '', str(tup))
2.44 ms ยฑ 6.26 ยตs per loop (mean ยฑ std. dev. of 7 runs, 100 loops each)

In [5]: %timeit '(' + ','.join(str(i) for i in list(tup)) + ')'
1.42 ms ยฑ 8.65 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

In [6]: %timeit '(' + ','.join(str(i) for i in tup) + ')'
1.41 ms ยฑ 6.51 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

In [7]: %timeit '(' + ','.join([f"{i!r}" for i in tup]) + ')'
788 ยตs ยฑ 5.8 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)
#

huh ```py
In [7]: %timeit '(' + ','.join([f"{i!r}" for i in tup]) + ')'
788 ยตs ยฑ 5.8 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

In [8]: %timeit '(' + ','.join(f"{i!r}" for i in tup) + ')'
923 ยตs ยฑ 5.14 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

In [9]: %timeit '(' + ','.join(map(repr, tup)) + ')'
809 ยตs ยฑ 5.71 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

#

interesting

#

map is faster than a generator expression?

#

even though it does a function call every time

radiant garden
#

you're also formatting the result of repr in the first cases

verbal escarp
#

every other major language struggles with the same issue. it's unavoidable that with more people also come diverging interests

#

while i don't 100% agree on the specifics of the python process (like how members of the SC are voted in), i'm quite confident that it works

grave jolt
#

no, by default {i} is actually format

#

{i!s} will be str

#

and {i!r} will be repr, which should be the cheapest

#

hmmmm ```py
In [10]: %timeit '(' + ','.join([f"{i!r}" for i in tup]) + ')'
781 ยตs ยฑ 8.71 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

In [11]: %timeit '(' + ','.join([f"{i!s}" for i in tup]) + ')'
800 ยตs ยฑ 8.29 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

In [12]: %timeit '(' + ','.join([f"{i}" for i in tup]) + ')'
785 ยตs ยฑ 4.26 ยตs per loop (mean ยฑ std. dev. of 7 runs, 1000 loops each)

white nexus
#

Python should have a way to alias class properties

#

like

# 
class Thing:
  color: int
  colour: int

always reference the same thing

quick snow
#

Sounds doable with a property, doesn't it? Or am I missing something?

white nexus
#

as far as I know, properties have to be on the instances and can't be on the class object

quick snow
#

Sure, so you need to define it in the metaclass

radiant garden
quick snow
#

!e

class ThingMeta(type):
    @property
    def color(self):
        return self.colour

class Thing(metaclass=ThingMeta):
    colour = 23

print(Thing.color)
fallen slateBOT
#

@quick snow :white_check_mark: Your eval job has completed with return code 0.

23
flat gazelle
#

You could write a descriptor that would do this

white nexus
#

what's a descriptor?

#

hm

#

!d descriptorhowto

fallen slateBOT
#
descriptorhowto

Descriptor HowTo Guide

Author Raymond Hettinger

Contact <python at rcn dot com>...

quick snow
#

!e

class AliasableMeta(type): pass
class Aliasable(metaclass=AliasableMeta): pass
class alias_of:
    def __init__(self, other):
        self.other = other
    def __set_name__(self, owner, name):
        delattr(owner, name)
        setattr(type(owner), name, self)
    def __get__(self, instance, obj_type):
        return getattr(instance, self.other)

class Thing(Aliasable):
    colour = "red"
    color = alias_of("colour")

print(Thing.color)
fallen slateBOT
#

@quick snow :white_check_mark: Your eval job has completed with return code 0.

red
quick snow
#

Slightly problematic, you'd really need one metaclass per class (or a smarter __get__), as this would not work as expected with multiple Aliasable subclasses

white nexus
quick snow
grave jolt
white nexus
#

what does set_name do

grave jolt
# grave jolt ?

I don't think it does anything else with the result of the repr

In [4]: dis.dis("x = f'{y!r}'")
  1           0 LOAD_NAME                0 (y)
              2 FORMAT_VALUE             2 (repr)
              4 STORE_NAME               1 (x)
              6 LOAD_CONST               0 (None)
              8 RETURN_VALUE
quick snow
#

__set_name__ gets called on descriptors when they are assigned to a name in a class definition. It's useful, because the descriptor (e.g. property) can know what it's called in this class. In this case, I just use it to delete the class attribute again and move it to the metaclass instead.

white nexus
#

so confusing

radiant garden
white nexus
#

Since it works, and I tweaked it around a bit, and am able to use it for my needs.

quick snow
white nexus
#

I did make changes, but given it derived heavily from that...

white nexus
peak spoke
#

Just don't? Only seems like it'd bring potential bugs for close to no benefit

quick snow
white nexus
#

the benefit is consistency with the rest of the library, which every Colour attribute or class is aliased to color, sadly

peak spoke
#

Doesn't seem worth it even for consistency with some other library

#

for example deletion would be a mess you need to deal with

#

and if anything I'd expect aliases to lead to more inconsistency in the long run

grave jolt
#

I hate interfaces with aliases

white nexus
#

tbh if a user deletes it thats on them ๐Ÿ‘€

grave jolt
white nexus
#

(fwiw this is for a discord.py fork so most rules can go out the window as discord.py was exceptionally odd)

feral cedar
#

just force people to convert to the right term, color

white nexus
#

its colour

#

not color

feral cedar
#

you lost the war

white nexus
#

my country says color but i don't care

#

I brought it up with the maintainers to see which they want to implement

raven ridge
#

call it "hue", sidestep the whole problem

white nexus
lusty scroll
#

such colorful language

deft pagoda
#

i thought they were called eye sounds

kindred scroll
#

What functionality does type checkers use to do what they do? Mypy for instance uses a parser that they've built a lot on, I was wondering if there's something like this in native Python or a devoted pypi library to determine types of certain objects at keypoints

flat gazelle
#

there is a way to get type hints at runtime, but I am not aware of a parser that can read type hints in a way sufficient for a type checker. Maybe libcst...

unkempt rock
#

where to ask installation questions for python

lusty scroll
surreal sun
#

https://youtu.be/KVKufdTphKs
Just thought I'd share this great talk here, explains the GIL and its purpose quite well

YouTube
AlphaVideoIreland
PyCon 2015 - Python's Infamous GIL by Larry Hastings

Youโ€™ve heard about Pythonโ€™s GIL. But what is it really? What does it do, both good and bad?

Come learn all about the Python GIL. Youโ€™ll learn about its history, all the problems it solves, all the problems it causes (that we know about!), and what it would take to remove the GIL.

โ–ถ Play video
surreal sun
#

Granted it's a tad outdated

swift imp
#

Give me functional or give me death!

boreal umbra
#

@swift imp can you explain what that is?

grave jolt
#

Inspect the function signature?

#

I guess that will work in most cases, but not if you have *args or **kwargs somewhere

swift imp
swift imp
flat gazelle
#

I don't think you want currying, partial application would make a lot more sense

#

ocaml manages currying with keyword arguments, but the semantics are quite complex and not super important

unkempt rock
#

What do you think about snippets such as this one?

# 
def require_perms(
        perm: Optional[Perms],
        allow_admin: bool = True,
        failure_handler: FailureHandler = default_auth_failure_handler
) -> Callable[[Handler], Handler]:
    """Decorator factory for view handlers that requires a certain permission."""
    def decorator(func: Handler) -> Handler:

        @functools.wraps(func)
        def wrapper(object_: Any, request: Request, *args: Any, **kwargs: Any) -> Response:
            if not request.user or (perm and not has_permission(request.user, perm, allow_admin)):
                return failure_handler(request)

            return func(object_, request, *args, **kwargs)

        return wrapper
    return decorator```
#

well first can you understand it on the first read

elder blade
#

Shouldn't it be Callable[[Callable], Callable[[Handler], Handler]]? huh I confused myself

unkempt rock
#

Yeah I know, 79 isn't reasonable imo

peak spoke
#

Don't see anything confusing there, maybe a more descriptive name for func could be used

cloud crypt
#

I'd probably define Decorator type as Callable[[F], G] (but that's completely personal imo)
also, ehhh, is there any way to use ParamSpec here?

#

since Anys don't seem nice, especially when they can be replaced

swift imp
spice pecan
#

How would *args support work for that?

#

IIRC currying doesn't pair well with varargs really. A workaround I had a while ago when playing around with custom implementations was using an argumentless call as a For real this time sort of thing, but this has multiple issues

flat gazelle
#

yeah, variadic functions with currying are not a great idea. keyword arguments also tend to work somewhat poorly, optional arguments even worse.

#

partial application solves some of this, but you are still limited to piping unary calls

spice pecan
#

yeah

flat gazelle
#

it would be possible to extend python to make functional programming not be a horrible pain, but it would be quite a few changes

spice pecan
#

Fairly fundamental ones at that imo

flat gazelle
#

python was originally designed about separating statements and expressions

#

moving away from that is not exactly easy

spice pecan
#

As much as I enjoy fp concepts, some of them don't really belong in Python in its current form. I think the ones we can implement using existing python constructs are enough in the majority of cases, and the need for syntactic support is not that strong

elder blade
#

In Python that is, since it is equivalent ^

flat gazelle
#

I have yet to see that

#

short of wanting the bound method object itself

light badger
#

i've never see that, but static methods are not supposed to be used for instances, otherwise it should be coded not static

spice pecan
#

Methods are just functions stored as attributes on type objects, and you can access the original function (from the type), or its bound method version (from the instance)

#

This is achieved through the descriptor protocol, particularly __get__ dunder

light badger
#

maybe i've seen such code..

spice pecan
#

I.E. str.split('hello world') is functionally equivalent to 'hello world'.split(), because they both access the same split function located on the type object

#

It's just that accessing it from an instance automatically binds the instance as the first argument

light badger
#

but i'm not sure if foo.bar is class foo and static method bar, or it was just class bar

spice pecan
#

Which is also why we explicitly specify self as the first argument to a method, unlike, say, C#, where this is implicit

spice pecan
light badger
#

ah no i think it was instanced object, but use object with static method in it's argument

#

i dont see link between functional programming and static methods in class

spice pecan
#

The link is the fact that bound methods are more of an OOP thing, and functional programming would generally favor a free function that the instance is passed to

#

The point was that instance.method(...) and type.method(instance, ...) would be the more object-oriented and the more functional ways (respectively) to write the same thing

grave jolt
#

although,

#

I think you're mixing authentication and authorization in the same function

#

Authentication is an infrastructure concern, whereas authorization is part of your business logic.

grave jolt
boreal umbra
#

an object. obviously.

candid pelican
#

what is the difference between standard libraries and external libraries??

deft pagoda
#

standard libraries come with your python installation

candid pelican
#

then the json library is standard?

deft pagoda
#

yes

candid pelican
#

the teacher is wrong

#

hehe

deft pagoda
#

Internet Data Handling

email โ€” An email and MIME handling package
**json โ€” JSON encoder and decoder**
mailcap โ€” Mailcap file handling
mailbox โ€” Manipulate mailboxes in various formats
mimetypes โ€” Map filenames to MIME types
base64 โ€” Base16, Base32, Base64, Base85 Data Encodings
binhex โ€” Encode and decode binhex4 files
binascii โ€” Convert between binary and ASCII
quopri โ€” Encode and decode MIME quoted-printable data
uu โ€” Encode and decode uuencode files
candid pelican
#

he said that json is a external librayy

white nexus
#

!d json

fallen slateBOT
white nexus
#

its not

candid pelican
#

i need to talk to him

sacred yew
unkempt rock
#

Ak wrote this function, so I'm not 100% sure

#

Oh lol

#

It is self

#

Because that decorator is used as class methods

grave jolt
#

ah

#

Then you might need to make Handler into a typevar (like H = TypeVar("H", bound=Handler)) or it might not work outside mypy

unkempt rock
#

Yeah maybe

grave jolt
#

There's a really long github issue (6 years) about how mypy treats every function attribute as a method

#

and this is the only reason it works ๐Ÿ™‚

#

(I think)

unkempt rock
#

Mypy will probably yell at us if it sees the code base though

#

Does this code even pass mypy?

plush nimbus
fallen slateBOT
#

Python/import.c lines 2395 to 2396

if (file != NULL) {
    fp = _Py_fopen_obj(path, "r");```
raven ridge
#

it's certainly weird, but not necessarily wrong. It's checking whether it was called with 1 argument or 2, and when it was called with 2 arguments, it opens the path and passes it down.

#

it's weird to completely ignore what value you were passed and just check how many arguments you were called with, but it's legal. Maybe it needs two modes for interface compatibility with something else?

plush nimbus
#

yeah stuff like _imp.create_dynamic(spec, False) or None triggering the fopen path is confusing.

As compat is a good idea for why it might.
I tried searching for any code that uses create_dynamic with a second argument anywhere and found nothing. Last change to the signature is 7 years ago though, so maybe there was something at the time...

plush nimbus
#

yeah so it looks like the original purpose of the file argument was to provide a fd to use instead of opening path, but this was removed ~python2.7.5
https://github.com/python/cpython/commit/f3a42dee9a44fd052285078aff009e8d0451d4fa#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3L2021
Seems strange that _PyImport_LoadDynamicModule (and now _PyImport_LoadDynamicModuleWithSpec) get the file opened for them when create_dynamic is called with 2 args, and have to open it themselves normally. AFAICT it should be possible to remove the whole if (file != NULL) { and just call _PyImport_LoadDynamicModuleWithSpec(spec, NULL) for all cases.
In fact I dont think LoadDynamicModule uses fp even when it is provided.

lone nexus
#

Is python suitable for creating discord bots?

#

any good tutorial for tthe same?

warm junco
grave charm
#

Does python just give the illusion of doing multiple stuff at one time?

  1. when using libs like curses or pygame
  2. threading or async
flat gazelle
#

depends on your definition of multiple things at once

#

and on your codebase in the case of threading

#

async waits for multiple things once, but executes in one thread

#

only one thread may execute python bytecode, but other things (e.g. pandas dataframe processing) can be run in parallel

grave charm
grave charm
flat gazelle
#

the CPU can run multiple different programs at once, since it has multiple cores

grave charm
#

But what about memory things?

#

They use the same memory?

flat gazelle
#

depends

#

RAM yes, but core also have some memory which is exclusive to them

boreal umbra
#

So apparently f-strings can round to a given number of significant figures, but there's currently no way for Python to know how many trailing zeros are significant. I suppose the only way to do that would be if the C struct for floats stored how many trailing zeros were written when a float literal is defined.

#

!e

from math import pi
print(f"{pi:.4g}")
fallen slateBOT
#

@boreal umbra :white_check_mark: Your eval job has completed with return code 0.

3.142
raven ridge
paper echo
#

!e ```python
a = 1
b = 1.1
c = 1.11
d = 1.111
print(f'{a:4g} {b:4g} {c:4g} {d:4g}')

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

   1  1.1 1.11 1.111
paper echo
#

hm, that'll teach me to read the docs

#

The precise rules are as follows: suppose that the result formatted with presentation type 'e' and precision p-1 would have exponent exp. Then, if m <= exp < p, where m is -4 for floats and -6 for Decimals, the number is formatted with presentation type 'f' and precision p-1-exp. Otherwise, the number is formatted with presentation type 'e' and precision p-1. In both cases insignificant trailing zeros are removed from the significand, and the decimal point is also removed if there are no remaining digits following it, unless the '#' option is used.
https://docs.python.org/3/library/string.html#format-specification-mini-language

#

so it's not significant digits like in science?

#

!e ```python
a = 1
b = 1.1
c = 1.11
d = 1.111
print(f'{a:.4g} {b:.4g} {c:.4g} {d:.4g}')

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

1 1.1 1.11 1.111
paper echo
#

ah, you need #

#

!e ```python
a = 1
b = 1.1
c = 1.11
d = 1.111
print(f'{a:#.4g} {b:#.4g} {c:#.4g} {d:#.4g}')

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

1.000 1.100 1.110 1.111
paper echo
#

there you go @boreal umbra โ˜๏ธ

#

so whether you use g, f, or e, if you use the # flag, then python will behave like "significant digits" in science - trailing zeroes are significant, leading zeroes are not.

boreal umbra
#

@paper echo while that's interesting, the # flag appears to naively assume that any trailing zero is significant, and that isn't true, either.

prime estuary
#

I think that decimal objects might be better suited for this, given they're base ten, and I think do store this sort of thing.

#

You'd want it to also propagate the sig fig value through calculations, I assume.

#

Ah yes @boreal umbra, decimal instances do store significant figure information.

grave charm
#

What's the difference between async and threads?

#

In my understanding the left one is async it does 2 things at the same time

while the right one is thread if the main code is waiting you can use the time to execute the other code

gleaming rover
elder blade
#

Thoughts on a f-bytes-string?

#

I have a use-case where I wish to use f-strings, but need the result to be bytes (so b'hello' for example).

#

Ah, I could use old-style %-formatting...

native flame
#

or bytes(f"foo {bar}", "utf-8")

elder blade
#

Doesn't that allocate two objects unnecessarily?

#

First the formatted string, then its bytes representation

flat gazelle
#

the issue is how do you decide which bytes representation to pick

elder blade
#

What do you mean?

#

How does Python decide the bytes-representation of b'hello'?

flat gazelle
#

it just takes the bytes

#

but if you have a unicode string, it gets a bit harder

#

python will only allow ascii in a bytes literal for this reason

#

well, there is __bytes__, but it's not implemented on all that many things

elder blade
white nexus
paper echo
raven ridge
#

And, as I said, not all floats start as literals, and even if they do start as literals they might wind up with fewer significant digits than the number of digits given in the literal, because conversation from decimal to float is lossy

radiant garden
# 
@dataclass
class SigFloat:
  wrapped: float
  sf: int
  __repr__ = lambda self: f"{self.wrapped:#.{self.sf}g}"

@fishhook.hook(int)
def __getattr__(self, key):
  if key == "_":
    return SigFloat(float(self), len(str(self)))
  elif key[0] == "_" and key[1:].isnumeric:
    trailing = len(key[1:])
    return SigFloat(float(self) + int(key[1:]) * 10 ** -trailing, trailing + len(str(self)))
  raise AttributeError(key)

and voila

print(1 ._) # "1."
print(1 ._0) # "1.0"
print(1 ._00) # "1.00"
#

@boreal umbra >:)

elder blade
#

This isn't really what this channel is for

unkempt rock
#

Oh

undone hare
#

I think some people would find this interesting

# 
while unchecked:
    for i, other in enumerate(unchecked):
        ...
    else:
        print("Couldn't find any match!")
        exit()
    
    unchecked.pop(i)
#

I get a warning about i being possibly undefined

#

as far as I can tell, that's impossible, right?

dusk comet
#

it is possible if unchecked is empty

native flame
# undone hare as far as I can tell, that's impossible, right?

not necessarily

In [1]: class Bad:
   ...:     def __bool__(self):
   ...:         return True
   ...:     def __iter__(self):
   ...:         return iter([])
   ...:

In [2]: def exit():
   ...:     pass
   ...:

In [3]: bad = Bad()
   ...: while bad:
   ...:     for i in enumerate(bad):
   ...:         ...
   ...:     else:
   ...:         print(...)
   ...:         exit()
   ...:     print(i)
   ...:
Ellipsis
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-3-c6bdc0fc5cb6> in <module>
      6         print(...)
      7         exit()
----> 8     print(i)
      9

NameError: name 'i' is not defined
undone hare
#

hmm

#

but it was properly type hinted as a list

undone hare
dusk comet
#

yes, but type-checkers isnt smart enough

grave jolt
#

hmmm, yeah, the type checker is probably just not smart enough

undone hare
#

Builtins?

grave jolt
#

oh ๐Ÿคฆ

undone hare
#

lol

dusk comet
#

exit object comes from _sitebuiltins

prime estuary
#

It's not always defined, you shouldn't use it other than in the repl.

white nexus
fallen slateBOT
#
Not gonna happen.

No documentation found for the requested symbol.

white nexus
#

๐Ÿ‘€

#

i thought that existed

dusk comet
#

it isnt defined only in repl

prime estuary
#

The site module (which is imported automatically during startup, except if the -S command-line option is given) adds several constants to the built-in namespace. They are useful for the interactive interpreter shell and should not be used in programs.

#

Anyway, exit() actually is in the typeshed builtins annotated with NoReturn, so type checkers should be able to understand that that code path will never continue.

undone hare
white nexus
#

yeah

#

!d sys.exit

fallen slateBOT
#
sys.exit 

sys.exit([arg])```
Exit from Python. This is implemented by raising the [`SystemExit`](https://docs.python.org/3/library/exceptions.html#SystemExit "SystemExit") exception, so cleanup actions specified by finally clauses of [`try`](https://docs.python.org/3/reference/compound_stmts.html#try) statements are honored, and it is possible to intercept the exit attempt at an outer level.
undone hare
#

I mean

white nexus
#

or if you really need to, os._exit

undone hare
#

that's the same thing

#

yes, I assume the site package is loaded

#

but like

#

who cares

white nexus
undone hare
#

is it

white nexus
#

yeah

undone hare
#

I mean

#

that's the only difference isn't it

white nexus
#

you shouldn't use it unless in a repr

#

!d exit

fallen slateBOT
#
exit 

quit(code=None)``````py

exit(code=None)```
Objects that when printed, print a message like โ€œUse quit() or Ctrl-D (i.e. EOF) to exitโ€, and when called, raise [`SystemExit`](https://docs.python.org/3/library/exceptions.html#SystemExit "SystemExit") with the specified exit code.
white nexus
#

the param is also different, slightly

undone hare
#

at that point I may just raise SystemExit myself and call it a day

white nexus
#

lol

raven ridge
#

there are good reasons not to write code that depends on site for things other than its intended purpose

white nexus
#

from sys import exit

peak spoke
#

It also closes stdin which may be unexpected if you catch it

white nexus
peak spoke
#

the site quitters

raven ridge
#

ooh, that's surprising.

white nexus
#

^

coarse siren
#

How do you guys feel about function prototypes for python? The idea is to create a "prototype" when the block of a function is omitted.
Examples:

>>> def x() -> int
<function-prototype ...>

import ctypes

@ctypes.CFUNCTYPE
def f(x: int) -> intโ€Š

import abc

class User(abc.ABC):
    @ab.abstractmethod
    def get_username(self) -> strโ€Š

I think it would also be nice to have a function prototype expression, but I'm not too sure how that would look (maybe something with lambda).

The idea arose in a thread about PEP 677 where I proposed the usage of "prototypes" as the new callable syntax, but Guido said it was too easy to accidentally forget the block... which is unfortunate because I thought it was an interesting proposal that would fix more than just the callable syntax.

raven ridge
#

I agree with Guido. Consider this very common mistake: py def foo() do_something() Today that results in SyntaxError: expected ':' on the first line, but if we had this syntax for prototypes it would instead need to raise an IndentationError: unexpected indent on the second line. Which is a decided regression for a common beginner mistake.

coarse siren
#

It should be possible to have a special case in the parse for this

#

I will try to draw up a reference implementation to see if this is actually possible

surreal sun
#

What would prototypes be used for? genuine question, I've never seen them before

#

Are they just having them set before use? like they exist but they don't truly exiset

coarse siren
surreal sun
spice pecan
#

I'm not sure I see any benefit compared to a function with a single pass or ellipsis, let alone some that would be worth adding a new type

# 
def func(arg: type) -> returntype: ...```
This is already the canonical way for stubs, and I think it makes a lot of sense to use the same for abstract methods and ffis (I don't know if this is possible already, but if it's not, implementing a custom decorator that does exactly that would be fairly easy)
#

And I'm not sure I understand how would this enable better Callable annotations, unless we're talking like

def comparison(lhs: T, rhs: T) -> int

def sort(collection: MutableSequence[T], key: comparison[T]):
    # Do things ```
#

Which should already be possible with a type alias (i think?) (albeit in a different syntactic form)

coarse siren
# spice pecan I'm not sure I see any benefit compared to a function with a single pass or elli...

You make some very good points. I guess I failed to emphasize how this would lay out the foundation for function prototype expressions, which would make it a full replacement for typing.Callable. Besides that, yeah it's probably useless, but the syntax is a lot cleaner than using an ellipsis and it solves more then PEP 677 (which just throws some typing thing into a built-in expression).

#

I think this would also lay out the foundation for a new lambda expression. Rather than () => <body> for lambda and () -> T for types, it would be () -> T: <body> for lambda and () -> T for prototypes (although this would lead to ambiguity when no return annotation is provided).

raven ridge
#

which is the normal case...

unkempt rock
#

is python a dynamically or statically typed language

raven ridge
#

Dynamically

#

In a statically typed language, you can't change the type that a variable holds over time.

# 
x = "1"
x = 2
#

Python is dynamically typed because the type of x can vary over time

unkempt rock
#

because its type checked at run time yeah

white nexus
unkempt rock
#

and is python on vm bytecode or just normally

unkempt rock
white nexus
#

and always be an object

raven ridge
unkempt rock
#

python is oop language

#

but it can change type still

dusk comet
verbal escarp
#

couldn't you use prototypes also to simplify overloading?

coarse siren
#

Ideally, yeah. Could also be used in .pyi files as well!

silk fjord
#

help pls

round nexus
#

If I had a child class which inherits the parent class init method but also require some additional arguments(can be kwargs), how would I go about writing the child's init function signature? do I include the parameters from the parent class or classify them into positional arguments?
example:

class A:
    def __init__(self, name: Optional[str] = None):
        self.name = name or "sample"


class B(A):
    def __init__(self, *args, age: int, **kwargs):
        self.age = age
        super(B, self).__init__(*args, **kwargs)

OR

class A:
    def __init__(self, name: Optional[str] = None):
        self.name = name or "sample"


class B(A):
    def __init__(self, name: Optional[str] = None, *, age: int):
        self.age = age
        super(B, self).__init__(name)```
#

both accomplish the same basically, Ig the only question is readability, or is there anything else I am missing?

#

also, the parent class could accept kwargs (I have just not shown them in the example)

elder blade
# round nexus both accomplish the same basically, Ig the only question is readability, or is t...

It usually takes a bit longer for the type checker to resolve the *args, so depends on how strongly you feel about that in the editor.

It also really depends on the attitude you have to it in my experience. For example, if the subclass "wants" name to be defined (as opposed to "idgaf just add this kwarg") I'd do the latter.

Another thing to take into consideration is how easy it is to update B when you update A. It's easier to add an argument to A in the former case (but you can't add a kwarg).

round nexus
elder blade
#

๐Ÿ˜… a more professional way to think about it is whether you want the fact that name can be passed to be implicit or explicit

round nexus
#

well, explicitly, its a positional arg tho

elder blade
#

You can think of it more as "you can technically pass name since B is a subclass of A" vs. "B fully implements everything A does so you should pass name if you'd like to"

#

I suppose I am sounding rather insane so I'll put it like this - both ways pass type hinting, the latter is more clear to someone reading the code.

round nexus
patent dove
#

hello

coarse siren
atomic egret
#

Not sure if this is the right channel for this, but my question is basically: where does python store the binaries that come from some packages. For example when I do pip install ipython, where is the actual ipython binary located? I need all possible paths it could be in, including both the user-wide and system-wide installs, but also in venvs, if one is detected

I'm basically working on a script that will find the path to any python binary like this. I already have the code to detect if venv is being used and the path to the venv directory, if that's not found, or the binary isn't present in there, it should fall back to the user path, then to the system path. I'm also open to suggestions if there's some better way to do this, such as an already written tool, however I do need it to happen fairly quickly, so preferrably I'm looking for a way to do this really quickly.

#

Note that I'm looking for both unix and windows paths

boreal umbra
#

venv can be any virtual environment.

#

Keep in mind that this is a discussion channel, so if you have any questions that relate to a problem you're trying to solve (and not to deepen your understanding per se), refer to #โ“๏ฝœhow-to-get-help.

true ridge
naive saddle
#

Wait, does this work for the actual packages themselves too? Last time I checked it seems like certain installations can have multiple site-packages ...

lusty scroll
naive saddle
#

yup that was what I expected - sigh

true ridge
#

yea, and it also depends how the package initially got installed. There might be other factors (scripts that hook through pth, and add/modify the finders). But if you install through pip, I think it simply uses a variation of sysconfig.get_path to determine the final installation location.

naive saddle
#

basically my usecase is creating a wrapper script that calls pip with the --platform option set to any so compiled wheels are ignored but the pure-python one isn't, sadly pip requires you to also pass --target to use such options. not sure if it'd be worth reimplementing pip's logic then.

west nova
#

anyone mind shining some light as to what happens under the hood with ctypes if the execution of a python script was stopped abruptly when the shared library was in use?

elder blade
west nova
elder blade
#

OOooooh! SystemInterrupt

#

There is a great article that goes into that system, hold on.

west nova
#

its either that or just a random error that causes the program to crash in mid execution really, wanted to know if it unloads the shared library correctly like in c

white nexus
elder blade
white nexus
#

the three central libraries about signal handling are:

  • signal
  • atexit
  • asyncio (any async framework)
west nova
#

now that i think about it a bit more, my question is more of a c question lol

white nexus
#

!d signal

fallen slateBOT
#
signal

This module provides mechanisms to use signal handlers in Python.

Similar names: pyside2.signal
white nexus
#

!d atexit

fallen slateBOT
#
atexit

The atexit module defines functions to register and unregister cleanup functions. Functions thus registered are automatically executed upon normal interpreter termination. atexit runs these functions in the reverse order in which they were registered; if you register A, B, and C, at interpreter termination time they will be run in the order C, B, A.

Note: The functions registered via this module are not called when the program is killed by a signal not handled by Python, when a Python fatal internal error is detected, or when os._exit() is called.

Changed in version 3.7: When used with C-API subinterpreters, registered functions are local to the interpreter they were registered in.

elder blade
white nexus
#

i have used atexit when i should have used a context manager ;-;

#

because I didn't want to nest the entire file

elder blade
#

....

west nova
white nexus
#

oh it may have been sys.stdout.flush()

#

side note, I reaaaally hate the inconsistency of the trashcan @fallen slate

cloud crypt
#

not sure if this belongs here, but is there a nice way to convert an AsyncIterable[T] to Iterable[T]?
other than something like an async list comprehension, i.e. ```python
async def async_list(async_iterable: AsyncIterable[T]) -> List[T]:
return [item async for item in async_iterable]

async def async_unpack(async_iterable: AsyncIterable[T]) -> Iterable[T]:
return iter(await async_list(async_iterable))```

prime estuary
#

The more appropriate place would be #async-and-concurrency, but yeah you'd need to iterate through it completely. An async iterator needs to await for each iteration step, so to do it synchronously you need to go through all of those.

cloud crypt
#

I see, the async channel looked like something slightly different than what I've been trying to answer
thanks!

boreal umbra
#

An instance method fills the first argument with the instance by which you accessed it. You can circumvent this accessing it through the class itself. As far as I know, there is no way to circumvent a class method filling the first argument. Am I correct?

peak spoke
#

You can use something like inspect.getattr_static

In [207]: class A:
     ...:     @classmethod
     ...:     def f(_): ...
     ...:

In [208]: A.f
Out[208]: <bound method A.f of <class '__main__.A'>>

In [209]: inspect.getattr_static(A, "f")
Out[209]: <classmethod at 0x28c20b04ca0>
boreal umbra
peak spoke
#

it pretty much just fetches it from the type's dict, wouldn't call it meta programming

spice pecan
#

Does the bound method have an attribute that exposes the underlying func maybe?

#

Sorta like __self__

peak spoke
#

it should be stored behind __func__ on both the classmethod/staticmethod and bound methods

royal panther
#

im hoping to spend some time over the holiday to whip up the annotated assertions pep and perhaps a standalone implementation

cloud crypt
#

annotated assertions?

royal panther
#

bascially making what pytest does in test files a language feature

cloud crypt
#

sounds fun for certain, haha

true ridge
lusty scroll
#

!e

class A:
  @classmethod
  def cm(*args, **kwargs): print(f"called with {args=}, {kwargs=}")
A.cm()
A.cm.__func__()
A.__dict__["cm"].__get__("type() called on this")()
A.__dict__["cm"].__func__()
object.__getattribute__(A, "cm").__func__()
fallen slateBOT
#

@lusty scroll :white_check_mark: Your eval job has completed with return code 0.

001 | called with args=(<class '__main__.A'>,), kwargs={}
002 | called with args=(), kwargs={}
003 | called with args=(<class 'str'>,), kwargs={}
004 | called with args=(), kwargs={}
005 | called with args=(), kwargs={}
lusty scroll
#

so the answer depends on whether #002, #004, and #005 are considered metaprogramming ;)

boreal umbra
surreal sun
#

I honestly don't understand why they chose the name to be power asserts considering it's just better error reporting with asserts iirc

#

so if an assert statement fails, it will breakdown how it failed rather than just saying it failed

dusk comet
swift imp
#

There is currently no way of annotating for an argument to be a bound method that takes zero arguments is there?

#

This is for annotating a descriptor method, which type checkers probably can't assess anyway but I like it for documentation

lusty scroll
swift imp
#

I guess? I'm not much of a coder outside of matlab and python

#

I'm currently annotating with. types.MethodType but I would like to indicate that the method should have zero arguments

dusk comet
#

maybe you should use Callable[[], Any] ?

lusty scroll
#

for a type hint, definitely

royal panther
swift imp
#

So that type hint is a bit disingenuous

#

Maybe I'm overthinking this

lusty scroll
#

you're saying there isn't a way to state at the point of the function definition that it explicitly takes no arguments . I guess you know that by the fact that it has no arguments, right? isn't that a guarantee it can never accept any arguments ? (even though there's no type hint to write)

swift imp
peak spoke
#

is there a reason for why super doesn't work with a descriptor's set and delete?

#

I don't like the look of super(self.__class__, self.__class__).name.__set__(self, value)

lusty scroll
#

it wouldn't be just super(Type, self) ?

swift imp
lusty scroll
peak spoke
swift imp
fallen slateBOT
#

descriptors.py line 87

def register(method: MethodType) -> None:```
swift imp
#

I don't know how to annotate that

swift imp
lusty scroll
vital tide
#

I know that python stores intigers that are really large (I don't have the compSci lingo to express this but like they can just get bigger without limit unlike in C where they have a specified length in bytes)

How can I store these intigers(independent of size) to a file?.
I know I could just do

numbers = [1, 6*10**700, 16]
with open("file.txt", "w") as f:
    f.save("|".join([str(i) for i in numbers])

But there has to be a better way.
Can you guys tell me how intigers are stored while running python? Maybe I could just use that method

#

Pls @ me

boreal umbra
#

@vital tide I'm not sure I understand. If you just want to write them to a text file as strings, you can.

vital tide
#

I just want a more disc space efficient method. Currently every decimal digit takes up 1 byte

#

I'm shure there's a more efficient method

boreal umbra
#

If you're limiting yourself to one byte, you can't represent anything over 256.

vital tide
#

I am not limiting myself to one byte

#

Look. At the moment, when I want to write the number 555666777 it takes up 9 bytes

feral cedar
#

you can use compression which should help a lot, since there's only 10 possible digits

boreal umbra
#

To bring the question back to the topic of the channel, I believe python dynamically selects the number of bits it will use to represent an integer.

vital tide
feral cedar
#

on the struct in C

spark magnet
vital tide
#

Don't know yet. Probably about 2**10

spark magnet
ruby willow
#

Store as string by converting base 10 ti base 256 with div and mod first then convert byte to ascii equivalent

#

Can the object be pickled? Try that first

#

Also it may matter if it is a signed or unsigned int if you manually serialize it to file

#

Highest bit maybe a integer sign bit

ruby willow
#

Reverse process to load string value to int

#

I had done something related to this computing string hash to int and back

#

It was a toy program lol

sacred yew
#

you need one value for the delimiter

white nexus
#

!d int you can pass a base here BTW

fallen slateBOT
#
int 

class int([x])``````py

class int(x, base=10)```
Return an integer object constructed from a number or string *x*, or return `0` if no arguments are given. If *x* defines `__int__()`, `int(x)` returns `x.__int__()`. If *x* defines `__index__()`, it returns `x.__index__()`. If *x* defines `__trunc__()`, it returns `x.__trunc__()`. For floating point numbers, this truncates towards zero.
ruby willow
#

Code will be simpler than my word hash code

sacred yew
#

since it's 0-9a-z

ruby willow
#

Do div and mod 256

#

Mod get low byte

#

Div by 256 then repeat until num = 0

#

Be sure to get ascii code of lowest byte in the loop and concat it to make base 256

#

You are now free to use any base lol

#

All your base are belong to us :3

#

Also note endian sequence if you store high or low byte first

#

You can also use bit shift operators if you want fancy shift left by 8 bits instead of integer div by 256

#

Faster but less flexible in terms of base

raven ridge
#

Can the integers be negative? Are they mostly large, or mostly small, or evenly distributed? How large are the largest ones? How small are the smallest ones?

ruby willow
#

I havent found upper bound yet when i tried

raven ridge
grave jolt
#

if it's 1024 integers... I'd definitely not think of any compression yet

pearl river
#

If it's really important to get perfect space efficiency, you could figure out a way to write the actual struct into the file

#

can probably be done with ctypes. Read the length of an int from the struct, write that length to the file, then get the actual parts from the struct and write them too.

spice pecan
#

!d int.to_bytes

fallen slateBOT
#
int.to\_bytes 

int.to_bytes(length, byteorder, *, signed=False)```
Return an array of bytes representing an integer.

```py
>>> (1024).to_bytes(2, byteorder='big')
b'\x04\x00'
>>> (1024).to_bytes(10, byteorder='big')
b'\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00'
>>> (-1024).to_bytes(10, byteorder='big', signed=True)
b'\xff\xff\xff\xff\xff\xff\xff\xff\xfc\x00'
>>> x = 1000
>>> x.to_bytes((x.bit_length() + 7) // 8, byteorder='little')
b'\xe8\x03'
```...
spice pecan
#

Using the last version, you can find the smallest possible size for the integer, and then prepend byte-length as a fixed length int in front of it

pearl river
#

oh right, forgot about that. That's more efficient than the int representation, even

#

since the int representation stores the int as an array of 32-bit integers

spice pecan
#

int.to_bytes and int.from_bytes are your friends

flat gazelle
raven ridge
# spice pecan Using the last version, you can find the smallest possible size for the integer,...

that's not necessarily maximally efficient. Imagine you've got a list of 1024 ints where 1023 of them are set to 0, and 1 of them is set to 256**255. In that case, the fixed length int you use to represent the length in bytes needs to be at least 2 bytes wide, so each of the zeroes would take up 3 bytes (2 for the "how many bytes is this int", and 1 for the value), and the one big one would take up 258 bytes (2 for the "how many bytes is this int", and 256 for the value), which adds up to 3*1023+258 or 3327 bytes.

The naive approach of writing "|".join(str(x) for x in lst) would instead take up only 2661:

#

!e print(len("|".join(str(x) for x in [0] * 1023 + [256**255])))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

2661
spice pecan
#

I meant the smallest possible size for the integer itself, prepending the length as a fixed-size integer was a separate suggestion

#

Although I do admit that it's not the best suggestion

#

And the wording definitely wasn't perfect as well, so I understand the confusion

raven ridge
#

I think @pearl river had this wrong, and thought that was as small as the file could get

#

but the framing overhead for length-prefixed bytes representations of the ints can be more than the savings gained by representing the ints using their minimum number of bytes

verbal escarp
#

do you know the size of the number beforehand?

spice pecan
#

Yeah, having fixed-size lengths can sometimes be a disadvantage rather than an advantage

raven ridge
#

though LEB128 is probably pretty close to optimal, even if not necessarily optimal

quasi hound
#

stupid idea but if 'hi' * 5 == 'hihihihihi' then 'this_is_a_sentence' / '_' == ['this', 'is', 'a', 'sentence']

# 
def __div__(self: str, sep: str) -> str:
  return self.split(sep)
#

and aaaaabbc - 'a' == 'bbc'

pliant tusk
#

!e ```py
from fishhook import hook

@hook(str)
def truediv(self, sep):
return self.split(sep)

print('this_is_a_sentence' / '_')```

fallen slateBOT
#

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

['this', 'is', 'a', 'sentence']
surreal sun
#

it's also quite intuitive

quasi hound
#

what does hook do

spice pecan
#

It is pretty intuitive, but I don't see how it would relate to multiplication

quasi hound
#

well it kinda doesn't, but division is very intuitive as splitting

spice pecan
quasi hound
#

oh

spice pecan
#

yes, that is true

#

You could technically use __rmul__ to also make the reverse operation of joining an iterable, but that's not quite as intuitive - it's only based on the idea that a / b = c -> b * c = a, ignoring the fact that it's not a mathematical operation

feral cedar
#

it's intuitive not because of the math operation, but because that's how paths are represented. dividing a string to mean splitting doesn't have that kind of..idk precedence?

white nexus
raven ridge
#

I still think pathlib's use of / is horrific

#

it is exactly the sort of operator overloading that we tell everyone who's learning OOP for the first time not to do

#

using the division operator to concatenate paths in Python is every bit as horrific as using the left shift operator to insert into streams in C++

gleaming rover
#

I like it too

gleaming rover
elder blade
raven ridge
#

yeah, using + for concatenation would be much more normal

grave jolt
#

I guess join could work

#

after os.path.join

#

well, at least / doesn't do any I/O (as fast as I can tell) unlike >> ๐Ÿ™‚

raven ridge
#

yeah, I'd say join

swift imp
#

Conceptually with respect to addition in math not so much

raven ridge
#

yes, and defining an operator's behavior based on how that operator looks, instead of the semantic meaning of that operator, is weird

charred wagon
#

You don't think a symbol can have different meanings in different contexts/domains?

raven ridge
fallen slateBOT
#

Lib/pathlib.py lines 851 to 855

def __truediv__(self, key):
    try:
        return self._make_child((key,))
    except TypeError:
        return NotImplemented```
raven ridge
#

You don't think it's weird that they needed to define the "true division" dunder to make this work?

charred wagon
#

Isn't that a consequence of the language pigeonholing the operator to be a division operator by naming the dunder that?

raven ridge
#

doesn't that answer your question about whether the symbol can have different meanings? Python has defined it to mean true division, and that's part of the data model

#

even if you can argue that Python shouldn't have pigeonholed the operator to just division, it did, and using it for things other than division is weird.

charred wagon
#

It's not so clear cut to me. Was that a deliberate decision, or an oversight? The fact that the pathlib / exists in the standard library instills doubt

visual shadow
#

Pathlib did it right imo

#

The API is a joy to use, why do I care whether it's internally called division

raven ridge
#

well, I strongly disagree. In fact, the proposal that started this entire conversation - about "dividing" a string into parts by adding a / operator for it - makes much more sense to me than the / usage in pathlib

#

I really, really don't understand why they didn't just use +

visual shadow
#

Because + just adds, without implying that a magic slash will be inserted

raven ridge
#

not least because / isn't even the path separator on one of the major platforms!

charred wagon
#

Worth noting there is another prominent case of this, which is the % operator. Does mod for integers but string formatting for strings.

visual shadow
#

It would make for even a further subtle problem. Because + never adds hidden things

charred wagon
#

And that's called __mod__

visual shadow
#

To me, that just wouldn't be intuitive. Slashes for paths is well understood

charred wagon
#

I'm indifferent I would've been fine with either slash or plus for pathlib

swift imp
raven ridge
#

for some APIs but not all

swift imp
#

Uh no

#

On windows it's completely valid

visual shadow
#

It's fair to say / isn't a thing on windows, since the interface prominently enforces \ instead.

elder blade
raven ridge
#

the Windows APIs themselves accept / but many libraries do not, and regardless, even for the Windows APIs, it's a path separator, but not the path separator

visual shadow
raven ridge
#

Would you really expect that the / operator adds \ or / to the str() representation depending on the platform?

elder blade
visual shadow
#

On pathlib objects, why not?

raven ridge
#

!e ```py
from pathlib import PureWindowsPath as PWP
print(PWP("a") / PWP("b"))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

a\b
raven ridge
#

seems really weird to me that / inserts a \

#

and in that case, why shouldn't \ work?

visual shadow
#

Simply put, you always use some_single_method for all joins, agnostic to platform

raven ridge
#

does + really never insert things?

elder blade
visual shadow
#

And the library deals with it

raven ridge
#

in [1] + [2] you get [1, 2] - it inserted a comma!

visual shadow
elder blade
#

+ just makes so much sense - a/b/c + d/b becomes a/b/c/d/b

visual shadow
#

Well, let me amend my stance some. Perhaps I could get used to + for it

elder blade
#

+ joins two things together - like two paths - while / divides two paths?!

visual shadow
#

But it would take me about the same time to get used to / really

elder blade
#

That is - of course - until you start reading / as "haha funny, kind of looks like a path separator"

raven ridge
elder blade
#

I want to clarify that I sound a bit salty here, but don't mean to ๐Ÿ˜”

visual shadow
#

You're fine!

raven ridge
#

Wouldn't you expect that ```py
(a + b).parts == a.parts + b.parts

white nexus
swift imp
raven ridge
#

technically, yes - but I shouldn't have said "appends", because / doesn't mutate

#

it returns a modified copy

elder blade
swift imp
#

Why would someone expect addition on some arbitrary programming construct follow linearity of mathematics

raven ridge
#

The current behavior is that py (a / b).parts == a.parts + b.parts Isn't that weird? Wouldn't it be more expected if ```py
(a + b).parts == a.parts + b.parts

swift imp
#

I wouldn't expect either

#

Not without context of the program

elder blade
#

In spec, in all of it's glory

#

I'd also say intuition ๐Ÿ˜—

visual shadow
swift imp
#

Just bc they're called arithmetic operations doesn't mean they have to be used strictly for that

charred wagon
#

And yet the language subverts its own standard, even from potentially it's inception considering %. I don't know how old that is but it's probably been around since the 90s

swift imp
#

It's like expecting repr to provide a string that can be evaled to get the object back

raven ridge
charred wagon
#

I don't buy the argument that the language intends for these to only be used as arithmetic operators

visual shadow
#

I'm on phone and I hate it. Give me some leeway here on syntax

elder blade
swift imp
#

It's a guideline

#

Not a rule

elder blade
swift imp
#

I'd bet large majority of python developers do not expect nor want it

#

Core devs included especially

raven ridge
#

and obviously it's not a rule that / must only be used for division, or that % must only be used for modulus. But no one is arguing that it is - I'm arguing that, if they needed to use some operator for this, using the operator that's already used for concatenation in other contexts to do concatenation in path contexts would make a lot of sense

#

and that using the operator that's used for division in other contexts to do concatenation in path contexts is weird

swift imp
#

I'm confused why it's so difficult to understand that different contextes are different

raven ridge
#

I don't think anyone is saying that.

elder blade
visual shadow
#

But along the same line of reasoning, the fact that + adds another slash is a non trivial cognitive load.

#

I don't find it intuitive, same way you consider / unintuitive

swift imp
raven ridge
raven ridge
charred wagon
swift imp
#

I'm saying the context of the object is enough to say otherwise

visual shadow
#

You find it intuitive after knowing about paths. That's chicken and egg imo

white nexus
visual shadow
#

Anyone can find something intuitive once you know how it works

charred wagon
#

Who would've thought we'd spend Christmas discussing a plus and a slash.

raven ridge
#

ok, but by that argument, wouldn't it be surprising that Path("a").joinpath("b") adds a / ?

charred wagon
#

Beautiful

white nexus
swift imp
#

Idk guys, I'm just saying, I read the docs and go in expecting nothing

elder blade
elder blade
raven ridge
swift imp
#

You guys keep saying it's a general expectation but it's an expectation for who?

feral cedar
#

for programmers

visual shadow
#

Consistency helps everyone, and it's worth discussing about

elder blade
white nexus
#

now divide print by whatever you want to print ๐Ÿคก

elder blade
#

Okay but like... this is what C++ does

visual shadow
feral cedar
#

the less weird overloading there is, the better. I definitely agree with geek here

elder blade
#

!e ```python
class Printer:
def init(self, original):
self.original = original

def __call__(self, *args, **kwargs):
    return self.original(*args, **kwargs)

def __lshift__(self, other):
    return self.original(other)

print = Printer(print)

print << 'hi'

feral cedar
#

make a metaclass so you don't have to instantiate it

fallen slateBOT
#

@elder blade :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 11, in <module>
003 | ModuleNotFoundError: No module named '__builtin__'
native flame
#

import builtins

raven ridge
#

just do print = Printer(print)

native flame
#

or use the global __builtins__ variable

#

or that

swift imp
raven ridge
#

you've just moved the goalposts, then. You started with:

I'm just saying, I read the docs and go in expecting nothing
And moved to saying that you expect arithmetic types to follow mathematical conventions

#

you clearly do expect some things, and what we're discussing here is what things it is reasonable to expect.