#internals-and-peps

1 messages ยท Page 120 of 1

undone hare
#

Please make me happy and don't use SMS OTP

#

Please use actual TOTP apps, and the protocol is completely free

sinful aspen
#

!coded

fallen slateBOT
#

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

mystic fable
#

Hey there,
so I did some minor benchmark on something I always wondered how Python would handle this

a = True
b = True
c = False
d = True
e = False

true = True
false = False

# which is faster

def case1():
  if a & b & c & d & e:
    return

def case2():
  if a:
    if b:
      if c:
        if d:
          if e:
            return
def case3():
  if false & true & true & true & true:
    return

def case4():
  if false:
    if true:
      if true:
        if true:
          if true:
            return

def case5():
  if True & True & True & True & True & True & True & False & True & True:
    return

def case6():
  if True:
    if True:
      if True:
        if True:
          if True:
            if True:
              if True:
                if False:
                  if True:
                    if True:
                      return

As you can see it's always 2 cases against each other (1v2, 3v4, 5v6) to see which is faster.
I used timeit with number=10000000 and ran it multiple times but numbers are so far almost always in the same range.

PS D:\random> py .\py_conditions.py
0.9419427
0.7143202
0.9368509999999999
0.5433409999999999
0.44758319999999996
0.44890989999999986
PS D:\random> py .\py_conditions.py
0.9518120999999999
0.7242126
0.9262693999999998
0.5291763
0.4499335000000002
0.4791475999999997
PS D:\random> py .\py_conditions.py
0.9553244
0.6448077
0.9519328999999999
0.5475068999999997
0.4734653999999998
0.4550950999999994
PS D:\random> py .\py_conditions.py
0.9648111
0.7847829
0.9934985000000001
0.5216031999999999
0.4517395999999998
0.47423090000000023

So why is it that after the first False the and & chained if condition isn't just skipped as a whole? At least this is my theory of what is going on. I assume it will go on checking each bool. Tho the very last two cases are interesting in that sometimes case 5 is faster and sometimes case 6. However nested seems to be generally faster.

#

I hope you get the point in why I used multiple values for different booleans, two and then using True/False directly.

peak spoke
#

the last two cases get the dead code taken out and are equal to a blank function, with the others & is a bit slower as it's a proper operator with all the things around that

#

They also don't short circuit as the bitwise ops can be overridden and the behaviour may change because of that, that's why you should use the boolean and and or operators

mystic fable
#

aight. almost forgot about and

# 
0.9826204000000001
0.6563553
0.6486467
0.9842187
0.6138250000000003
0.5497526000000001
0.4566882999999997
0.5040013999999999
0.4553172999999999

(2 + 3 * n) are the and results. It's definitely faster than bitwise but still the nested if's are a tad faster.

peak spoke
#

I believe nested ifs and the equivalent with one if and and chaining compile to the same thing

grave jolt
#

@mystic fable If you use and instead of &, case1..case4 compile to exactly the same bytecode.

unkempt rock
#

How do you learn that information? the documentation for if statements doesn't mention that kind of information

grave jolt
#

I just write to functions and run dis.dis on both

#

The documentation should explain the semantics of if, not all details of how CPython implements them.

paper echo
#

@unkempt rock also because & more or less corresponds to __and__, and by default represents bitwise and. and is the short-circuiting boolean operation, and it's the correct choice here, purely from a language semantics perspective. that & happens to give the same answer in this one case isn't important.

#

or maybe i understood your question?

#

but for things like what i just said above, it's all in the docs somewhere, but not necessarily that well explained or kind of obtuse

#

so, experience and talking to other people who already know how it works is a good way to learn!

flat gazelle
#

bool supports & and | as non short circuiting boolean operators

#

but you very rarely need that

native burrow
#

hello , i was parsing a table , so to get informations i need , i gotta clean the output from "html comments and tags" , how can i do that?

visual shadow
paper echo
unkempt rock
#

they are all dead

grave jolt
unkempt rock
#

ok

sand goblet
# 
s = ""
for n in range(something):
    s += str(n)```
In CPython is the line inside the loop realistically considered amortized O(1)? Or is that not necessarily true?
Assuming thereโ€™s only ever one reference to s
#

Since it using realloc instead of always creating an entirely new array in memory

charred pilot
#

yeah

sand goblet
#

Thanks

sand goblet
#

If you append to a list, and it doesnโ€™t have enough space and needs to resize, but the realloc is successful without having to move everything to a different place in memory, is that O(1), or is it still O(n) or something else?
Like does it still need to set the extra memory to some default value? Or does it not need to do that?

fossil zenith
#

Hmmm I think that the algorithm would always be considered O(n) because the O(1) that is a particular case, the worst case means resizing O(n)

sand goblet
#

But what is it when that specific thing happens?

carmine garnet
# sand goblet If you append to a list, and it doesnโ€™t have enough space and needs to resize, b...

it seems to use PyMem_Realloc to resize the list, which according to the docs is modeled after C realloc, which either resizes the memory block if there is free space immediately following it (which would be O(1)?), or allocates new memory and copies to it the contents of the initial block, which, I believe, would be O(n)

here are my sources for all of that if you want to read more
https://github.com/python/cpython/blob/main/Objects/listobject.c#L45
https://docs.python.org/3/c-api/memory.html#c.PyMem_Realloc
https://en.cppreference.com/w/c/memory/realloc

#

so the short answer is that it's O(n) in the worst case

#

as Jaime said lol

sand goblet
#

Ok, then it sounds like itโ€™s O(1) if it doesnโ€™t need to move it to a different area in memory

#

Thanks

fossil zenith
#

Hey guys where can I read which PEPs are being discussed? I would like to read about how Python is going to evolve

grave jolt
halcyon trail
#

Do python lists have capacity separate from their size?

sand goblet
#

Right

grave jolt
#

yeah, but you can't get it without hacks

halcyon trail
#

Yeah, I mean for an immutable string type, unlike a mutable one, it's not a given

#

I think at least on say, JVM, immutable strings do not have capacity. I think it's because python quietly treats your immutable string as mutable if the refcount is 1.

#

Or rather, I should say, cpython does.

sand goblet
#

It tries to use realloc when that happens

halcyon trail
#

That said, it's still better to never write code like that, tbh

#

even realloc is no guarantee of anything really, that kicks things back to implementation details of the allocator

sand goblet
#

If youโ€™re adding to the end of it

#

Someone earlier said you could realistically expect for it to be amortized O(1)

halcyon trail
#

On common platformss, in common cases, yes, I think it will be.

#

Well, and certainly on CPython.

#

But why not just use "join", right

grave jolt
halcyon trail
#

Thinking more about it, I'm really not sure that realloc alone would be enough to save you from getting quadratic behavior. To not get quadratic behavior, you're counting on the allocator geometrically over-allocating

charred pilot
#

yeah, join looks better and is definitely constant

halcyon trail
#

The problem is, and this is honestly a reasonably educated guess but not certainty, I doubt that most allocators work that way

#

allocators aren't going to give you 10% extra memory, as your allocations get large. It's just a weird thing to do.

#

For small memory sizes, yes, it may happen

#

I can't honestly see why though, if you ask for 4KB of memory, you'd get 5K, etc. You may get 4KB rounded to a multiple of 64 or 128 or something like that

#

but, to get amortized behavior, you need some constant C > 0 such that whenever you ask to allocate N bytes, you always actually get at least (1+C)*N bytes. It doesn't really make sense for an allocator to do that.

grave jolt
#

if you aim for performance, also don't forget to not use a generator expression with str.join (on CPython) ๐Ÿ‘€

#

because internally it will turn it into an array first anyway, but not with a list

halcyon trail
#

that's pretty weird

charred pilot
#

it has to find out how much memory to allocate i guess

halcyon trail
#

but surely the performance difference can't be that huge. at least, I wouldn't write things differently for performance reasons in that situation.

#

it's just not a language where worrying about constant performance factors makes that much sense

sand goblet
grave jolt
halcyon trail
#

right

charred pilot
grave jolt
#

yeah, it can't really know until it iterates through the entire thing

sand goblet
#

By generating all the values and then generating them again with the same starting point

grave jolt
#

wdym by 'generate them again'?

peak spoke
#

it can't consume the iterator twice

halcyon trail
#

you can't assume that you can run through a generator twice

#

generator/iterator

#

things can have side effects

flat gazelle
#

yeah, it creates an array and uses that

halcyon trail
#

not all iterators can be iterated twice at all, etc

flat gazelle
#

unless it is given a list, then it skips that step

sand goblet
#

that makes sense

halcyon trail
#

the only real alternative here is to do more like what a list would do, and treat it as a series of extend calls

#

then you don't need to create a separate array

#

but then you will be doing reallocation

flat gazelle
#

yeah, I am pretty sure strings don't support overallocation since they are immutable

halcyon trail
#

there was just a whole discussion about this

#

I think that while your point makes sense, I don't think it's true

#

because CPython as an internal optimization will actually mutate strings if there's only reference to them

grave jolt
#

I checked the CPython source, strings don't have an additional capacity attribute

visual shadow
#

overallocation and opportunistically extending are two different things

halcyon trail
#

Then maybe they really do just call realloc. But that would still be a mutation of course.

flat gazelle
#

ye, it is odd that the exact reallocations end up good enough to be faster than any other method for concating strings

halcyon trail
#

what do you mean by "exact reallocations", you mean exact allocations?

flat gazelle
#

ye, a += "?" allocates exactly 1 extra byte for the ? afaik

halcyon trail
#

reallocates

#

I mean, when you say "faster than any other method", it just happens to be the method chosen in python, a very slow language, I wouldn't read too much into it ๐Ÿ™‚

#

In a GC language you worry a lot about memory usage too, not just performance.

flat gazelle
#

the string concat is mostly implemented in C, so I would expect it to at least somewhat match expectations

halcyon trail
#

The python list growth factor of 9/8 probably isn't very optimal for performance, but it is quite nicely conservative with memory usage

flat gazelle
#

that is true

halcyon trail
#

realloc probably helps a lot with smaller allocations, and most strings tend to be relatively small, unlike say lists, where assuming that lists are small will eventually make your users sad

#

But, all said and done, doing += on a string in a for loop should probably just be avoided

flat gazelle
#

agreed

visual shadow
#

aye, relying on implementation details should always be considered bad form.

grave jolt
#

and if you're creating a 50MB string in memory and then dump the entire thing somewhere, there is a chance you're better off dumping it in chunks in the first place

halcyon trail
#

Yeah, a contiguous string isn't usually the right data structure at that point

flat gazelle
#

if your program is CPU bound to that extent, you are better off in rust/nim/probably even pascal tbh most of the time

sand goblet
halcyon trail
#

You'd probably use a rope at that point

flat gazelle
#

yeah, rope concat does tend to be faster

halcyon trail
#

Strictly speaking if the refcount is 1, there is no reason to create a new python object regardless of what happens, allocation wise

#

there's always overhead with the python object itself

#

When the refcount is 1 presumably it simply calls realloc, which may or may not do a new allocation, and uses the memory from realloc

#

For user code there wouldn't be any reason to care whether the realloc was serviced by expanding or a new allocation

halcyon trail
#

it's an alterate data structure for backing a string

#

the most common and simplest implementation is just a contiguous sequence of characters, an array

#

ropes are not contiguous

#

https://en.wikipedia.org/wiki/Rope_(data_structure)

Rope (data structure)

In computer programming, a rope, or cord, is a data structure composed of smaller strings that is used to efficiently store and manipulate a very long string. For example, a text editing program may use a rope to represent the text being edited, so that operations such as insertion, deletion, and random access can be done efficiently.

#

what's nice about them is that they're not entirely node based either, in the sense that the nodes can contain sequences of characters. This makes a lot of common string operations super super efficient. For example, concatenation becomes O(1). You just create a new rope with the first and second string in nodes.

#

I think it's used often to hold, say, all the text in a file, in an IDE/editor

#

not exactly

#

Well, yeah, that sort of thing is called a tree

#

Well, just a tree, tbh

#

It's a type of binary tree

#

trees are usually implemented as nodes with pointers to other nodes, though there are exceptions

#

so people don't usually call them "linked trees"

deft pagoda
#

i have an implementation that pretty prints

# 
In [1]: from sacks.sequences import Rope

In [2]: r = Rope('once upon a midnight dreary')

In [3]: r
Out[3]: Rope('once upon a midnight dreary', leafsize=8, type=str)

In [4]: r.prettyprint()
14
โ”œโ”€7
โ”‚ โ”œโ”€7 - 'once up'
โ”‚ โ•ฐโ”€7 - 'on a mi'
โ•ฐโ”€7
  โ”œโ”€7 - 'dnight '
  โ•ฐโ”€6 - 'dreary'

In [5]: r[:5] = 'twice '

In [6]: r
Out[6]: Rope('twice upon a midnight dreary', leafsize=8, type=str)
carmine garnet
magic python
#

wondering about thoughts on

B008: Do not perform function calls in argument defaults. The call is performed only once at function definition time. All calls to your function will reuse the result of that definition-time function call. If this is intended, assign the function call to a module-level variable and use that variable as a default value

From : https://github.com/PyCQA/flake8-bugbear

which is triggered by something like:

def f(x : pathlib.Path = pathlib.Path('something.txt')):
    x.write_text('this is pointless')

which gives ./c.py:3:27: B008 Do not perform function calls... (same error above).

What would the alternative be here ? Seems safe to ignore - not sure if i'm missing something though.

astral gazelle
#

Default x to None and check in the function?

magic python
peak spoke
#

should be safe ignore, but while I'm not sure why I don't really like creating the default like that

#

It'd still work like a normal default, unless you accept None as a value

magic python
#

feels a bit weird writing pathlib.Path twice, unless. you're specifically talking about passing a function result as a default arg

#

i don't really get what's up with using the result of a function as a default arg tho, thought maybe i was missing something

peak spoke
#

the return type may be mutable, or the function's return changes depending on other state

#

so you may get unexpected values if you don't do it intentionally

magic python
#

๐Ÿค” i guess the assumption is that the function is understood

astral gazelle
#

It tells you how to handle it if this is intended

magic python
astral gazelle
#

Im not sure about function calls, default arguments only evaluated once is a common source of bugs tho, maybe theyre just trying to get you to avoid it altogether

paper echo
magic python
spark magnet
#

f-strings are the best

halcyon trail
#

f-strings are pretty cool

#

% is pretty not idiomatic, except that logging still uses it, probably forever ๐Ÿ˜•

magic python
#

yeah - i was wondering if % would be considered as such at this point (to me they are).

re logging, hadn't thought of that (i don't log ๐Ÿ˜ฌ ), weird that it does that

peak spoke
#

you don't use the operator there directly, but pass in the arguments to the log call and it uses % internally which won't change because of compat

paper echo
#

logging "style" seems like such a strange and ill-thought-out feature

magic python
#

idk what compat is

#

unless it's shorthand for compatibility

#

it seems the reason is to do with not evaluating the string until the very last moment, i don't understand enough to know why % doesn't and f does tho

peak spoke
paper echo
#

because if the log message is never emitted, an f-string still is evaluated and has to construct the string

magic python
peak spoke
#

f strings get evaluated immediately as they're read so it can't be done with them. But it could be done with .format; it just didn't exist when logging was put in place

magic python
#

.format would be much nicer imo

paper echo
peak spoke
#

those don't apply to the log messagse themselves

magic python
#

but you can't do "{x} ... {y}".format( x = f(a), y = g(b)) can you, in logger? or can you ๐Ÿค”

paper echo
#

no, why would you?

peak spoke
fallen slateBOT
#

Lib/logging/__init__.py lines 365 to 368

msg = str(self.msg)
if self.args:
    msg = msg % self.args
return msg```
magic python
paper echo
#

oh, wait really?

#

i thought you could control the log parameter style too

#

i guess no

paper echo
#

i remember what it was

halcyon trail
#

I just use f strings

#

Don't really care about the non laziness for the things I log

#

it's a bit of an unfortunate design

paper echo
#

there are other reasons to use the logging framework

#

the laziness is kind of silly anyway, string-ifying is rarely an expensive operation

#

control over where output goes and how it's formatted, log levels, and named loggers are the main appeal for me

#

to have all that be independent of the program source code and controllable within each application is extremely valuable to me

#

you can liberally use logging all over your application and the user can turn it on or off at their discretion

#

and use lots of debug logging output that you don't have to delete later

quasi lintel
#

okk..nice

static bluff
#

You guys remember in The Martian when Matt Damon says "In the face of overwhelming odds I'm faced with only one option. I'm gonna have to science the shit out of this."?

#

I'm feelin that ๐Ÿ˜„ ๐Ÿ˜›

#

@paper echo on the topic of logging, is there a way to have a logger queue its messages until a file handler is attached? I've got a situation right now where I'm measuring some runtime data, then creating the logs directory, then creating the logger, then logging the runtime data

#

I'd prefer to create the logger with no handlers, measure the runtime data, log the runtime data (it being placed into a cache at this point), then make the logs directory, then the handlers, and then have the cached messages get logged

#

Its just a better sequence, less backtracking (in my mind)

swift imp
#

Speaking of logging, which is something I need to get adjusted to, does anyone know a good guide on incorporating logging into a module?

static bluff
severe lichen
#

What's the best way to track development of Python? In particular - track new PEPs and change in their statuses. Ideally, with an RSS feed.

charred wagon
severe lichen
#

Thanks!

#

Unfortunately, it's mailing list, not RSS feed, but better than nothing.

undone hare
#

Don't they have a RSS feed for the mailing list?

rancid lantern
#

I was trying to see if there was a tool for automatically generating an RSS feed from a mailing list, but to my disbelief I can't find an obvious approach]

charred pilot
#

why does

a = "hey"
def foo():
  print(a)
  a = 10
foo()
``` error?
error is
`UnboundLocalError: local variable 'a' referenced before assignment`
unkempt rock
#

Python assumes any names which have been assigned to inside a function to be local before the function even runs, and there is no local value for a before the print

peak spoke
paper echo
paper echo
#

There might be performance reasons to do the logging in a separate thread or enqueue log messages though, so maybe you still want your thing

#

But if your program crashes ๐Ÿชฆ โ˜ ๏ธ

paper echo
static bluff
#

Its all happening in a single thread though. Its just that the location of the logs directory is OS/runtime specific, and so the local envrionment has to be mapped before the logs directory is created

#

So its measure runtime -> creates logs directory -> create logger -> log runtime details

visual shadow
#

Why not create log dir, create logger, then do whatever

static bluff
#

I'f prefer create logger -> measure runtime -> cache runtime details -> create logs directory -> create file handlers -> write cache

visual shadow
#

Okay. With the disclaimer that that work flow doesn't really align with loggers in general, whose aim is to log things as they happen, you could always do a workaround if you really wanted.

#

Create a temp or inmemory file to log to, make a logger. Log to this temp file.

#

Then at the end of the run, copy the contents out to the actual log locations and call it good

static bluff
# 
def startup():

  logger = Logger('Application')

  runtime = RuntimeAuditor()
  runtime.measureEnvironment()
  runtime.logEnvironment() # <- message is cached
  runtime.createLogsDirectory()

  logger.createFileHandlers() # logs directory now exists, so handlers can be created
  logger.writeCachedMessages()

  # other stuff, logged as it happens

def startup():
  runtime = RuntimeAuditor()
  runtime.measureEnvironment()
  runtime.createLogsDirectory()

  logger = Logger('Application')

  runtime.logEnvironment()
#

@visual shadow its not really a problem, its just sequencing. But I'm trying to set things up to 'log as I go' as seamlessly as possible. These are my potential sequences

visual shadow
#

I think always setup logger before doing anything that you want to log.

static bluff
#

I get that, but I think I need to reiterate. I can't make the logger until I know where the logs directory will go

visual shadow
#

You can log to an intermediate place

static bluff
#

So to re-quote Matt Damon, in the face of overwhelming odds I'm faced with only one option - to science the shit out of it. Your suggestion is one option, a custom logger class with a delay function is another. And my original question was just to see if you guys knew of a built in way to handle this

#

(Its not that serious, I just like that quote XD)

halcyon trail
#

"delay functions" just aren't really a thing for loggers, hence the reactions

static bluff
#

XD Fair enough

halcyon trail
#

in most cases, nothing significant happens before you know where to log

#

If significant stuff does happen, then you'd have to weigh your options. In C++ where it's a bit more involved getting the logging going, I have the global logger log to stdout (or is it stderr) by default. Then the python script that launches the C++ redirects stdout/stderr to a particular file.

#

The C++ eventually switches the logging to a proper file but catching stdout/stderr likke that is useful anyway

#

in case of a crash etc

#

In python life tends to be simpler, and usually I don't need logging to parse a config file or command line options, which is the only thing happening before you know where to log

static bluff
#

Python, my first and truest love

paper echo
#

@static bluff i don't quite understand the need to do it like you describe. it sounds like you're still doing a lot of weird stuff just to be different or because it reminds you of how things work in other languages

#

the only reasons i can see for deferring logging to disk are: 1) performance, 2) thread safety

#

it seems like your concern is just stylistic? in which case i think maybe you should be using common lisp instead of python, where you can write your own arbitrary (and arbitrarily convoluted ๐Ÿ˜‰) abstractions

#

that said i am very interested in how to make logging better, thread-safe-er, and non-blocking

#

and i'd be really curious about how people do logging in high-concurrency applications and such

visual shadow
#

Logging is thread safe right?

halcyon trail
#

what do you mean by "thread safer", it is thread safe

paper echo
#

streamhandler just has an internal lock right?

halcyon trail
#

probably

visual shadow
#

Yeah I'm guessing they use locks to achieve safety

paper echo
#

iirc there were cases when log messages would mess with each other or conflict with the program's other outputs to stderr/stdout

halcyon trail
#

that shouldn't happen with threads

paper echo
#

but either way, you have to now stop your event loop (for example), acquire a lock, do i/o, then release the lock, then proceed

#

it'd be nice to dump a log message to a queue and have it eventually go to disk or whatever

#

i know there's a QueueHandler but i haven't messed with it

#

and i haven't seen anyone use it

halcyon trail
#

well, this is python, so the logger is logging in the same thread, so it's already really inefficient

paper echo
#

right

visual shadow
#

Yeap, queue logger would do it, but eh I have never needed to use it yet atleast

halcyon trail
#

in high performance situations, the logger is in another thread

#

well, depending what kind of performance you need, I should say

#

but if latency for your main event loop is the main concern, then yeah

paper echo
#

true, it is the kind of thing where if you need performance you might not want python

halcyon trail
#

"might" ๐Ÿ˜‰

paper echo
#

hey, pypy, graalpython, and cython all smoke cpython on even nontrivial code

halcyon trail
#

I mean tons of languages smoke pypy etc

paper echo
#

i've found that cython basically gives you a 2x runtime speedup just by copying and pasting your python code without any changes

halcyon trail
#

At any rate, it really depends how far down you want to go. At its basics, you'd typically have the logger running in its own thread, and then you have a MPSC lock free queue, if you wanted

paper echo
#

just on principle i'd be happy to have something like that

#

maybe even within a structured concurrency framework like trio or anyio

halcyon trail
#

but even then there are still optimizations, e.g. you don't want to do any string formatting in main threads, only in the logging thread. But I've seen even C++ loggers that advertise themselves as high performance miss that optimization so eh

paper echo
#

i also want a logger that accepts a callable

halcyon trail
#

yeah, that is the nice way to do logging

#

unfortunate that python's syntax for it is kind of shitty

paper echo
#

90% of the time my "expensive" pre-logging work can't be easily encapsulated in "string formatting", unless i start wrapping things in callable LoggerPrepare-er things that implement the exact __str__ method i want

#

that could be a nice abstraction actually

halcyon trail
#

gotta envy Kotlin, logger.info { "Lazily evaluated string " }

paper echo
#

does kotlin have lazy evaluated blocks in general?

#

that's a cool feature

halcyon trail
#

it has lambdas

paper echo
#

oh that's a lambda

#

nice

#

tricky syntax

halcyon trail
#

It seems that way and Ruby basically contorted itself back in the day to have syntax like this, "blocks"

static bluff
# paper echo <@!404546189372162058> i don't quite understand the need to do it like you descr...

Sorry for not responding, was having a smoke. Its not so much that I want to be different, as I have nooooooooo clue what I'm doing. I've been coding long enough to be good with the language but software architecture is new to me. That, and there isn;t exactly a for-dummies book for a project like mine. I'm gonna start school in the fall! Beyond that, there is a method to my madness. I'm inching up on a critical juncture in this project and then I was going to share it here and hopefully get some feedback

halcyon trail
#

but it's actually really simple

#

{} is a lambda. If you pass a lambda as the last argument to a function, it doesn't have to be inside parens. If its the only argument, you can drop the parens entirely. That's it.

deft pagoda
halcyon trail
#

Swift does exactly the same thing.

paper echo
#

heh, crystal went the other direction and followed ruby, no first-class functions of any kind!

#

but honestly i don't miss them in crystal

halcyon trail
#

Kind of amazing how they achieved the same result as Ruby but with a one sentence explanation instead of endless blog posts about procs vs blocks

#

It's just very cool because obviously in these brace languages, keywords look like if (something) { ... }

#

and now, your lambda syntax can basically match that, so you can write really nice looking code and DSL's, that's still extremely easy to reason about

paper echo
paper echo
#

or is it if({...}, something)?

halcyon trail
#

Yeah, the logging cookbook even has a multi process recipe for logging, it's essentially async since all the IO happens in the logging "server", the handlers just send the messages to the logging server

#

well if is a keyword

#

but you can write a function my_if

#

my_if (something) { ... } is the same as my_if( something, { .... })

#

yeah

paper echo
halcyon trail
#

Yeah. It's just a brilliant idea, IMHO.

paper echo
#

i always forget to read the logging cookbook when i have questions

halcyon trail
#

It's crazy how much mileage you get from a handful of simple ideas like this in Kotlin, IMHO. The nice lambdas + extensions enable so much.

paper echo
#

does if even have to be a keyword at that point?

halcyon trail
#

for practical reasons it's better if it is

paper echo
#

i can see why it'd be a "primitive" function, i guess you want it to be a reserved word so people don't overwrite the if function lol

halcyon trail
#

other than that though you can write your own if complete with else

#

that works like the real one, just with some syntactic limitations

paper echo
#

even in common lisp if is a primitive (a "special form")

halcyon trail
#

actually, maybe not quite, there are some challenges in getting the else tow ork

#

Yeah, I mean you do need some primitives

#

and you need something like if to implement if anyway

#

so instead of making something like if the primitive, just make if the primitive

#

however some lisp primitives are definitely redundant

grave jolt
halcyon trail
#

e.g. you don't actually need let, or let*

#

you can recreate let/let* by creating a lambda and passing the arguments into the variables and then calling it immediately

#

@grave jolt if you can cast the boolean to an integer, then sure

#

although that's a pretty ugly feature, IMHO :-). If you don't have the cast then of course, you're just back to figuring out how to convert false to 0 and true to 1 without an if ๐Ÿ™‚

fresh wave
#

yo yo its ya boi python back at it again coding

#

making games

#

making kids happy

grave jolt
charred pilot
#

of course

grave jolt
#

in that case you don't need a primitive if ๐Ÿ‘

fresh wave
#

a shut up map lol

#

oh hash not hush

halcyon trail
#

but then that implies you can implement an entire hash map without if....

fresh wave
#

im too tired for dissssssssssssss

grave jolt
#

!ot @fresh wave If you want an off-topic conversation, we have off-topic channels

fallen slateBOT
fresh wave
grave jolt
fresh wave
#

theres alot of channels holy

#

the channels rised

#

ot beard wtf lol

grave jolt
halcyon trail
#

I guess, it would still be very bizarre to choose to make hash maps "built ins" but not if

fresh wave
#

imagine replying to ur own comment

#

dats like pinging urself for mod but ur a mod

deft pagoda
#

I'd like to register implementations of an abstract base class, but something weird is happening... First, if a child class of an abc doesn't implement an abstract method it will be in the __abstractmethods__ attribute of it:

In [36]: class Spam(ABC):
    ...:     @abstractmethod
    ...:     def ham(self):
    ...:         ...
    ...: 
    ...: class Nee(Spam):
    ...:     ...
    ...: 
    ...: class Newt(Spam):
    ...:     def ham(self):
    ...:         ...
    ...: 

In [37]: Nee.__abstractmethods__, Newt.__abstractmethods__
Out[37]: (frozenset({'ham'}), frozenset())

So, to register only implementations of Spam one might think you could just look at this attribute:


In [39]: class Spam(ABC):
    ...:     registry = { }
    ...:     def __init_subclass__(cls):
    ...:         if not cls.__abstractmethods__:
    ...:             Spam.registry[cls.__name__] = cls
    ...: 
    ...:     @abstractmethod
    ...:     def ham(self):
    ...:         ...
    ...: 

In [40]: class Ham(Spam):
    ...:     ...
    ...: 
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-40-47f12c5bcb46> in <module>
----> 1 class Ham(Spam):
      2     ...
      3

But this blows up.

#

I'm wondering if there's a nice solution, as it is I'm just going to filter out the abstract classes afterwards.

halcyon trail
#

is this maybe an ordering thing

#

maybe init subclass gets called too early?

deft pagoda
#

it does, super().__new__ is called before _abc_impl

halcyon trail
#

so, possibly this could be solved with having two separate base classes

#

SuperSpam defines init subclass

deft pagoda
#

could try

halcyon trail
#

doesn't seem to help. I guess it's an issue of the relative ordering of the metaclass new

#

and init subclass

#

probably my next try here would be to use ABC meta instead

#

Create a new metaclass that does the ABC meta stuff first, and then decides whether to register it

deft pagoda
#

mixing meta classes ๐Ÿ˜ฆ

#

yeah, could just inherit ABCMeta though

# 
In [44]: class RegisterImp(ABCMeta):
    ...:     def __new__(mcls, name, bases, namespace, **kwargs):
    ...:         cls = super().__new__(mcls, name, bases, namespace, **kwargs)
    ...:         if not cls.__abstractmethods__:
    ...:             cls.registry[cls.__name__] = cls
    ...:         return cls
    ...: 

In [45]: class Spam(metaclass=RegisterImp):
    ...:     registry = { }
    ...:     @abstractmethod
    ...:     def ham(self):
    ...:         pass
    ...: 

In [46]: Spam.registry
Out[46]: {}

In [47]: class Ham(Spam):
    ...:     ...
    ...: 

In [48]: Spam.registry
Out[48]: {}

In [49]: class Ham(Spam):
    ...:     def ham(self):
    ...:         pass
    ...: 

In [50]: Spam.registry
Out[50]: {'Ham': __main__.Ham}

this does the trick

halcyon trail
#

yeah, i did the same except I tried to create a base class for easier use

#

but then you have to filter it out ๐Ÿ™‚

# 
class RegisterABCMeta(ABCMeta):
    def __new__(cls, name, bases, dct):
        c = super().__new__(cls, name, bases, dct)
        if not c.__abstractmethods__ and c.__name__ != 'RegisterABC':
            c._registry[c.__name__] = c

        return c


class RegisterABC(metaclass=RegisterABCMeta):
    pass


class Spam(RegisterABC):
    _registry = dict()
    @abstractmethod
    def ham(self):
        ...
#

i remember having to do this silly "filtering" when I was doing registration, before I started using init subclass

#

I don't know a better way

deft pagoda
#

i don't think you need to filter it -- the base class should have an empty frozenset so shouldn't be registered

halcyon trail
#

it's also arguably nicer to pass the registry as an argument, I think

#

it got registered in my case, not sure why

#

wait, what do you mean, empty frozensets do get registered

#

we register everything that has no abstract methods

deft pagoda
# 

In [54]: bool(frozenset())
Out[54]: False
halcyon trail
#

RegisterABC itself will have no abstract methods

#

so it gets registered

deft pagoda
#

oh, i used the wrong metaclass

# 
    ...:         if not cls.__abstractmethods__ and bases:
    ...:             cls.registry[cls.__name__] = cls

can check for bases instead

halcyon trail
#

I tried to pass the registry as an argument but I can't quite remember how to pass it along, I use this stuff once in a million years

#

that's a good idea

static bluff
#

I wanted to get you guys' thoughts...

# 
    def __init__(self, **settings: Union[None, bool, int, float, str]):
        Controllers.application = self # globalize the main application object

        # NOTE: Initializing the application requires a 'pre-init' procedure, to startup logging and
        #         the global exception handler. This has to happen before anything else so that errors
        #       during startup and debug startup details can be recorded

        Controllers.console = Console('Applcation', delay=True, **settings)

        Controllers.runtime = RuntimeAuditor(**settings) # measure runtime, file structure, etc
        Controllers.runtime.createLogsDirectory() # logging cannot begin without the logs directory
        Controllers.console.createFileHandlers() # begin logging
        Controllers.runtime.logRuntimeDetails()

        Controllers.excepthook = ExceptionHandler() # override built-in sys.excepthook
        Controllers.excepthook.logInitialSettings()

        # the application's pre-init procedure is complete โ€” execute callbacks
        self.dispatchEvent(Event('pre-init'))

        # initialize wxPython โ€” 'internal-pre-init' and 'internal-init' callbacks are executed
        self.applicationInternal = self.ApplicationInternal(self)
        self.logApplicationInitialSettings() # log wxPython startup details

        Controllers.settings = SettingsController(**settings) # compute initial global settings
        Controllers.settings.logInitialSettings()

        # On OSX โ€” create an application-level 'master' toolbar
        # On Windows/Linux โ€” create a default toolbar that is copied and attached automatically to
        #                    every window (optionally, on a per-window basis)
        Controllers.toolbar = Toolbar(master=True, **settings)
        Controllers.toolbar.logInitialSettings()
# 
        # On OSX โ€” create an application-level 'master' accelerator table
        # On Windows/Linux โ€” create a default accelerator table that is copied and attached
        #                    automatically to every window (optionally, on a per-window basis)
        Controllers.accelerators = AcceleratorTable(master=True, **settings)
        Controllers.accelerators.logInitialSettings()

        # the application's startup procedure is complete โ€” execute callbacks and write a message
        Controllers.console.info('Initialization complete.')
        self.dispatchEvent(Event('init-complete'))
#

Sorry if the comments are a bit unclear, my back is killing me and I just wanted to get this up here

#

This is the best I can do to get around the logs-directory-not-existing issue we were talking about earlier

halcyon trail
#

Not sure what you mean by "measure runtime". This code looks a lot like java though.

#

I'm not really sure what you need to do to "get around the issue" still. You just parse arguments/config, then create the logging directory if necessary, then initialize the logger. that's pretty much it.

static bluff
halcyon trail
#

If there's something specific you need to get around in the first place, it's not clear what it is

static bluff
halcyon trail
#

I need a head scratching emoticon but i don't see one

#

ah well

static bluff
#

Its that the logs directory goes in a different place depending on the OS. Within the app bundle on OSX, in an application support directory on Windows, and in the cwd if running as an unbundled script

#

All told, there are like, 10 - 15 steps that have to be taken to sort of 'measure' the local environment

#

Little baby steps, nothing difficult

halcyon trail
#

okay, but it seems like that's the only one relevant to logging?

static bluff
#

For logging its: bundled or not, what OS, get the current working directory, based on those three values get the 'external' root of the application - where the logs directory will go and possibly other support directories - possibly make an application support directory, make the logs directory, and then make the master log file

#

Thats like, half the steps performed by the runtime auditor, so I figure there's no point splitting its work into two

#

But please- I posted it so I could get a general critique. I'm trying to learn to do this stuff right

halcyon trail
#

I would still split it up, not much downside

#

but if you feel like you don't need logging there then it doesn't matter much either way

static bluff
#

๐Ÿ™‚ Thanks!

#

Nothing else jumps out?

halcyon trail
#

Well, Applcation is misspelled ๐Ÿ™‚ Overall it feels very strange to me to be working with classes and instance attributes so much in the init function, like I said before, it feels like Java

#

I don't know whether Controllers et al is code you wrote yourself or if it is some framework that expects to be used in a certain way, if it's the latter then that's just life

signal tide
#

**settings: Union[None, bool, int, float, str] shouldnt that be hinted as a dict?

peak spoke
#

Only if you want your values to be dicts

signal tide
#

doesnt ** return a dict?

peak spoke
signal tide
#

ah

"When using the short form, for *args and **kwds, put 1 or 2 stars in front of the corresponding type annotation. (As with Python 3 annotations, the annotation here denotes the type of the individual argument values, not of the tuple/dict that you receive as the special argument value args or kwds.)"```
#

nvm then

halcyon trail
#

yeah I had to look that up because I don't usually hint settings

static bluff
#

But general, is Python not an object oriented language which should therefore take advantage of an object oriented approach? What would you do differently?

halcyon trail
# 
def main():
    ...
#

I would say that python is a mixed paradigm language. You use OO when it makes sense, use procedural when it makes sense, etc

signal tide
#

are you just using Controllers as a namespace?

halcyon trail
#

@signal tide Also, I looked it up, sadly there is no way to hint **kwargs properly, which is pretty surprising

#

there's no way to currently use a typed dict, in particular

static bluff
halcyon trail
#

Err, if you have circular imports, more likely you want to target that problem directly

signal tide
#

i agree it seems like you might have an issue with your project structure

static bluff
halcyon trail
#

it's not about "wrong with". Circular dependencies aren't good, all other things being equal. That doesn't mean it's never the right decision, but it rarely is.

#

I also don't know what you mean by a "toolkit"

static bluff
#

For example, the RuntimeAuditor needs to be able to use the console to log the runtime details, and the console needs to access a few attributes on the auditor - just as a formatting thing

halcyon trail
#

So usually it'd be better to "break out" some of that stuff so that you can avoid the circular dependency

#

in one or even both directions

signal tide
#

do you need separate controllers for each object?

halcyon trail
#

If the auditor needs to be able to log, don't give it the whole console, or any of the console in fact

#

just use the standard hierarchical logging approach, logger = logging.getLogger(__name__)

#

If the console just needs a few attributes from the auditor, don't make console depend on auditor; just take in the attributes you need as arguments. Then whoever is creating both auditor and console can pass those arguments along.

static bluff
#

Functionally, I see little difference between that route, and storing the logger in a controllers namespace and referencing within the auditor controllers.console.whatever()

halcyon trail
#

Not sure what that's in response to

static bluff
#

the first one ๐Ÿ˜›

halcyon trail
#

still not sure...

static bluff
#

Basically, here's my thought

paper echo
static bluff
#

I've broken my API, public and private, into small manageable pieces - utility objects I loosely call controllers. For every group of similar functions theres a controller, you know, a toolbox object. Its object oriented, clean, organized. But it makes perfect sense to me that a) different parts of the API are going to need to be able to access each other all the time and b) that a namespace for storing all these utility objects so 'the api' can be accessed from outside my programs internals seems, I dunno, useful?

#

I recognize that circular imports and a big warning sign especially when they sneak up on you, but I've organized things this way on purpose. It makes perfect sense to me. That said, I'm on here learn from you all

halcyon trail
#

the "namespace" is the package system

#

I have to be completely honest. You say "you're here to learn from us all", but it usually feels more like you already made up your mind and you want a bullet proof thesis to convince you to do it another way

unkempt rock
#

is this good code layout or spaghetti coed?

unknown.png
halcyon trail
#

I'm not going to provide you with said bullet proof arguments, it's too time consuming

unkempt rock
#

someone give an opinion on my code pls!

static bluff
#

Well I've got to be honest with you quick, I've tried to implement a lot of what you've suggested, only to come full circle after others have told me that your approach is backward- or too pure at least

unkempt rock
#

:|

quasi hound
#

looks okay to me

halcyon trail
#

Not sure who the "others" are, I pretty much try to stick to not-very-controversial suggestions with you (because even those become controversial)

static bluff
#

When I say I'm here to learn, tbh my man, learn from everyone but you. You're also really, quite unpleasant. If we're being honest

halcyon trail
#

Saying something like "just use logger = logging.getLogger(__name__)

#

this isn't some crazy opinionated thing, this is just the 99.9% standard way of using logging

static bluff
#

Save it quick, we've given each other enough of our time

sly trout
halcyon trail
#

dude... you never gave me anything lol

#

Just be clear on that

#

Nor do you get to tell people to save it in a public channel. Be clear on that too.

static bluff
#

Reaching out to you specifically to aask your opinion, spending a good hour talking with you, and then going away and spending a week trying your approach only to be walked back from it by others on here isn't giving you some of my time?

quasi hound
#

i got a quick question/problem that keeps biting me in the ass, does anyone know how i can get around this:
(this is just a quick example but same concept)

class Person:
  def __init__(self, name: str, friends: List[Person]):
    self.name = name
    self.friends = friends

everytime i do something like this i see class Person is not defined on line 2 and im betting there's something in the typing library that can help or something like that

halcyon trail
#

No, it's not. I have nothing to gain from our interactions. I was trying to help you, that is all. "Giving" implies that the other person is "getting" something

paper echo
static bluff
#

Like I said quick, save it ๐Ÿ™‚

halcyon trail
#

Like I already said, that's just not a thing you can say here

quasi hound
#

uh anyone know

paper echo
#

And @unkempt rock RotCheck confusingly named, it does not check, it makes sure the rotation is correct. It should be called internally by Rotate

#

@quasi hound use from __future__ import annotations at the top of your file

#

Or use List['Person'] instead

quasi hound
#

isn't future for python2 only

paper echo
#

No

static bluff
#

For the record, my setup is quite similar to that of wxPython, a framework with a long history and a good reputation. And most of the people I talk to have no problem wrapping their minds around it. Yet, most of the people I talk to have no idea what you're talking about quick, when I relay what I've tried to learn from you. I know its probably very rude of me to say this, but I'm pretty sure its you whose programming from a minority perspective. And I'm certain its you whose not trying to have an open mind. I certainly am trying. Only say this because you've actually made me feel like I'm a total idiot in the past, only to speak to, like, anyone else on here pretty much and be told that while I've got some strange habits, my approach is sound

halcyon trail
#

If you talk to people that think that having a bunch of top level objects in some namespace all depending on each other in circular fashion is sound, then great, I guess you can decide to listen to those people. Not sure what I can add to that.

static bluff
#

No, me neither.

halcyon trail
#

great, let's stay on topic going forward

unkempt rock
unkempt rock
boreal umbra
unkempt rock
#

also sry i did not know

blissful comet
#

Looking for feedback on advanced array slicing. We're debating if it should be included in the subset of python supported by py2many (transpiler). Any data on how prevalent it's use is?

https://twitter.com/arundsharma/status/1415038272183488513
https://github.com/adsharma/py2many/issues/422

Arun Sharma (@arundsharma)

Advanced array indexing in https://t.co/IPWtp4wDiG as in:

a[1:4] or a[:-1]

Twitter ยท
GitHub
py38 & cpp: [:-1] causes: Advanced Slicing not supported ยท Issue #4...

It seems this syntax is producing an exception only on pycpp, and only on py38 GIT_PAGER=cat git grep '[:-1]' py2many/scope.py: return ScopeList(self[:-1]) pycpp/plugins.py: retur...

halcyon trail
#

I don't have any data. I can just offer that in my personal usage, negative indices into slices is very very common

#

the three argument form of slicing, I almost never use

flat gazelle
#

I have only ever used the 3 arg form of slicing for verifying tic tac toe wins, but 2 args slices with negative indicies I have done quite often.

halcyon trail
#

especially in strings, it's just super common. "give me the last 4 characters" s[-4:]

peak spoke
#

I'd say almost all uses of step will be reversing, but negative indices are very handy and commonly used

flat gazelle
#

there are over 136 instances of it in the django codebase

blissful comet
scarlet plover
#

hello

#

am new here

sly trout
# unkempt rock ๐Ÿ”จ

Not what i really meant. Let me explain:
You should name your functions in a snake_case format def rot_check:, but you use CamelCase format that is for class class Newclass:
Check this out: https://www.python.org/dev/peps/pep-0008/
If you are lazy to follow pep standards - other developers won't read your code.
Your code itself looks fine, but try too follow standards. Use flake8 and black for that.

Python.org
PEP 8 -- Style Guide for Python Code

The official home of the Python Programming Language

unkempt rock
#

but RotCheck() looks nice tho

sly trout
unkempt rock
#

is this annotation local to python?

flat gazelle
#

yes, C# has a different naming convention than python

unkempt rock
#

oh

#

oof

flat gazelle
#

few languages have the same naming conventions

unkempt rock
#

i will change it

halcyon trail
#

at least python has a unified convention, C++ people can't agree about anything ๐Ÿ˜›

charred pilot
#

yeah, it's nice that pep8 exists

#

same with rust, you just cargo fmt and you're done

halcyon trail
#

most languages these days have auto formatters

sacred yew
#

but most langs dont have one specific style guide

#

unlike pep 8

verbal escarp
#

anyone know the reason why (1,2) != (1,2,None)?

#

and even weirder (1,2, None) < (1,2)

#

=> False but (1,2, None) > (1,2) => True

#

None > nothing ๐Ÿ˜’

unkempt rock
#

Tuples comparision first compares each individual elements, if they're all equal, the lengthier tuple is "greater"

peak spoke
#

Not sure why the tuples should be equal

prime estuary
#

It's the same rules you use to sort words, "lexographically".

verbal escarp
#

because None stands for nothing?

peak spoke
#

It's still an additional element in the tuple

prime estuary
#

Same applies regardless what the 3rd element is.

verbal escarp
#

if i wanted to compare length, i'd use len(tuple), but i'm trying to compare values of "parts"

#

not sure what the correct terminology is here

#

so i'd expect a comparison of nothing to None

unkempt rock
#

You can filter out the Nones before comparing, but it would be odd that indexing two equal tuples might give back different results

prime estuary
#

Here's the description in the language reference:
https://docs.python.org/3/reference/expressions.html#value-comparisons

For two collections to compare equal, they must be of the same type, have the same length, and each pair of corresponding elements must compare equal (for example, [1,2] == (1,2) is false because the type is not the same).
Collections that support order comparison are ordered the same as their first unequal elements (for example, [1,2,x] <= [1,2,y] has the same value as x <= y). If a corresponding element does not exist, the shorter collection is ordered first (for example, [1,2] < [1,2,3] is true).

verbal escarp
#

is there a mathematical reason for this behaviour or is that a shortcut?

prime estuary
#

Well with these rules any set of tuples will be sortable and behave consistently if their members do.

verbal escarp
#

except for those that aren't

#

like version numbers

prime estuary
#

No, those behave consistently, and for versions it's what you want.

#

All (2, 7, X) versions are before (3, 0, 0).

verbal escarp
#

yeah, but only if you have them neatly in a 0-padded tuple

unkempt rock
#

No? The length doesn't matter if some elements don't compare equally

# 
In [14]: (2, 7, 0) < (3,)
Out[14]: True
#

The length only comes into play when all coressponding elements are equal

prime estuary
#

If it was (2, 7, 0) < (2, ), then the corresponding elements are equal, so the second goes first.

unkempt rock
#

The first would go first since it's longer

verbal escarp
#

well, if you parse things, you might end up with (2,3) < (2,3, None)

#

just as an example

prime estuary
#

Sure, in that case you'd want to avoid doing that.

verbal escarp
#

you could argue that in this case one could work with 0-padding the tuples, but versions can have an infinite number of parts by definition

prime estuary
#

None is not "nothing".

#

If you want a different comparison behaviour, you could do it yourself, or make a class that implements something different.

verbal escarp
#

so one would need to check the length of the tuples to get the padding right.. sigh

prime estuary
#

This is what tuples, lists and strings all do, it was chosen to be consistent.

verbal escarp
#

yay for consistency.. if only versions were invented back then

unkempt rock
#

I think this behaviour feels natural, especially when you're sorting collections

prime estuary
#

Well actually sys.version_info is a tuple of the Python version, and is used for comparisons often...

verbal escarp
#

exactly my problem ๐Ÿ™‚

#

and it doesn't adhere to the versioning convention

prime estuary
fallen slateBOT
#

Objects/tupleobject.c lines 684 to 689

/* Note:  the corresponding code for lists has an "early out" test
 * here when op is EQ or NE and the lengths differ.  That pays there,
 * but Tim was unable to find any real code where EQ/NE tuple
 * compares don't have the same length, so testing for it here would
 * have cost without benefit.
 */```
verbal escarp
#

which packaging Version tries to follow, which sucks

prime estuary
#

It usually would be the case the lengths would match for tuples.

verbal escarp
#

so it's a case of corner cutting :p

#

oh well.

peak spoke
#

what the comment mentions doesn't change the behaviour apart from its performance characteristics

verbal escarp
#

well, it might make a difference with None

#

then you could indeed say None stands for "nothing" and compare true

prime estuary
#

Nothing is like any other object.

unkempt rock
#

It'd be horrible if None didn't make a difference in comparisions, (None, 1, 2, None, None) is quite different from (1, 2)

In [7]: a = (1, 2)

In [8]: b = (1, 2, None)

In [11]: print(b[2])
None

In [12]: print(a[2])
---------------------------------------------------------------------
IndexError                          Traceback (most recent call last)
<ipython-input-12-434ad6561f14> in <module>
----> 1 print(a[2])

IndexError: tuple index out of range
``` also, equal tuples not producing the equal elements when you fetch the same index
#

Sounds like a question that'd fit in a help channel or #discord-bots if you're working with the Discord API, this channel is for discussions about the Python language itself

bold terrace
#

Ok

#

Im Sorry

verbal escarp
#

it's basically the same for left-0-padding on integers

flat gazelle
#

Still breaks tup[-1]

#

You can't index into integers by digit

verbal escarp
#

001 is the same as 1, but 100 is not the same as 1

unkempt rock
#

None is still a meaningful object in Python, even though it's usually used to represent "Nothing"

verbal escarp
#

not arguing that

#

but the meaning changes with its position, just like 0

unkempt rock
#

That's only for versioning though, isn't it? There are libraries to handle versions

verbal escarp
#

packaging should handle versioning, but i'm not going into that ๐Ÿ™‚

flat gazelle
#

!e

from collections import namedtuple
Version=namedtuple('Version','major minor patch' ,defaults=[0,0,0])
print(Version(major=3, minor=6)==Version(3,6,0))
```namedtuple gives you a way to do this neatly enough
fallen slateBOT
#

@flat gazelle :white_check_mark: Your eval job has completed with return code 0.

True
verbal escarp
#

actually, packaging Version uses a namedtuple behind the scenes

#

but it's tricky

peak spoke
#

It really sounds like you'd want the behaviour for a small subset of uses it gets while breaking everything else. Padding the tuple is not particularly difficult while checking for the trailing Nones would need to be done for every sequence and completely tank performance in some cases

flat gazelle
#

The important idiomatic difference is that None is in fact a value and it makes sense to have it as a collection element. Unlike something like raku's slip which represents the absence of a value in a data structure, None is more often than not a meaningful result from an operation and as such it makes sense to consider it a discrete element than just padding.
With regards to numbers, math just ends up creating multiple ways to write the same number.
People complain a lot about Optional.map in java special casing null for a similar reason as to why it would be really bad idea to have

tup==tup2 and tup[-1] != tup2[-1]
```hold for some pair of native tuples.
paper echo
#

doesn't javascript also have undefined as the true "absence of value" object, while null is like python None?

verbal escarp
#

i'd still argue with the similarity to zero padding, where position and context determines meaning

flat gazelle
#

well, even then [undefined, undefined] is different from [], whereas in raku [slip, slip] is indeed empty

paper echo
#

interesting

#

why is it called slip

flat gazelle
#

because it slips into the array and you can construct some that actually hold elements, for eaxmple [slip(1, 2, 3), slip] is [1, 2, 3]

verbal escarp
#

what purpose does that serve?

flat gazelle
#

it means you don't have to [x for y in z for x in [y + 1, y - 1]] and instead do @z.map(-> $y {slip($y - 1, $y + 1)})

#

it is a convenient thing, but it makes things extremely complex to specify

#

currently, in python (a, b) is pretty obvious in meaning, but if trailing Nones were ignored in some form, the semantics would be a bit more complex

#

and I mean, 0 padding is a syntax error in literals, so it is just int that considers that if it is given a specific base (10 by default), and there it is useful so that int('00001111', 2) works

dusty verge
#

Let me guess, you are a lua programmer amogorkon?

verbal escarp
#

nope

dusty verge
#

What language are you getting this null padding from

verbal escarp
#

i'm pythonista through and through

#

maths

dusty verge
#

ah, fair enough haha

verbal escarp
#

๐Ÿ™‚

paper echo
#

i assume -> $y { } is a lambda

flat gazelle
#

yes

#

a lambda taking $y as a parameter

lusty scroll
#

what language is that?

flat gazelle
#

raku

lusty scroll
#

oh ok, looks fairly Perl-like

flat gazelle
#

it is what became of perl 6

verbal escarp
#

i assume -> $y { } is a lambda <- is that equivalent to lambda y:..?

#

or is there a difference in what you can do?

flat gazelle
#

yeah, it is about the same

arctic ledge
#

I get this error please help usage: convert.py [-h] --labels LABELS [--noviz] input_dir output_dir convert.py: error: the following arguments are required: --labels

#

I tried this way

#

python convert.py input_dir C:\Users\Raghava\Downloads\cocojson output_dir C:\Users\Raghava\Downloads\cocomxl

#

This is the file I want to run

paper echo
paper echo
flat gazelle
paper echo
#

but what would a "2 element slip" be?

#

if slip is "absence of something"

#

i'd be happy to read the raku docs on this if you have a link

#

i've never seen this concept in any other language

flat gazelle
#

I think I slightly misexplained slip. slip constructs an empty slip, which is a representation of the lack of value, but a slip may contain 0 or more values and it will slip those into whatever data structure it is part of.

halcyon trail
#

I can't see how you're getting the null padding thing from maths. From a math perspective, tuples of different sizes are elements of different spaces, you can't say that they're equal without having some kind of mapping between the spaces

#

at least, that's the common way in which you'd discuss product spaces. If you want to have a space that includes tuples of all sizes, you can of course do that, but you'd still need to define equality
Maybe I just don't know the math you're thinking of; if you're thinking of e.g. category theory, I dont' have any background in that.

paper echo
flat gazelle
#

yes

paper echo
#

that's kind of a wild feature

flat gazelle
#

ikr

paper echo
#

don't want to imagine how that's implemented

verbal escarp
halcyon trail
#

1 == 1,0000?

#

oh, that's a decimal, you're european

verbal escarp
#

yes i am ๐Ÿ™‚

#

sorry for the confusion

halcyon trail
#

nah, it's just as valid as our way, though i really wonder how we ended up doing things the opposite way lol

#

I think it's pretty different because numbers all live in a single space. The digits are just a representation; the actual value of the number is a summation over the digits.

#

With a tuple, there's no implied summation, it's a product space

verbal escarp
#

although one could argue that a number is an infinitely large vector of digits with place values

halcyon trail
#

yes, a vector, not a tuple

verbal escarp
#

the definitions get murky there

halcyon trail
#

and even for a vector, it's not really a vector, it doesn't obey the vector laws, if you consider the digits as components

#

Not really, these things have fairly rigorous definitions. vectors and tuples in math mean fairly specific things.

verbal escarp
#

well, what you can do with either one of them depends on the context

#

so operations like vector multiplication don't have to be defined for all cases

halcyon trail
#

Yes, but a context where a mathematician would say (1,2) = (1, 2, 0) is extremely rare

#

or non-existent

#

it's extremely different from your examples of zero padding numbers

verbal escarp
#

not really, actually. sparse matrices come to mind

halcyon trail
#

err, sparse matrices are usually represented by lists of 3 tuples (assuming a rank 2 matrix)

verbal escarp
#

i'm just saying that "leaving stuff out and assume the rest is zero" isn't all that uncommon in maths

halcyon trail
#

as a general approach, no, but that's different than talking about it in the specific context of tuples

verbal escarp
#

as long as you have a well defined context

halcyon trail
#

part of the definition of tuples, in math, as that they are ordered

#

for two tuples to be equal, the corresponding elements have to be equal

verbal escarp
halcyon trail
#

sorry, I don't follow. But at any rate, I think the existing tuple behavior is pretty reasonable. The good news is that you can write a dataclass which works the way you want, which is better than the tuple even

#

since it has named fields

verbal escarp
#

now you lost me. how would you go about it with a dataclass?

halcyon trail
# 
@dataclass
class Version
    major: int
    minor: int = 0
    patch: int = 0
#

something like that?

verbal escarp
#

ah, defaults

#

problem is that versions can, by design, have infinitely many parts

halcyon trail
#

I haven't seen that many versioning schemes with more than 3 parts, but if you wanted to support such things, then you could do it easily enough with a class that holds a list

#

I'm actually curious where you have seen versions with more than 3 parts

verbal escarp
halcyon trail
#

what's .postx?

verbal escarp
#

well, my point is that even if i would restrict myself to 5 parts, which should cover all regular cases, i would not cover the specification

verbal escarp
#

Public version identifiers are separated into up to five segments:

Epoch segment: N!
Release segment: N(.N)*
Pre-release segment: {a|b|rc}N
Post-release segment: .postN
Development release segment: .devN

halcyon trail
#

interesting. I have almost never seen more than 3 used.

#

At any rate though, it seems to me like you'd want to have a class for this regardless, even aside from this aspect of it, there's also all the special bits of text that can appear or not

#

rc, etc

verbal escarp
#

postN usually is used when you messed up with a detail in setup.py and only notice after hitting the enter button to upload to pypi ๐Ÿ˜„

halcyon trail
#

yeah, I'm just not sure offhand what the real benefit of it is over bumping the patch version in semantic versioning

paper echo
#

not everyone uses semver

#

nor is semver a perfect fit for all development cycles

halcyon trail
#

Well, I didn't say either of those things ๐Ÿ™‚

#

I just asked what the benefit was over bumping a patch

verbal escarp
#

well, the distinction is that it's obvious you didn't fix a problem in the code but with the code release itself

halcyon trail
#

ime, most projects not using semver, are generally using fewer numbers, and a less formal scheme (i.e. one with fewer "hard rules" about when to bump which number). Not more numbers, and more rules, which it seems like this is. Hence my curiosity.

verbal escarp
#

hey, i didn't invent those rules ๐Ÿ™‚ i really wished versioning was as simple as using tuples of integers and that's it

halcyon trail
#

It's an interesting distinction. I guess I'd probably need more field experience to appreciate the trade-offs.

#

Well, you still have RC's

#

so it wouldn't quite be a tuple of integers. unless you think RC's are useless too, which I don't have strong opinions about either.

lusty scroll
#

or 1.0-pre, 1.0-beta, 1.0~<git_commit> ...

verbal escarp
#

packaging Version ignores those parts.. curious

#

Version("1.0-pre2").release
(1, 0)

halcyon trail
#

pretty odd

verbal escarp
#

๐Ÿคทโ€โ™‚๏ธ

paper echo
#

maybe you change the docs or something but not the code

lusty scroll
#

then there's debian style epochs in the version

verbal escarp
#

i noticed Version returns a plain tuple while internally they use a namedtuple, my guess is they ignore the stuff when they return the .release tuple in order to accomodate people who expect a tuple of ints?

#

sigh..

#

i suppose a better approach might be to have a class of VersionDigits that encapsulates those parts and only compares to other VersionDigits

#

without losing the info in the stringy parts

#

or plain refuse to parse to integers, but then sys.version_info becomes deprecated

#

it's a rock and a hard place situation

halcyon trail
#

Using plain tuples for this stuff seems kinda silly, to me

#

I'm not sure how beholden you are to existing stuff

verbal escarp
#

funnily, Version refuses to take plain tuples, so you can't even turn sys.version_info into a Version object directly

halcyon trail
#

I would probably write a class from scratch that holds a list of integers + an enum (for alpha/beta/rc/final)

#

and put all the parsing logic inside its init, write == to work the way I wanted (in your case, if one list is shorter than the other, then the other list has to have all zeros to compare equal)

verbal escarp
#

if it were up to me, i'd go for a tuple of int|tuple(int) or some other well defined, recursive structure ^^

#

so any additional part would really be meaningful

halcyon trail
#

wait, why do you need a recursive structure

verbal escarp
#

well, (1,0) > (1,) - which is weird

halcyon trail
#

if you just had a class that held a list, then you could have those compare equal

#

also, tuple(int | tuple(int)) is not a recursive structure

verbal escarp
#

i know, too lazy for a full spec ๐Ÿ˜‰

#

i meant that the existence of any additional sub-part would hold significance and shouldn't be discarded and must hold up to comparisons

halcyon trail
# 
class version:
    numbers: List[int]
    
    def __eq__(self, other) -> bool:
        common_length = min(len(self.numbers), len(other.numbers))
        if self.numbers[:common_length] != other.numbers[:common_length]:
            return False
        if any(x != 0 for x in self[common_length:]) or any(x != 0 for x in other[common_length:]):
            return False
        return True
verbal escarp
#

nice pictures, blabla text ๐Ÿ™‚

gusty marsh
#

And I donโ€™t think that fits here and can be classed has advertising IMO

sand goblet
#

Is the difference between โ€œfunctionโ€ and โ€œbound methodโ€ that bound method implicitly passes the object youโ€™re accessing it from as the first argument? Or is there more to it than that?

flat gazelle
#

that is the important one

#

but you can also assign arbitrary attributes onto functions, but not onto bound methods

sand goblet
#

Ok thanks, I didnโ€™t know about that

# 
class Test:
    def f(self):
        pass

t = Test()
Test.f.x = "hey"
print(Test.f.x)
print(t.f.x)```
peak spoke
#

It also can't get bound to an another object

sand goblet
#

This works somehow
But not if you try to assign it directly, like t.f.x = "hey", like you said
I guess because itโ€™s just looking it up through the class and not the instance

lusty scroll
#

is there a reason __builtins__ would usually be a module but sometimes a dict?

lusty scroll
hot token
#

which makes sense cause you need an object to bind it to

rough leaf
#

hi

#

just joined

verbal escarp
#

fun fact, i've used ```python
def aspectize(cls, decorator):
for name, func in cls.dict.items():
if not name.startswith("__"):
setattr(cls, name, MethodType(decorator(func), cls))

aspectize(Test, logged)

#

as a first experiment with aspectizing

#

replacing bound methods

halcyon trail
#

what does "aspectizing" mean

verbal escarp
#

you know aspect-oriented programming?

halcyon trail
#

I've heard of it

#

it seems like in this case you're basically applying a decorator to every member function of a class?

verbal escarp
#

yeah

halcyon trail
#

I'd probably expect such a thing to be done via a class decorator

#

although perhaps the implementation would not change too much. i haven't written that many class decorators.

verbal escarp
#

a while ago i ran into problems with pyqt, which were very hard to debug, and running a regular debugger would've killed my GUI, so i came up with the idea to decorate methods with a call to a logger to track down where my problem happened

halcyon trail
#

yeah, this is a first class feature in a lot of lisps

#

called advice there

#

I remember using it for debugging

verbal escarp
#

but those classes were pretty big and i didn't want to decorate all methods, so i came up with aspectizing

halcyon trail
#

emacs ๐Ÿ™‚

verbal escarp
#

track down all callables in a module, check for a pattern and replace it with a decorated version

#

it worked so well that i decided to make it even a feature in https://pypi.org/project/justuse/ ๐Ÿ™‚

PyPI
justuse

A beautiful, simple and explicit way to import modules in Python with advanced features.

#

although, i haven't really gotten the chance to put it to use (no pun intended) yet

#

just mentioned it because replacing bound methods is trickier than i anticipated, but easy once you know how to go about it

#

just don't make the mistake of trying to assign a function directly

halcyon trail
#

You probably should read that, your solution doesn't do, afaics, any filtering to make sure the thing you're replacing is actually a bound method

#

so if there's a class variable for example, things would get weird

verbal escarp
#

well, it gets tricky with properties

halcyon trail
#

Depending I guess whether or not you want to decorate those too, at least they are indeed functions, unlike a class variable.

verbal escarp
#

yeah, i remember i specifically needed those

halcyon trail
#

it seems like a nice trick for logging that I can imagine being useful , I can't really see myself being super excited to use this generally though

#

I remember advice in lisp, very double edged, very useful but also could be a big headache.

verbal escarp
#

i think my first contact with AOP was with java.. aspectj?

#

it sounded like a cool idea back then, but never got the chance to try it, until my debugging problem

#

although, it's a good tool to have in the toolbox, just in case

halcyon trail
#

yeah, I guess in python AOP is somewhat subsumed into the concept of decorators

verbal escarp
#

if a debugger slows down your program too much and you only need to get a glimpse at a specific part of your code

#

although in python people immediately think @ when they hear "decorator"

halcyon trail
#

well, yes, because that's what it's called ๐Ÿ™‚

verbal escarp
#

:p

#

yeah, but you can also decorate a function by calling a decorator with the func as argument, which got a bit burrowed under the pile of syntactic sugar

halcyon trail
#

Sure. I'm not sure if it's buried, I think most, at least, non-beginners understand that's what @ does

#

but obviously @ is how decorators are used 99% of the time

verbal escarp
#

naturally, and for good reason

halcyon trail
#

It's a pretty neat thing. At this point, there aren't a lot of python features that I like a lot that I haven't seen in some other mainstream languages, but python decorators are one of them.

#

Definitely more cumbersome to have an equivalent feature in a statically typed language

verbal escarp
#

especially since they removed those restrictions on those, it's really cool

halcyon trail
#

which restrictions?

grave jolt
halcyon trail
#

madness

#

why do we want that?

verbal escarp
#

the most use i got out of that so far is with callbacks in pyqt

halcyon trail
#

Oh I see

#

okay, you can't have things like [0] there even

#

Yeah, I don't really care, although I imagine it only comes up once in a blue moon

#

I guess that they did this because it was no work; it basically just made the implementation simpler since there was an artificial restriction in place

verbal escarp
#

according to the pep the only reason why the restriction was in the first place was because of guido's gut feeling ^^

#

although, i must confess i also have a weird feeling about it

#

with those decorators, i run into a pattern i haven't seen before

#

it goes something like this

# 
class Foo:
  def __init__(self):
    @self.button.connect
    def _():
      ... do button stuff...
#

i'm torn between leaving those functions anonymous and naming them - anonymous because they never get called from anywhere except as callback, named to better have the IDE assist

halcyon trail
#

I don't understand, what does the decorator do

#

register them as a callback?

verbal escarp
#

yes

halcyon trail
#

that's awful

#

it's like doing [print(x) for x in some_list]

verbal escarp
#

well, otherwise you'd need to register them as callback in one place and define them somewhere else entirely

#

this way, you have both in one place

halcyon trail
#

you can register a local lambda without using lambda syntax

#

sorry, without using @ syntax

verbal escarp
#

yeah, but lambdas can't be as complex as functions, and i need that complexity ๐Ÿ˜„

halcyon trail
#

sorry, I said lambda by accident

verbal escarp
#

no, it's correct, i would absolutely use a lambda if it was possible

halcyon trail
#

my pointer is that you can simply do:

def __init__(self):
    def connect_callback():
        ...
    self.button.register_connect(connect_callback)
verbal escarp
#

ah, as a closure

halcyon trail
#

the whole point of the @ syntax is to re-assign the result, using it purely for side-effect is dubious

#

similar to a list comprehension, it's supposed to be an expression, which is why people tend to frown on "throw away" list comprehensions

#

But personally if the callback is more than 2-3 lines I wouldn't define it locally

verbal escarp
#

well, callbacks by nature are throwaway functions and rarely called by two different things

#

so lambdas would be the natural fit

halcyon trail
#

I don't think that the only criteria to factor out something as a function is that it has to be called more than once

#

it's one good criteria, but not the only one

#

complexity, clarity of naming, testability, etc

#

if connect_callback is 2 lines, then it's very simple and probably the overhead of following it around to a new location is more than just understanding it at the site of init

#

if connect_callback is 100 lines, then I want it as a separate function, I don't really care if it's only called once

verbal escarp
#

sure, but you'd also want the definition as close as possible to where you register the function usually, less jumping around in the code

halcyon trail
#

Sure, that's a trade-off as well. But again, the length of the thing is pretty important here. If it's 100 lines and it has a clear name, it's better to just see the name, so you can have an idea what it does. Reading the 100 lines and understanding them is going to be a lot of work anyway, so the overhead of goto definition and back in your IDE becomes negligible

verbal escarp
#

agreed

halcyon trail
#

At any rate I find that decorator pattern with the function named _ kinda weird. Maybe if it caught on more widely, I'd be more accepting.

#

Right now it's in the same bin as [print(x) for x in my_list] for me

verbal escarp
#

yeah.. as i said, i'd prefer a nice and clear lambda syntax over that def _()

#

it's kind of weird to me, too

halcyon trail
#

Yeah, with nice lambdas you could just do

def __init__(self):
    self.button.register_connect {
        ...
    }
#

but alas

#

this is python ๐Ÿ™‚

paper echo
#

in a metaclass, i want to perform a certain check on subclasses of a certain base class, but not perform the check on the base class itself. this was my workaround for the fact that the class doesn't actually exist yet when i'm doing the check.

i currently have this ugly hack. is there a less ugly way to do it?

class _DocumentBase:
    """Base class for Document. Ugly hack to get the check in DocumentType to work."""
    pass

class DocumentType(type):
    def __new__(meta, name, bases, namespace):
        # Only Document itself should have this exact `bases`.
        is_base_document_type = bases == (_DocumentBase,)
        if not is_base_document_type and "collection_name" not in namespace:
            raise TypeError(
                "Document classes must have a 'collection_name' class attribute."
            )
        cls = super().__new__(meta, name, bases, namespace)
        return attr.s()(cls)

class Document(_DocumentBase, metaclass=DocumentType):
    """Base document class."""
    collection_name: ClassVar[str]
verbal escarp
#

i must confess, i never actually worked with metaclasses like that before, usually i'd rather go for composition or token-based patterns

wide shuttle
#

In your hack, you assume that there are no other base classes, going by your exact tuple comparison

#

Couldn't you forgo the additional sentinel base class and check if bases is an empty tuple?

verbal escarp
#

couldn't you check the mro to check where you are and act on that?

paper echo
#

that's a good point @wide shuttle , i could make DocumentType itself "private" and assume the user doesn't have access to it for subclassing

#

@verbal escarp bases kind of is the MRO

#

i guess i could create the class, then check its MRO, but given that i'm already assuming i'm at the "top" then it's no different

#

top? bottom?

verbal escarp
#

heh

halcyon trail
#

When I ran into this problem I just checked the name of the class ๐Ÿคทโ€โ™‚๏ธ

#

probably a little worse than what you did

#

what's a "token based pattern" ?

verbal escarp
#

well, my thought was that it might be more robust for subclassing to check the mro, but i don't know

halcyon trail
#

Checking if bases is an empty tuple, and making the meta class private, isn't so great

#

It's good to have a convenience base class for using metaclasses but its' not composable if the user has another metaclass in mind

#

I went through this very recently with ABC where I needed ABCMeta

paper echo
#

that's a good point

halcyon trail
#

You could of course just say "I don't care if you want to compose with another metaclass" but that's not so hot IMHO

paper echo
#

in this particular case it shouldn't matter much because this is supposed to be a django-like API

#

writing ABCs for these would be really strange

#

and it's for an internal tool anyway so i'd be willing to just say "that's not supported"

halcyon trail
#

Well, idk, maybe they are using an ABC to do some kind of special registration so they create the classes by factory, etc etc

paper echo
#

but it would be nice to not break things without needing to break them

#

could do it with a decorator too like attrs

halcyon trail
#

I've only used metaclasses a handful of times so I cannot claim to be an expert on their use cases ๐Ÿ™‚

paper echo
#

in fact in this particular case the metaclass wraps attrs anyway

#

maybe i'll just wrap @attr.s

halcyon trail
#

My 0.02, I'd probably rather use a decorator then a metaclass

#

all other things being equal

#

One thing you should be aware of, you may or may not care about, but

#

wrapping @dataclass or @attr.s in even the most trivial way "breaks" mypy

paper echo
#

yeah, mypy i think has attrs hard-coded

halcyon trail
#

yeah

verbal escarp
#

@halcyon trail token based patterns.. imagine the flyweight pattern where each instance only holds a set of trait tokens (enums) which determines whatever properties this thing has and allow methods only if this or that trait

paper echo
#

pyright uses some kind of "dataclass protocol" that attrs now suppoorts too

halcyon trail
#

I was very very very sad when I discovered this. You can't even write your own decorator to change defaults in dataclass/attrs

#

that seems like a fairly odd pattern, to me

verbal escarp
#

i had a lot of fun with that pattern, actually

halcyon trail
#

Instead of having a token T with values A, B, C, why not just choose between mixins A, B, C?

verbal escarp
#

well, benefit is you can switch tokens easily and composition is well defined (maybe use a frozenset as key)

paper echo
#

was it here where people were talking about haskell/rust-like typeclasses/traits in python? there's a library for it... seemed kind of misguided

#

but there were some benefits that i don't remember

halcyon trail
#

It's not just composition though, you said that the tokens are determining what public properties it has

#

that's affecting the public API

paper echo
#

@verbal escarp what would this pattern look like in python?

halcyon trail
#

@paper echo it does seem pretty bizarre to me. But then, customizing behavior "per type" is often surprisingly clumsy in python. I know I've used @singledispatch for this before with mixed feelings

paper echo
#

oh that's what it was

#

i was like a super powered version of singledispatch that worked with protocols and had some other nice features

#

but it was weirdly couched in terms of "hey dont you wish you had rust traits in python"

halcyon trail
#

Yeah. I can kind of understand desire for that.

#

Right, that is a weird way to sell it.

verbal escarp
paper echo
#

i do kind of like the idea of being able to imbue a class with extra behavior without modifying the class itself, and of grouping them together under a single name (although structural typing kind of reduces the need for grouping)

halcyon trail
#

yeah, I mean the extra behavior without modifying the class itself is one of the biggest criticisms of classical inheritance

paper echo
halcyon trail
#

and it's genuinely a real headache in some cases

verbal escarp
halcyon trail
#

and easily my number one issue with Kotlin. By a large margin.

paper echo
#

that seems like a lot of indirection without the benefits of typeclasses/traits

#

unless the observer is global and all method calls on all instances of all classes get routed there

halcyon trail
#

It's a lot more dynamic than typeclasses/traits. Typically a lot more dynamic than you really need.

#

IMHO

paper echo
halcyon trail
#

If you compare the use cases where you usually see decorators and mixins, it's quite "static", which is intentional

#

decorators/mixins/metaclasses

verbal escarp
#

well, imagine a thing that may change behaviour on the fly, like a character in a game that holds different items

paper echo
#

the name typeclass i feel is so off for this library

halcyon trail
#

Yeah, I can imagine that being useful, but at that point it's just not really an alternative for the things that were originally being discussed

paper echo
# 
from classes import typeclass

@typeclass
def greet(instance) -> str:
    ...

@greet.instance(str)
def _greet_str(instance: str) -> str:
    return 'Iterable!'

greet(1)

is greet really a "typeclass"? i'd sooner call it a generic method.

#

i guess my main gripe is the name

halcyon trail
#

the thing originally being discussed is the ability to make a class a "document", which is probably always going to be a fixed, static, decision, per type

paper echo
#

yes, this is for an in-house mongodb ORM/ODM

halcyon trail
# 
class MyDocument(Document):
    ...

@Document
class MyDocument
    ...

class MyDocument(metaclass=MetaDocument):
    ....
paper echo
#

i wanted to build it on top of attrs instead of going the descriptor route like django

halcyon trail
#

in all 3 approaches, it's totally static

#

Out of curiosity why do you actually need to add stuff

paper echo
halcyon trail
#

why not just write free functions that process attrs classes

paper echo
#

because reasons

halcyon trail
#

it's easily the simplest approach and also the one that won't drive mypy crazy

paper echo
#

trust me, i wish we could

halcyon trail
#

Well, if you are stuck with that, as annoying as it seems, you may want to keep it as a second decorator

# 
@Document
@attr.s
class MyDocument:
    ...
paper echo
#

i don't think we're necessarily stuck with it. but it will be a harder sell because people are used to traditional inheritance-based OOP

halcyon trail
#

well, decorators are not inheritance based, of course, but yes, they will let you use member function syntax

verbal escarp
#

oh, there's that

#

OOP ๐Ÿ˜

halcyon trail
#

member function syntax though is actually a bit risk for a type where you're going to do define arbitrary properties

paper echo
#

but even if we do go the attrs or pydantic route, we still want some fixed connection between the underlying mongodb collection and the class definition

#

there is some additional logic to be had

verbal escarp
#

i wished OOP worked :p

paper echo
#

e.g. on saving to db, we want to update the instance an updated_at timestamp

halcyon trail
#

Yeah, you can do that with a free function easily though

paper echo
#

we want to be able to get a mongodb collection and as you see here i want to enforce that document classes state a collection name

#

yeah of course, you definitely can

#

but it gets a bit funky because now saving mutates an argument

halcyon trail
#

Sorry not trying to argue, if you can't convince people with veto power, then you can't, that's life

verbal escarp
#

yeah

halcyon trail
#

Well, it mutates an argument anyway... it's just in one case, that argument may be self ๐Ÿ™‚

paper echo
#

no its not about veto power, but im not entirely convinced that free functions instead of traditional classes make python better

halcyon trail
#

x.save() vs save(x)

paper echo
#

right, but mutating self is a lot less "unusual"

halcyon trail
#

oh, that's a pretty odd take tbh

#

People have been complaining about this in Java for ages

paper echo
#

in fact thats the main reason i like traditional OO style, because you mutate self and don't fucking mutate anything else

#

syntax follows usage

#

i am probably weird in this

halcyon trail
#

well, traditional OO languages don't have any safeguards against mutating any other argument. And there are also issues, what if two things need to be mutated

#

IMHO the better solution is that the type system simply enforces what can cannot be mutated

paper echo
#

i can't remember the last time i've had to mutate 2 things ๐Ÿ˜› but that is a great point

halcyon trail
#

and then it's also evident from the function signature. Whether it's a member or a free function

verbal escarp
#

i find x.save() harder for testing than save(x)

halcyon trail
#

The lack of control over mutation is a pretty big issue IMHO in many mainstream languages, OO doesn't really do much to mitigate it

verbal escarp
#

not harder, just more work

halcyon trail
#

Hmm, why do you find it harder for testing? I actually don't see much difference.

paper echo
#

i agree, its more just the semantic/syntactic separation

#

@halcyon trail you might have to do monkeypatching or mocking with x.save(), arguably less so with save(x)

verbal escarp
#

well, you need to set up an instance etc. first

halcyon trail
#

Maybe I'd agree, if writing a class from scratch, all other things being equal, things that mutate the object, I'd prefer as member functions

#

the problem is, in this case, not all things are equal

verbal escarp
#

while with save(x) you could just pass in a mock

halcyon trail
#

That's a fair point I suppose, though I'm usually only mocking a very small fraction of classes

#

@paper echo not all things are equal in the sense that mixins, decoratrs, and metaclasses are all much, much more complicated pieces of python machinery than free functions.

paper echo
#

that is also true

halcyon trail
#

So, for me personally, I'd really need to see some concrete benefits before going that way, just "mutators should be members" isn't really enough

paper echo
#

i think there's also encapsulation and namespacing benefits

#

i know there are free-function equivalents of course

halcyon trail
#

Yes, the encapsulation benefit is keeping it as a free function ๐Ÿ˜›

paper echo
#

and i wouldn't really want to "port" this reasoning to a language that isn't already OO-first

#

there's a reason i like Julia after all

halcyon trail
#

https://embeddedartistry.com/fieldatlas/how-non-member-functions-improve-encapsulation/

Embedded Artistry
How Non-Member Functions Improve Encapsulation - Embedded Artistry

11 March 2020 by Phillip JohnstonWeโ€™ve received permission from Scott Meyers to re-publish this article on our website after the version on Dr. Dobbs went down. Iโ€™ve taken the liberty of converting some of the bold print to code font, as well as clarifying some code example indentation/bracketing. Even if you donโ€™t completely agree with โ€ฆ Contin...

#

If you're interested

paper echo
#

side note: specifically annotating things as mutable could be a pretty easy win for mypy/typing. imagine

@attr.define
class Point2d:
    x: float
    y: float

def copy_to(src: Point2d, dst: Mutable[Point2d]) -> None:
    dst.x = src.y
    dst.y = src.y
halcyon trail
#

Scott Meyers, if it rings a bell, wrote about this topic, probably like 20 years ago, it's pretty interesting

paper echo
#

i don't know the name but this is a good point

Encapsulation is a means, not an end.

halcyon trail
#

Well, unfortunately a Mutable annotation like that for a type, doesn't give you much, in the general case

#

With something like a dataclasses, where it's obvious what mutates and what doesn't, ok, it would help

#

but say you have a real class

paper echo
#

without Mutable, assigning anything to any attribute of the class would be a type checking error

#

maybe it needs a better name

halcyon trail
#

Yeah, but you can have member functions that perform mutations

#

how do you know if you can call those members?

verbal escarp
#

hmm.. i'd rather have annotations for side-effects

paper echo
#

right, maybe it needs a better name ๐Ÿ˜›

halcyon trail
#

The answer is, of course, now the members need that annotation

#

you need to know which member functions are mutating, which are not

paper echo
#

i like that

halcyon trail
#

congrats, now you've re-invented C++ const ๐Ÿ™‚

#

and Rust's mut

#

that's how they both work

verbal escarp
#

"behold, this function is mutating stuff" and "this function shouldn't mutate anything" - give a warning if it does

paper echo
#

don't worry, i explicitly took inspiration from mut

verbal escarp
#

i think many people would be surprised to get a heads up for side-effects if they didn't expect any

halcyon trail
#

it makes me sad that const/mut hasn't made any real headway into GC languages

paper echo
# 
_Thing = TypeVar('_Thing', bound='Thing')

@attr.s
class Thing:
    foo: int

    def inc_foo(self: Mutable[_Thing]):
        self.foo += 1
#

i'm into it

halcyon trail
#

you need better syntax though, nesting everything in Mutable like that is awful

paper echo
#

yes

halcyon trail
#

Anyhow. But in GC langauges, the solution instead are basically "partial" interfaces, i.e. non-mutating ones

#

So you have Sequence, and MutableSequence

#

Mapping, MutableMapping

#

Python, Java, Kotlin, probably C#, do stuff like this

paper echo
#

i will say, crystal breaks exactly 0 ground in this area

#

it doesn't even have the interface hierarchy

verbal escarp
#

never heard of crystal

halcyon trail
#

Yeah. It's pretty sad because it's trying to repurpose one tool to solve a different problem, a problem that's really really important and deserves its own first class feature

#

That said, maybe the feature is just immutable data structures anyway

paper echo
#

i was going to say

#

languages that have immutable-first data structures are doing fine here

halcyon trail
#

realistically there are multiple languages whose immutable data structures will beat the pants, speed wise, off python's mutable ones, in general usage

#

Yeah, but again, almost no mainstream languages really

#

for some definition of mainstream

paper echo
#

oh wait

#

nim has this

halcyon trail
#

exactly ๐Ÿ™‚

paper echo
verbal escarp
#

from my pov python could be immutable-first, if there was a push for it

paper echo
#

nim is the only non-"functional" language that i know of with this feature @halcyon trail

#

unless D also has it

#

i believe D does have something like it actually, with its pure annotation

# 
immutable int[3] table = [6, 123, 0x87];
halcyon trail
#

yeah, so D has it

#

D tries really hard to be all things to all people

#

so it has GC, it has @nogc, it has const, but it also has immutable, etc

#

it has all things, except uptake

paper echo
#

lol yep. too big

#

i think the main selling point is that C++ is still even bigger and D is better on a "per feature" basis

#

as in, goodness / num_features is still supposed to be higher for d than for c++

verbal escarp
#

and then came rust

halcyon trail
#

Yeah, it hasn't really worked out well though. C++ still has a bigger community, better tools, better libraries, and no worrying about will the GC turn on or off, etc

paper echo
#

yep