internals-and-peps

[lusty scroll]

im still learning that stuff

!d aiter

[fallen slate]

aiter

aiter(async_iterable)```
Return an [asynchronous iterator](https://docs.python.org/3/glossary.html#term-asynchronous-iterator) for an [asynchronous iterable](https://docs.python.org/3/glossary.html#term-asynchronous-iterable). Equivalent to calling `x.__aiter__()`.

Note: Unlike [`iter()`](https://docs.python.org/3/library/functions.html#iter "iter"), [`aiter()`](https://docs.python.org/3/library/functions.html#aiter "aiter") has no 2-argument variant.

New in version 3.10.

[lusty scroll]

i have a piece of code that seems to encoumter random time between tasks complete (that aren't awaited) and was wondering what the mechanism there is

like how it decides when to do what

[pliant tusk]

!e there are some use after frees inside of the implementation of memoryview. they wouldnt be common to hit, should i bother reporting them? ```py

memoryview Use After Free (memory_ass_sub)

uaf_backing = bytearray(bytearray.basicsize)
uaf_view = memoryview(uaf_backing).cast('n') # ssize_t format

class weird_index:
def index(self):
global memory_backing
uaf_view.release() # release memoryview (UAF)
# free waf_backing memory and allocate a new bytearray into it
memory_backing = uaf_backing.clear() or bytearray()
return 2 # ob_size idx

by the time this line finishes executing, it writes the max ptr size

into the ob_size slot of memory_backing

uaf_view[weird_index()] = (2 ** (tuple.itemsize * 8) - 1) // 2
memory = memoryview(memory_backing)
memory[id(250) + int.basicsize] = 100
print(250)```

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

100

[pliant tusk]

its because memory_ass_sub only checks if the view is released at the very beginning, and index gets called after that

[feral island]

yes, please report them. these are potential security issues

[pliant tusk]

👍

should i use Bug report or Security Vulnerbility?

[feral island]

I guess let's start at Bug

[pliant tusk]

https://github.com/python/cpython/issues/92888 let me know if i need to add anything else to it

im building 3.12 to test it on there rn, but it should exist (based on the source code)

[quick night]

!e

from sys import getsizeof as gf

a = [1, 2, 3]
b = []
b.append(1)
b.append(2)
b.append(3)
print(f'{gf(a) = }, {gf(b) = }')

why are they not equal? does list.append try to resize the vector after appending?

[fallen slate]

@quick night :white_check_mark: Your eval job has completed with return code 0.

gf(a) = 120, gf(b) = 88

[feral island]

there's a bunch of heuristics for how much space is allocated for lists. I'd have expected the .append() call to allocate more though

[west tinsel]

Don't know the specifics, but python overallocates when appending to lists so it doesn't have to allocate each time an item is added

[quick night]

so it like allocated more memory in the first append, then added the other elements in that memory?

[west tinsel]

Yup

!e

import sys # provides getsizeof function
data = []
for k in range(10):
    a = len(data) # number of elements
    b = sys.getsizeof(data) # actual size in bytes
    print("Length: {0:3d}; Size in bytes: {1:4d}".format(a, b))
    data.append(None) # increase length by one

[fallen slate]

@west tinsel :white_check_mark: Your eval job has completed with return code 0.

001 | Length:   0; Size in bytes:   56
002 | Length:   1; Size in bytes:   88
003 | Length:   2; Size in bytes:   88
004 | Length:   3; Size in bytes:   88
005 | Length:   4; Size in bytes:   88
006 | Length:   5; Size in bytes:  120
007 | Length:   6; Size in bytes:  120
008 | Length:   7; Size in bytes:  120
009 | Length:   8; Size in bytes:  120
010 | Length:   9; Size in bytes:  184

[quick night]

ah ic

[pliant tusk]

@feral island yea 3.12 has the same issue

[west tinsel]

https://github.com/python/cpython/issues/92868

Configuring with --enable-optimizations seems to make it run slower for me??

| Test Case | Normal Runtime | Optimised Runtime |
|-----------|----------------|-------------------|
| Match     | 2.96s          | 6.50s             |
| Logical 1 | 1.65s          | 3.48s             |
| Logical 2 | 1.62s          | 3.37s             |

My process is ./configure && make clean && make -s -j6 then benchmark, then ./configure --enable-optimizations && make clean && make -s -j6

Then benchmark again

[rugged harbor]

might some who knows how python imports work pitch in here?
#help-broccoli #help-broccoli message

[rose schooner]

it got backported to 3.10

[rose schooner]

turns out we had fixed it in an earlier pull request (see above message)

results in 3.12: ```py

from sys import getsizeof as gf

a = [1, 2, 3]
b = []
b.append(1)
b.append(2)
b.append(3)
print(f'{gf(a) = }, {gf(b) = }')
gf(a) = 80, gf(b) = 88

damn

[quick night]

👀 👀

[rose schooner]

this is my proudest achievement yet (with godlygeek)

[quick night]

yea, i wouldve expected [1, 2, 3] to be better managed

[quick night]

nice

[rose schooner]

a recent pull request 2 hours ago (https://github.com/python/cpython/pull/92915) now matches the size actually ```py

from sys import getsizeof as gf

a = [1, 2, 3]
b = []
b.append(1)
b.append(2)
b.append(3)
print(f'{gf(a) = }, {gf(b) = }')
gf(a) = 88, gf(b) = 88

[heady mauve]

I am working on reverse engineering the structure of .pyc files for an upcoming (read: long time pondered) project and I am struggling to find any recent information. I've managed to find info on the marshalling format, but all I can find for what else is in the files is this article which uses Python 2.5: https://nedbatchelder.com/blog/200804/the_structure_of_pyc_files.html
Needless to say, that's a little out of date and does not match up with what I'm seeing. I've tried looking at Lib/py_compile.py and Python/marshal.c and am thoroughly lost.

[pliant tusk]

is there any plan to change how the garbage collector works so that it is only able to collect while python is within the ceval.c loop? Because of how it works now (it can collect any time a new object is allocated anywhere), arbitrary python code can run in unexpected places, leading to many different bugs if a malicious __del__ is used on a circularly referenced object.

[feral island]

not aware of any plans for that. The behavior you propose would make it more likely that memory allocation fails because there's no available memory.

[pliant tusk]

fair enough. Perhaps a way to ensure that objects with a python __del__ can only be freed from the ceval.c loop?

[feral island]

That sounds hard to implement, but I'm not an expert in the GC

And to be clear maybe making the GC run in fewer places is an acceptable tradeoff, it's true that there's probably a lot of lurking reentrancy bugs that get triggered when arbitrary Python code runs in unexpected places

[pliant tusk]

im curious if you could just skip objects in the pool if they have the specific tp_del that calls __del__

[pliant tusk]

but yea there are a good number of them. I was just looking at listobject.c and found a place that you use the bug in almost every method that makes a new list (slicing, addition, multiplication, etc)

[raven ridge]

No, because destroying the other objects in the cycle would break the cycle and drop the reference count for the objects with a Python __del__ down to 0, at which point they'd be destroyed by the regular reference counting mechanism

[pliant tusk]

Could you do the check in the regular reference counting mechanism and move them into a different queue there?

And then handle that queue when you know that calling python code is safe

[feral island]

that seems difficult because the cycle may be created after an object already exists

you'd need to skip any cycle from which an object with a __del__ is reachable

[pliant tusk]

Ah that would be tricky

[rose schooner]

officially merged now

so now we can't complain about it

[grave jolt]

!e
TIL

try:
    1 / 0
except ():
    print("wtf")

[fallen slate]

@grave jolt :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 2, in <module>
003 | ZeroDivisionError: division by zero

[flat gazelle]

In [16]: a=NameError, ZeroDivisionError

In [17]: try:
    ...:     1/0
    ...: except a:
    ...:     print("HI")
    ...:
HI```TIL part 2

[feral cedar]

huh, so you can just except nothing

[astral gazelle]

and still be disappointed

[heady mauve]

Does Python use little endian in .pyc files always or only when sys.byteorder == 'little'?

[radiant garden]

Must the argument be a tuple, or simply an iterable?

What about a tuple subclass? A fake tuple subclass?

[feral island]

you mean the argument to except:?

[radiant garden]

Yeah

Or is that just syntactic?

[feral island]

It's required to be a tuple of exception classes, including subclasses of tuple

based on check_except_type_valid in Python/ceval.c

[peak spoke]

Is there anything in the interpreter that requires a specific type and doesn't allow any subclasses?

[flat gazelle]

>>> [1,0][A()]
<stdin>:1: DeprecationWarning: __index__ returned non-int (type bool).  The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
0
```I found this

[feral island]

occasionally, grep for CheckExact

In 3.11 we added support for a closure tuple to exec() and that must be an exact tuple, not a subclass

[quick snow]

IIRC __match_args__ has to be a tuple (or list?) as well

[peak spoke]

Why wouldn't a subclass work in those cases? At worst it should be able to invoke the method from the class it needs if custom behaviour is unwanted

Though I guess at that point it can do the check to be explicit, though I believe I ran into that before

[feral island]

For the exec() case I don't think there was a good reason, Larry was just being cautious

In general it can be tricky because subclasses can mean any call can invoke arbitrary Python code and do unexpected things

[peak spoke]

!e ah it was exec/eval too

class A(dict):
    def __getitem__(self, item):
        raise Exception

print(eval("name", A(name="val"), {}))

Should this use the exact check for globals too then? As it uses the getitem from dict directly, even though a subclass was passed

[fallen slate]

@peak spoke :white_check_mark: Your eval job has completed with return code 0.

val

[west tinsel]

From 2 to 3, have there been any new keywords?

Other than match/case

[peak spoke]

True, False, async and await at least

[west tinsel]

Thanks! Blanked on that

[next lion]

nonlocal as well it seems

import keyword
import platform

print platform.python_version()
print keyword.kwlist

# 2.7.18
# ['and', 'as', 'assert', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'exec', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'not', 'or', 'pass', 'print', 'raise', 'return', 'try', 'while', 'with', 'yield']

# ['False', 'None', 'True', 'and', 'as', 'assert', 'async', 'await', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield']

Yeah print exec were removed and ['nonlocal', 'True', 'await', 'async', 'None', 'False'] were added

[peak spoke]

was None assignable like True/False, or was it just not a keyword but still a "keyword"?

[next lion]

SyntaxError: cannot assign to None

Nope

It is weird that those are keywords though, they dont really fit in with the rest

as in None, True and False

[native flame]

async/await weren't 2-to-3, right? they were added in 3.4 or something

or maybe i'm misinterpreting the question

[feral island]

yes, I think soft keywords in 3.6 and hard keywords in 3.8 or something

[next lion]

Actually why make those keywords? Did that give a speedup over raising the error the same way it is done for literals?

[feral island]

I guess for making the parser less hacky. In retrospect we could have just waited for the PEG parser and kept them soft keywords

[next lion]

I see, humm maybe it would have been too late by then, seems there was a lot of code assigning to these back then https://mail.python.org/pipermail/python-dev/2004-July/046294.html

[feral island]

oh sorry you're talking about True/False/None, not async/await

I wasn't around when the changes were made for the former so I'm not sure

[next lion]

Yeah I think async/await make sense being keywords, they dont really have a "value", they are just statements. True/False/None are the only keywords that are expressions

[feral cedar]

...?

[next lion]

?

[feral cedar]

isn't ... a keyword?

[next lion]

Well, it is not in the list, I guess it is just not assignable

But you also couldnt make a variable with just those chars, so I guess they are analogous to how you cant assign to numbers

[feral island]

... is not a keyword since it's punctuation

[sturdy timber]

>>> ... = 1
  File "<stdin>", line 1
    ... = 1
    ^^^
SyntaxError: cannot assign to ellipsis here. Maybe you meant '==' instead of '='?
>>> False = 1
  File "<stdin>", line 1
    False = 1
    ^^^^^
SyntaxError: cannot assign to False

Does the "here" in the ellipsis message have any meaning? (seems to imply that you could assign to it somewhere)

[feral island]

agree "here" should probably be removed

though note that you can assign to Ellipsis

[boreal umbra]

!e

class Foo:
    Ellipsis

[fallen slate]

@boreal umbra :warning: Your eval job has completed with return code 0.

[No output]

[boreal umbra]

it works!

[next lion]

I mean it works with any expression though :P

[next lion]

Numbers have the same error message, I think it was just copied from that, although that also begs the same question

!e 4 = 4

[fallen slate]

@next lion :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     4 = 4
003 |     ^
004 | SyntaxError: cannot assign to literal here. Maybe you meant '==' instead of '='?

[feral island]

also dicts

!e {a: b} = 3

[fallen slate]

@feral island :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     {a: b} = 3
003 |     ^^^^^^
004 | SyntaxError: cannot assign to dict literal here. Maybe you meant '==' instead of '='?

[next lion]

Maybe the "here" is just saying that if you assign the literal to a var then you can reassign it?

[feral island]

that doesn't make much sense to me. You still can't assign to a literal, only to a variable that holds a literal

!e (x for x in []) = 3

[fallen slate]

@feral island :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     (x for x in []) = 3
003 |     ^^^^^^^^^^^^^^^
004 | SyntaxError: cannot assign to generator expression

[feral island]

!e (lambda x: x) = 3

[fallen slate]

@feral island :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     (lambda x: x) = 3
003 |      ^^^^^^^^^^^
004 | SyntaxError: cannot assign to lambda here. Maybe you meant '==' instead of '='?

[feral island]

and we're not very consistent 🙂

!e (a := 3) = 4

[fallen slate]

@feral island :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     (a := 3) = 4
003 |      ^^^^^^
004 | SyntaxError: cannot assign to named expression here. Maybe you meant '==' instead of '='?

[boreal umbra]

should be "Maybe you meant to do something that makes sense???"

[lapis fractal]

This should be an Easter Egg on April Fools

For every error

[boreal umbra]

PR this pleaseeeeee

or the PR itself can be an april fools joke for next year

[lapis fractal]

should be an announcement on python .org

[feral island]

or maybe we should make assigning to a lambda mean something

[boreal umbra]

Many novice Python developers see an error message and think that they need to figure out how to "make the error message go away". This is, of course, the wrong mindset when resolving erroneous code. To mitigate this, we believe that all exceptions should involve a reality check: the code you wrote doesn't even make sense.

[next lion]

What could it mean? 🤔

[feral island]

the best I can come up with is calling the lambda

[feral cedar]

with the right side as an argument?

[next lion]

Calling and then assigning the result?

[quick snow]

It could just assign to the lambda? Like globals()[lambda: 42] = "foo"

[feral island]

oh I like that

[pliant tusk]

@feral island wrote a POC for the bytearray bug you reported recently #bot-commands message

[next lion]

But you wouldn't be really able to recover it from that thought

[sturdy timber]

Should be for mapping over an iterable, i.e. lambda x: x * 2 = [1, 2, 3] should set x to [2, 4, 6] 😃

[quick snow]

Thankfully I'm not in charge of Python

[feral island]

nice! that's a good demonstration that this is a security issue

[pliant tusk]

yea, its just one byte of memory corruption but it was enough lol. I commented it on your bug report

[next lion]

This is rather cursed looking but it would be a nice way of setting lambda partials x)

foo = lambda x: x * 2 = [1, 2, 3]
foo() == [1, 2, 3, 1, 2, 3]

bar = foo = (baz := lambda x: x * 2 = [1, 2, 3]) = 5

rip parser

[elder blade]

Wait- is there dictionary unpacking? That would be nice 😋

{'a': a, 'b': b} = x

...will be the same as...

a, b = x['a'], x['b']

[feral island]

there isn't, but that could actually be useful. it comes up on python-ideas every so often

[quick snow]

I mean now that match can do exactly that, why not

!e

x = {"a": 42, "b": 23}

match x:
    case {"a": p, "b": q}:
        print(p, q)

[fallen slate]

@quick snow :white_check_mark: Your eval job has completed with return code 0.

42 23

[rose schooner]

they all share a generic error message in the invalid_named_expression rule https://github.com/python/cpython/blob/main/Grammar/python.gram#L1116-L1118

[fallen slate]

Grammar/python.gram lines 1116 to 1118

| !(list|tuple|genexp|'True'|'None'|'False') a=bitwise_or b='=' bitwise_or !('='|':=') {
    RAISE_SYNTAX_ERROR_KNOWN_LOCATION(a, "cannot assign to %s here. Maybe you meant '==' instead of '='?",
                                      _PyPegen_get_expr_name(a)) }```

[rose schooner]

these lines in invalid_assignment handle the rest https://github.com/python/cpython/blob/main/Grammar/python.gram#L1131-L1132

[fallen slate]

Grammar/python.gram lines 1131 to 1132

| (star_targets '=')* a=star_expressions '=' {
    RAISE_SYNTAX_ERROR_INVALID_TARGET(STAR_TARGETS, a) }```

[rose schooner]

i swear 3.11/3.12 just keeps changing major opcodes https://github.com/python/cpython/commit/e48ac9c1003c3816198cbfb6132a995150f9b048

[native flame]

assigning to lambdas doesn't make any sense to me

[lusty scroll]

it literally is

pity the poor folks trying to keep up their bytecode parsing tools

[lusty scroll]

what is the target of the assignment with = [1, 2, 3] ?

[dusk comet]

dunder debug is also a special name

[rose schooner]

me but i don't manipulate it, i use it

[lusty scroll]

thats what i mean.. clumsy term

bytecode parsers

i am one such person as well

i had been keeping up for a while but then fell behind and now im just waiting for things to stabilize

personally I think the performance gains in 3.11 so far are worth it

[rich bloom]

h

[west tinsel]

If I'm modifying cpython, what's the best way to get the names from the typing library auto-imported?

from typing import *, but I don't want to keep writing it

[feral island]

builtins.__dict__.update(typing.__dict__) in site.py maybe?

[west tinsel]

Thanks!

[peak spoke]

What are the plans with typing documentation when the generics from typing get removed?

for example if I wanted to know how to pass the generic parameters to dict, but typing.Dict won't be there as it'll be removed

[feral island]

No real plan yet. https://github.com/python/cpython/issues/89515

[west tinsel]

For type checking, is instanceeof or type(...) == ... preferred?

Examples are FunctionType, str and int. Are those ever used in real world code?

[quick snow]

Try to avoid type checking. If you must, use isinstance over type(...) is ... (and never type(...) == ...)

[spice pecan]

type(...) == only really makes sense if you need specifically that type and not a subclass of it. The only time I had a legitimate use for it was when I needed an int, but not a bool, and even that one is pretty questionable

Most of the time you don't need to use isinstance and should just rely on types being correct, and in other cases you'd use match or isinstance instead of type(...) ==/is

[quick snow]

but even then you'd want type(...) is, not ==.

[spice pecan]

Equality would still work, but yeah, I used is instead

type(...) == is probably one of the most common "you REALLY should rethink this" flags

[quick snow]

!e

class Evil:
    def __eq__(self, other):
        return True

print(type(23) == Evil())

[fallen slate]

@quick snow :white_check_mark: Your eval job has completed with return code 0.

True

[spice pecan]

That's more of an edge case, int and bool aren't that evil (maybe they should be)

[boreal umbra]

class Evil:
    def __eq__(self, other):
        return True
    def __hash__(self):
        return 42069
    def __iter__(self):
        while True:
            yield None

[grave jolt]

bool being a subclass of int is already evil enough 🙂

I wonder how many Python services out there accept true as a timestamp

[boreal umbra]

isn't true an alias for 1 in C as well?

[raven ridge]

C doesn't define true at all by default, but if you #include <stdbool.h> then true becomes an alias for (technically, macro that expands to) 1.

[boreal umbra]

I remember learning that it wasn't originally part of the language, but wasn't sure about the mechanics of it. thanks 😄

[raven ridge]

yeah, stdbool.h was added in C99, and (because of backwards compatibility) you need to opt into it.

[boreal umbra]

how stable is C? I assume "very", esp as compared to python?

"stable" might not be widely used in the sense that I'm thinking of. ie, changes to the language spec being infrequent.

[raven ridge]

the changes tend to be smaller, though to be fair it's also a much smaller language.

backwards-incompatible changes are much rarer, too. Though the problems of compatibility are different - when you compile C code you can tell the C compiler which version of C your code is written in, and the same compiler can compile code written for many (frequently all) versions of C

[west tinsel]

Thanks!

[west tinsel]

And thanks!

[white nexus]

<@&831776746206265384>

ofc its bast

[torpid bridge]

@dire cobalt please don't advertise your projects here, this is not the channel for it

[dire cobalt]

whoops sorry

[white nexus]

oh i thought it was a scam link

[torpid bridge]

I mean.. that is another advertisement, so please just don't

[dire cobalt]

ok

what about a spoiler

||hostgator is the best site maker i use||

[torpid bridge]

That's better, but it also belongs in offtopic, if anywhere, and we'd rather it be part of an active discussion rather than just thrown out there. Advertising/getting attention in general is not something we like having

[dire cobalt]

ohhhh.

anyways nobody is n my game engine support link

or i meant server

i got moderation bots at my server

||im cold||

[torpid bridge]

This is the wrong channel. This channel is for discussion of python internals, and PEPS. Please take this to one of our off topic channels

[dire cobalt]

oh then where do i go

[torpid bridge]

#ot0-psvm’s-eternal-disapproval

[boreal umbra]

@mild shard see #❓｜how-to-get-help

[west tinsel]

How does the lexer differentiate between EQUAL "=" and EQEQUAL "=="?

If I add COLONCOLON "::", example_list[::-1] stops working because :: is processed as COLONCOLON

[feral island]

that seems different? x[::-1] should be parsed as two separate colon tokens

[west tinsel]

I want that to happen but when I add a :: token it reads it as the double colon token

[feral island]

sure, and that's exactly how == works too

I'm no expert in parsing, but I think you may need some complicated workaround. For example, the lexer could be context-sensitive and know to make :: one token in some places and two tokens in others. Or the parser could accept the :: token in subscripting and treat it like two :

I think an analog is C++ where they made >> work in nested template parameters (vector<vector<int>>), so the parser has to go through some shenanigans too distinguish it from >> for right-shifting

[west tinsel]

Oh hmm I see

[dusk comet]

random::randint(0,1)

[west tinsel]

Figured it out

Matching ':' ':' worked but I had to put the rule before cases of ':'

So it didn't go down the ':' path to an invalid_ rule or so

[west tinsel]

In mypy, where do I want to look if I wanted to change the behaviour when annotating types?

Currently I get "Invalid type comment or annotation" when writing variable: ANNOTATION where the annotation is new syntax that I've added

[feral island]

semanal.py probably? I'd grep for that error message and work backwards

[safe hedge]

Can you have a method definition which is dynamic?
Something like:

def somefunc_for_x(self):
    return self._underlying_func(self.x)

---

# Calling code
myobj.somefunc_for_yellow()
myobj.somefunc_for_brown()

But such that I wouldn't have to actually define a separate method for each possible x?

And if it is possible, how terrible of an idea is it?

[jovial flame]

Wouldn't something like that be possible by defining the function argument?

[safe hedge]

Of course you could do something like:

def somefunc_generic(x):
    return self._underlying_func(getattr(self, x))

But I just wondered if it was possible to have the actual method name be evaluated dynamically?

I was thinking mostly a situation where I would want the generic function to be a property called like an attribute.

[jovial flame]

The second solution looks good to me to be honest. You would still call it as a method, just with the actual attribute as an argument. But at that point, why wouldn't you call it directly?

[safe hedge]

The use case I had was an dataclass object with several lists of integers, and I wanted to provide the mean/median of each as a property, without needing to define a method for each. But that may just be unnecessary tbh
e.g.

def mean_somelist(self):
    return mean(self.somelist)

---

myclass.mean_list1
myclass.median_list2

[quick snow]

I mean of course you can, with __getattr__

[grave jolt]

Maybe you could make a special collection that knows how to take its own mean?

myclass.foos.mean()
myclass.foos  # actual `foo`s

[safe hedge]

Tbh I think i was over-engineering/thinking this. I don’t think calling mean(attribute) is really that hard haha

[pliant tusk]

You could use a custom list subclass for the data class so you get the mean and median on the list

[dusk comet]

import functools

class X:
    def __class_getitem__(cls, key):
        print(cls, key)

class Y:
    @classmethod
    @functools.cache
    def __class_getitem__(cls, key):
        print(cls, key)

class Z:
    @functools.cache
    def __class_getitem__(cls, key):
        print(cls, key)

X[int] # ok
Y[int] # ok
Z[int] # TypeError: Z.__class_getitem__() missing 1 required positional argument: 'key'

why? 😭

[quick snow]

My guess: if a class has a function called __class_getitem__, it gets automatically converted to a classmethod.

[prime estuary]

Yep, it's special cased along with __init_subclass__ and __new__:
https://github.com/python/cpython/blob/main/Objects/typeobject.c#L3082-L3095

[fallen slate]

Objects/typeobject.c lines 3082 to 3095

/* Special-case __new__: if it's a plain function,
   make it a static function */
if (type_new_staticmethod(type, &_Py_ID(__new__)) < 0) {
    return -1;
}

/* Special-case __init_subclass__ and __class_getitem__:
   if they are plain functions, make them classmethods */
if (type_new_classmethod(type, &_Py_ID(__init_subclass__)) < 0) {
    return -1;
}
if (type_new_classmethod(type, &_Py_ID(__class_getitem__)) < 0) {
    return -1;
}```

[prime estuary]

But if it's not a FunctionType, this does nothing so you'll have to do it yourself.

[dusk comet]

ok, thanks
now i understand what is happening

[boreal umbra]

How is it that threading can make certain IO-bound jobs faster per thread if only one instruction is executed at a time?

[feral island]

Not sure if I fully understand your question, but the GIL is dropped during many IO operations, which allows other threads to execute

[rose schooner]

any reason why traceback limits are by default 1000 instead of sys.getrecursionlimit()?

[visual shadow]

It doesn't if there's only one task. What it does do however, is utilize the wait time of the other task also, giving the net effect of a faster total runtime. Think 5 second task where 4 second is wait time. 1 task would take 5 sec, 2 would take 10 second without threading, 6 with threading. (But first task wouldn't be done till 5 seconds even then, just wait time is used well)

[elder blade]

The GIL doesn't need to be held while waiting for the socket to return data of the I/O response (the thread is stuck in an OS call)

The other thread can then do other things while it is waiting

[wet sand]

is python optimized for recursion ? if not how can i write functional looking code?

[old hearth]

where are instances of a class stored?

is there a convenient list/dict with all the instances of a class somewhere ?

or is this something that has to be manually setup?

[elder blade]

Nope

[feral island]

You can get such a list with gc.get_referrers I think, but not efficiently

[elder blade]

Yeah you could keep it in a weakreferenced list on the class

[feral island]

Instances are just stored wherever on the heap

[old hearth]

does the python interpreter/runtime know internally if a class has instances?

[feral island]

don't think so, other than through the GC

[dusk comet]

!e

class Meta(type):
    def __del__(cls) -> None:
        print(f'{cls}.__del__()')

class X(metaclass=Meta):
    pass

x = X()
print('x = X()')
del X
print('del X')
del x
print('del x # note now X isnt collected')
print(object.__subclasses__()[-1])
print('X is collected only at shutdown time')

[fallen slate]

@dusk comet :white_check_mark: Your eval job has completed with return code 0.

001 | x = X()
002 | del X
003 | del x # note now X isnt collected
004 | <class '__main__.X'>
005 | X is collected only at shutdown time
006 | <class '__main__.X'>.__del__()

[old hearth]

i almost understand this code... Don't know much about metaclasses and why inherit from type?

[feral island]

hm I thought __subclasses__ just held a weakref, so the class should get collected

[old hearth]

so I can use Foo.__subclasses__() to get instances of Foo?

[feral island]

yeah it's still around because there's a reference cycle between the class and its __dict__ I think. gc.collect() makes it go away

... 
>>> object.__subclasses__()[-1].__name__
'X'
>>> del X
>>> object.__subclasses__()[-1].__name__
'X'
>>> import gc
>>> gc.collect()
29
>>> gc.collect()
0
>>> object.__subclasses__()[-1].__name__
'Completer'

[feral island]

no, you get subclasses of Foo, not instances

[old hearth]

oh, lol. ok

[feral island]

correction: the reference cycle involves the class's MRO and attributes, not its __dict__ directly

... 
>>> gc.get_referrers(X)
[(<class '__main__.X'>, <class 'object'>), <attribute '__dict__' of 'X' objects>, <attribute '__weakref__' of 'X' objects>, {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, 'gc': <module 'gc' (built-in)>, 'X': <class '__main__.X'>}]

the dict is my shell's globals

[wet dawn]

hey

#!/usr/bin/env python3

import pickle
import pickletools

from flag import FLAG


def check(data):
    return len(data) <= 400 and all(
        opcode.code.encode() != pickle.REDUCE
        for opcode, _, _ in pickletools.genops(data)
    )


if __name__ == "__main__":
    print("Welcome to the pickle games!")
    data = bytes.fromhex(input("Enter your hex-encoded pickle data: "))
    if check(data):
        result = pickle.loads(data)
        print(f"Result: {result}")
    else:
        print("Check failed :(")

help me with this please

[boreal umbra]

this isn't a help channel; see #❓｜how-to-get-help. also be sure to say what you need help with--this is just code.

[boreal umbra]

@onyx epoch your message was off-topic. see #❓｜how-to-get-help

[west tinsel]

https://github.com/python/typeshed/issues/7953

How come something like dict[NoneType, str | list[str]] would break things?

[boreal umbra]

@feral island this one's about you 😮

[feral island]

If you're using a DictReader and assuming that all the keys and values are strs, you'll now get a false positive error

now = if we use your proposed annotation

[west tinsel]

Oh that makes sense, thanks!

[boreal umbra]

@ebon relic this isn't a help channel. Also most of us don't know how to read that. See #❓｜how-to-get-help

[ebon relic]

ok, sorry

[grave jolt]

Thought experiment: which keywords cannot be made into soft keywords?

(example: try and except cannot (together), because this would be ambiguous:

try: int
except: list[str]
``` )

[sturdy timber]

That takes else, finally, and lambda in a similar way

del would be ambiguous as you can do del (a)

assert as you can do assert (a)

yield too

[native flame]

break, continue

pass

[sturdy timber]

if[1]: int, so if too

and elif

[native flame]

oh thats a good one

raise, return, await

[native flame]

while, for the same reasons

[sturdy timber]

Do False, None, and True count lol?

[boreal umbra]

The official terminology is that they're keywords, but I don't agree with that. They're constants that refer to objects. The "true keywords" are purely part of the grammar of the language.

[grave jolt]

Seems like async can still be soft?

[grave jolt]

Hmmm, that is not ambiguous

!e
This is not valid syntax

foo[0]: int

[fallen slate]

@grave jolt :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | NameError: name 'foo' is not defined

[grave jolt]

Oh wait, it is?

wtf

[sturdy timber]

wait yeah I just realised why does that work haha

[grave jolt]

from and import are out (at least together) because

from . import (a)

Might be a method call

[sturdy timber]

nice

[native flame]

hahaha

hmm for(x) in y could be a function call + membership test but the colon would make it unambiguous

global nonlocal and not go out the same way as del

wth with[x]: int is valid syntax lol

[rich cradle]

does assert go out as well?

[native flame]

yea

[rich cradle]

ah right

[grave jolt]

!e

def foo():
    global (x)

[fallen slate]

@grave jolt :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 2
002 |     global (x)
003 |            ^
004 | SyntaxError: invalid syntax

[grave jolt]

Nope to first two

[native flame]

O

[rich cradle]

return (a) could be a function call I think, so that goes away

[west tinsel]

This looks like a fun game 😂

[native flame]

yeah we got return

these are left, i think

[west tinsel]

I think we can remove in

Wait nvm

[native flame]

and, or, is, in seem safe

[rich cradle]

I think as is safe because it's only used in conjunction with other keywords in that list afaik

[boreal umbra]

it was originally, so that's unsurprising. same with await

[native flame]

await(x) could be a function call though

[boreal umbra]

!otn a async def await

[fallen slate]

:ok_hand: Added async-def-await to the names list.

[quick snow]

So basically if there were no type hints, most keyword could have become soft?

def and class should be fine

[dusk comet]

can # be a soft keyword? 😄

[radiant garden]

[
    ... # needs something here to ambiguate
    for x, 
    in (y)
]

[boreal umbra]

I figure this isn't a serious question, but considering that # can only appear in an expression if it's part of a string, and nothing after it is parsed otherwise, no.

[raven ridge]

They're not constants, because they can't be assigned to. They're keywords that refer to constants.

!e True = 42

[fallen slate]

@raven ridge :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     True = 42
003 |     ^^^^
004 | SyntaxError: cannot assign to True

[raven ridge]

That's a SyntaxError exactly because it's a keyword.

[boreal umbra]

this is just debate about ideal nomenclature.

!e

def = 5

[fallen slate]

@boreal umbra :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     def = 5
003 |         ^
004 | SyntaxError: invalid syntax

[boreal umbra]

though if it were truly symmetric, shouldn't the error message be "cannot assign to def"?

[raven ridge]

Symmetry of error messages isn't required. What's required to make something a keyword is that it's part of the grammar and special cased in the parser.

At one point, that wasn't the case for True and False - they used to just be regular built in variables. They were promoted to keywords so that reassignments would be prevented.

[boreal umbra]

my opinion is that the terminology should treat words that are special cased by the parser and which refer to objects differently, and that they should be called constants. I recognize that the language spec doesn't reflect this usage. I'm not really sure what there is to debate.

[raven ridge]

It's hard to have any sort of productive conversation if you can't agree on the meanings of the terms involved. True and False cannot be made into soft keywords and must remain special cased in the parser, exactly because True = 42 must fail.

That's the point wookie was making, and that point is correct.

[boreal umbra]

I wasn't arguing that they can. and I never commented on the validity of wookie's point. I just used it as an opportunity to soapbox about a disagreement I have with the official terminology.

[raven ridge]

Ok, so how would you express that conclusion ("True and False can't be made into soft keywords, aka context sensitive keywords, because assignments to them are invalid") using your preferred terminology?

[boreal umbra]

"True and False cannot be made into soft keywords, because soft keywords could be used as names for arbitrary objects, whereas True, False, and None are special cased to refer only to certain immutable objects."

the fact that the objects in question are immutable is actually irrelevant.

[raven ridge]

Do you see constants as a sub category of keywords, then? Some, but not all, keywords are constants?

[boreal umbra]

In the context of Python, where users cannot define constants, I see what the Python spec calls "keywords" as the union of two disjoint sets: constants, and what I call "keywords".

[raven ridge]

That seems like a very unhelpful way to view things. By your definition, True and False and None are not keywords, and so fix's question about what keywords could never become soft keywords doesn't even apply to them. By everyone else's definition, it does.

[boreal umbra]

I was never trying to frame my point in the context of fix's question.

[rose schooner]

True, False, and None are actually parsed as real keywords and results in a (python-level) ast.Constant node with values Py_True, Py_False, and Py_None respectively

[boreal umbra]

!e

import keyword
print(keyword.kwlist)
__debug__ = 'hi'

[fallen slate]

@boreal umbra :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 3
002 | SyntaxError: cannot assign to __debug__

[boreal umbra]

!e

import keyword
print(keyword.kwlist)

[fallen slate]

@boreal umbra :white_check_mark: Your eval job has completed with return code 0.

['False', 'None', 'True', 'and', 'as', 'assert', 'async', 'await', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield']

[boreal umbra]

this isn't intended to prove any particular point. it's just interesting.

[rose schooner]

__debug__ is handled in the compiler

[wispy sierra]

Uhhh constants can't usually be assigned to either in programming languages that have them

[grave jolt]

well, we live in Python 🙂

[tacit hawk]

is the work on subinterpreters taken as relevant? I heard people saying it's not worth since there is the multiprocessing module

[feral island]

the people working on it definitely think it's important. One difference is that subinterpreters could allow much faster communication between processes

Personally I'm not too convinced of the benefits of subinterpreters, especially if adopting them requires invasive C API changes

[tacit hawk]

For me it's relevant, I was wondering about the general option, it seems there is some resistance

[quick snow]

What's the point of operator.concat? Is it faster than operator.add (which, contrary to documentation, also works for sequences)?

[grave jolt]

I think it can be optimized with the C-module _operator

[peak spoke]

Looks like there's both number add and sequence concat in the interpreter, not really sure why as both fall back to the other

[swift imp]

There is no protocol for concatenation right?

[peak spoke]

and exposing that through operator seems weird

[grave jolt]

I remember vaguely from Armin Ronacher's talk that there are two "type slots" for number-adding and sequence-adding

so maybe concat uses this slot directly in the C implementation?

[feral island]

yes, that's right

operator.concat is PySequence_Concat and operator.add is PyNumber_Add

[peak spoke]

You can see it if you try to add something incompatible to a string as it doesn't use the generic error message but mentions concat

In [9]: "a" + 1
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [9], in <module>
----> 1 "a" + 1

TypeError: can only concatenate str (not "int") to str

[peak spoke]

Why is that a thing? (and why is it exposed in operator when the operator is the same)

[feral island]

That's just because that's the error message in the implementation of str.__add__

(implemented in the PyUnicode_Concat function)

[feral island]

"why" is probably lost in early history. There are different structs containing "number" slots and "sequence" slots, so I guess they put the + implementation in two places so numbers or sequences wouldn't need to declare the other kind of slot

[grave jolt]

"concatenate" sounds like a harder word for beginners to programming than "add" to be honest

also with no direct hint about +

[feral island]

note that operator.concat(1, 1) fails

[grave jolt]

...but I bet someone actually relies on the particular error message 😜

[feral island]

yeah not sure I would defend the fact that operator.concat exists at the Python level, seems like it's leaking an implementation detail

interestingly there's no operator.repeat exposing PySequence_Repeat

[swift imp]

That is interesting

[prime estuary]

The different "duplicate" slots do have different parameter types and the like matching how they'd behave, which is probably why they were separate - but when Python-level classes came along with magic methods, making two copies there would be confusing.

[feral island]

sq_concat and nb_add are both binaryfunc slots, not sure how the types are different

[prime estuary]

Yeah those two specifically aren't, but for instance sq_repeat is (PyObject *, Py_ssize_t) which doesn't match nb_multiply.

[feral island]

That's true. The two __getitem__ slots (sq_item and mp_subscript) also have different signatures

[lusty scroll]

question: why does python bother freeing memory and cleaning up garbage when it's exiting ?

I've noticed it can take more time if there were many objects (my guess is especially things with possible reference cycles) in memory when exiting

this is the line that seems to take the most time in PyFinalize_Ex():

/* Destroy all modules *
_PyImport_Cleanup(tstate);```

[flat gazelle]

it is afaik a fairly new feature where __del__ is guaranteed to get called, since it could be deleting temporary files etc.

[lusty scroll]

and the most time consuming step there is

/* Once more */
_PyGC_CollectNoFail();```

[lusty scroll]

Ohh ok

I can see why that'd be desirable if files are left waiting to be implicitly closed, I suppose

/* Call tp_clear on objects in the
final_unreachable set.  This will cause
the reference cycles to be broken.  It may also cause some objects in finalizers to be freed.*/
m += gc_list_size(&final_unreachable);
delete_garbage(tstate, gcstate, &final_unreachable, old);   <<<<```

the delete garbage line .. maybe that's where the __del__ methods get run from?

just for anyone curious: so in my particular case, when I happened to look, it was deallocating a tuple, which called _Py_Dealloc on one of the tuple items, which turned out to be an Exception, which led to the traceback being cleared, then a tb frame, which led to a dict dealloc (locals or globals maybe), which led to another dict dealloc, and so on ..

fun stuff 😄

glad I didn't have to write that code

oh, one more thing, the last call to free before I looked at the callstack caused glibc to do a bunch of shuffling around (malloc_consolidate()) which I expect is certainly out in python's hands

no idea how long that took

basically turned into an impromptu compacting garbage collector there

[rose schooner]

what happened to 3.10 and what's opcache

[feral island]

it's useful when you're embedding Python and repeatedly initializing/finalizing it in the same process

[west tinsel]

Is there any reason to use prefer any of PyObject_TypeCheck and PyObject_IsInstance?

I'm working on rust bindings for Python and they both look like they'd work

[feral island]

the former bypasses __instancecheck__, right?

as well as the .__class__ access isinstance() does

[prime estuary]

Docs for the latter mention those hooks, plus also __bases__ on the class you pass. So it's for checking if the object should be considered an instance, but you need the former for a safer check before derefererencing pointers etc.

[west tinsel]

static inline int PyObject_TypeCheck(PyObject *ob, PyTypeObject *type) {
    return Py_IS_TYPE(ob, type) || PyType_IsSubtype(Py_TYPE(ob), type);
}

int
PyObject_IsInstance(PyObject *inst, PyObject *cls)
{
    PyThreadState *tstate = _PyThreadState_GET();
    return object_recursive_isinstance(tstate, inst, cls);
}
static int
object_recursive_isinstance(PyThreadState *tstate, PyObject *inst, PyObject *cls)
{
    /* Quick test for an exact match */
    if (Py_IS_TYPE(inst, (PyTypeObject *)cls)) {
        return 1;
    }

    /* We know what type's __instancecheck__ does. */
    if (PyType_CheckExact(cls)) {
...

What do __instancecheck__ and __bases__ mean in practice?

And the context is that PyDictProxy_Check and PyByteArray_Check aren't in the API so they're being written in rust

[prime estuary]

It's what allows collections.abc.* classes to have "virtual" subclasses, accepting things that have the right methods. It'd also allow say a dynamic proxy or mock class to pretend to be its target.

So for those use cases you probably want to be then accessing members of the struct, so you want PyObject_TypeCheck(), which can't be lied to.

[west tinsel]

Ok, got it!

Thanks both of you

How should I access the underlying dictionary for mappingproxy?

iirc, accessing .mapping worked but idk if it's part of the API

[grave jolt]

Well, it doesn't start with an underscore

Something something Hyrum's Law 🙂

[rose schooner]

like in the C API?

yeah just do ->mapping i guess

in python code it would be ```py
@type.call
class A:
def or(self, other):
return other

print(A | mappingproxy_object) # {...}

[grave jolt]

hm?

!e

@type.__call__
class A:
    def __or__(self, other):
        return other

print(type(A | int.__dict__))

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

<class 'mappingproxy'>

[rose schooner]

hmm

[grave jolt]

What did you expect to happen? 🙂

[rose schooner]

!e ```py
@type.call
class A:
def eq(self, other):
return other

dictionary = int.dict == A
dictionary['a'] = 5
print(int.a)

[fallen slate]

@rose schooner :white_check_mark: Your eval job has completed with return code 0.

5

[rose schooner]

@grave jolt there we go

[grave jolt]

I remember how I implemented a searchable set like this x)

a spying object that remembered its __eq__ calls

[elder blade]

Does that not undermine the purpose of a mapping proxy?

[west tinsel]

I'm not modifying the underlying dict, could that still be problematic?

[feral island]

doesn't this possibly segfault if you don't clear some caches?

I think Serhyi found some fun bugs like that

[dusk comet]

def _get_mappingproxy_dict(mp: MappingProxyType[_T, _G], /) -> dict[_T, _G]:
    referents = gc.get_referents(mp)
    if len(referents) != 1:
        raise RuntimeError('Mapping proxy refers to several objects', mp, referents)
    dct = referents[0]
    assert isinstance(dct, dict)
    return dct

you need import gc

[rose schooner]

pretty sure you also need stuff for the typehints

[feral island]

this works too:

!e ```from types import MappingProxyType

d = {}
mp = MappingProxyType(d)

class Sneaky:
def eq(self, other):
self.other = other
return False

s = Sneaky()
mp == s

assert d is s.other
s.other["a"] = "b"
print(d)```

[fallen slate]

@feral island :white_check_mark: Your eval job has completed with return code 0.

{'a': 'b'}

[snow cave]

yo

[snow cave]

can someone pls talk to me

[west tinsel]

https://docs.python.org/3/c-api/dict.html

Is there any specific reason methods for mappingproxy aren't listed in the API?

I'm considering calling Python methods now for things like length and copying

[feral island]

looks like there's no public API for it

[rose schooner]

yeah because it doesn't have a public API
i've searched through it and all i've found is PyDictProxy_New

everything else is a static function

[west tinsel]

Is there a reason it doesn't have a public API?

[rose schooner]

idk

its methods are just one-liners though

also they seem to also support anything else other than a dictionary
but it doesn't work with just initializing and requires changing ->mapping

[humble ridge]

Hi everyone
So here what I'm trying to do, I'm new to programming in general " started a few months ago "
and I'm learning python " that why I'm here "

so what I'm doing right now if I see a built in function or an operator I try to figure out what the sequence of logic behind it what kind of functions is that and how I make one that give me the same output, I don't like abstraction or using tools that I don't know how it's built or at least I have an idea how it built

so my question, is this good practice ? to try to implement my own function and look how every feature works, and rewrite in pure python ?

or should I just keep going with normal python and work with it's abstraction

and at the same time, I go for C language to learn it and do that stuff there better then doing it in python ?

( here an example, I was following the course PY4E, and there two functions translate() + maketrans() , so I tried to do something similar with the logic that I learn so far )

• yesterday I tried to figure out how in operator work, I hope you get my point by now, I don't want only use tools without knowing how actually it's work and built

someText = "Hello, There"

def stringRemover(str1, str2):


    strToArray = []
    removedPart = tuple(str2)

    for letter in str1:
        if letter in removedPart:
            continue
        else:
            strToArray.append(letter)

    result = ''.join(strToArray)
    return result
    
newString = stringRemover(someText, "!\"#$%&'()*+,-./:;<=>?@[\]^_`{|}~")

print(newString)

[boreal umbra]

@humble ridge

I don't like abstraction or using tools that I don't know how it's built or at least I have an idea how it built
abstraction is a critical part of programming. there simply isn't enough time to learn how everything works "all the way down".

so my question, is this good practice ? to try to implement my own function and look how every feature works, and rewrite in pure python ?
reimplementing things on your own is a good way to learn, but it would be impractical (if not impossible) to accomplish this for every abstraction that you use.

and at the same time, I go for C language to learn it and do that stuff there better then doing it in python ?
if you want to develop an OS, implement an interpreted language, or write code that runs in resource-constrained environments, learning C would make sense. but it's not practical for most domains of software development.

[humble ridge]

that really what I hope to do to understand stuff all the way down
I know that the abstraction is a critical part of programming or otherwise we'll write a 0 1 code 😂

but I want to have an idea at least, I don't to be only a coder " write just some code for something to run and gluing pieces together " , my goal to be more like a computer scientist or engineer

they understand a lot of low level concept which give them the ability to build almost everything they want, and solve problem on they are own if needed or wanted , in short like people who invent the stuff we are using today

I'll not build everything from scratch or reinvent the wheel I want to look up for existing algorithm and apply it, and for the daily basis or for work I'll use the built-in stuff and the ready stuff for sure

====
so what I need to do to reach that point what I need to learn to apply, I know it's hard and gonna take a lot of time but I'm really willing to put the time and effort I just need any kind of guide or a path

[radiant garden]

the problem is that modern processors are unreasonably complex

the work of hundreds of people over years

by all means go for it, but know when to stop

maybe before reading all 4800 pages of the intel 64 and IA-32 architectures
software developer’s manual

[humble ridge]

ikr ?
I wanted like a roadmap or topics I need to go over to have better understanding
but I'll try to figure it out on my own, a lot of people making great stuff from scratch for the sake of knowledge , I hope I'll be able to something similar to that someday

for example :
https://youtu.be/FaILnmUYS_U
https://www.youtube.com/playlist?list=PLowKtXNTBypGqImE405J2565dvjafglHU

and there a lot more, I really love these people, they have such a deep understanding for CS and programming in general and I think by that knowledge they can kind of keep up with complexity of today

[dusk comet]

is that true?

[onyx drift]

@dusk comet Good question. It's not in 3.11.0b3, and given that we're well into feature freeze for 3.11 I guess that's not going to change (unless there's a special exemption from the release manager). The relevant issue appears to be this one: https://github.com/python/cpython/issues/82786.

And if it's not going into 3.11, looks like the mandatory field in future.annotations needs an update. It still mentions 3.11.0.

Python 3.11.0b3 (main, Jun  4 2022, 07:16:06) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from __future__ import annotations
>>> annotations
_Feature((3, 7, 0, 'beta', 1), (3, 11, 0, 'alpha', 0), 16777216)

[onyx drift]

@dusk comet Finally found the post I was looking for: PEP 563 is indeed deferred for 3.11: https://mail.python.org/archives/list/[e-mail]/message/VIZEBX5EYMSYIJNDBF6DMUMZOCWHARSO/. The 3.11 docs do have a note about this: https://docs.python.org/3.11/library/__future__.html

[west tinsel]

Is there an equivalent of PyDict_Next for mappingproxy?

So iterating key-value pairs using the API

[fallen slate]

Hey @mossy roost!

It looks like you tried to attach a Python file - please use a code-pasting service such as https://paste.pythondiscord.com

[mossy roost]

i'm so proud about my first projetc ^^

https://paste.pythondiscord.com/ofebibufah

[elder blade]

Do threads to Python's mailing list go through an approval process?

I opened a new thread and I got a success message but the thread does not pop up in the mailing list 😔

Oh- nvm I got an email!

[raven ridge]

I believe the first message from new members is moderated

[elder blade]

https://tenor.com/view/omuniqued-raeswae-wave-hello-sassy-gif-20900962

[rose schooner]

yeah i was confused about that so my first thread (in python-ideas) i submitted two times

the moderator moderating had a problem with that (and was also confused why i submitted the same thing two times)

[feral island]

I'd recommend writing to typing-sig instead for that

[elder blade]

What do you mean? My mail was sent to typing-sig. Should I write to them again about awaiting moderator approval?

[rose schooner]

wait for a day

[feral island]

sorry I somehow thought you were writing to python-ideas

[mellow maple]

hey people
Can someone look into this? the problem and the code is pretty simple i think:
#help-pineapple message

[lusty scroll]

id is the memory address, at least in cpython

sizeof for a list doesnt increase uniformly (or even always increase) as you add items to a list though

what is the result you were expecting to see @mellow maple

[mellow maple]

I was expecting to see the address of list change when more mem was required.

[rose schooner]

expect to see ctypes.c_void_p.from_address(id(list_object)+int.__basicsize__).value change when more memory is required

[lusty scroll]

the list doesn't move i don't think

does it?

i don't recall seeing anything about objects moving around which would seem to take a bunch of extra steps to actually use the objects

[prime estuary]

The list object itself is only a header, the pointer array is allocated separately so that it can reallocate as required.

[grave jolt]

In CPython objects can't change their addresses because other objects hold them as pointers.

In some languages it's done differently because it allows heap compaction IIRC

[prime estuary]

Yep, also it's potentially useful for optimisations - you could do things like put objects with similar lifetimes together or that are otherwise related, or use location to encode say if it's GC enabled or something.

PyPy does have a moving GC so it's objects can move.

[rose schooner]

is there a reason why the python integer structure doesn't use the whole 32/16 bits of digit?

[elder blade]

Yes I believe the bits hold some type of flag significance

[severe lynx]

i think it is the underlying OS not python. the flag functionality is called 2;s compliment for negative values. https://en.wikipedia.org/wiki/Two's_complement

[dusk comet]

i guess 30-31 bits are significant, and 1-2 bits are flags (for example, flag of last digit, or flag of negative number)

>>> class X: __slots__ = ()
...
>>> sys.getsizeof(X())
32
>>> sys.getsizeof(object())
16

why objects with empty __slots__ takes 32 bytes instead of 16?
8 bytes - refcount, 8 bytes - reference to type, what else?

[pliant tusk]

!e py class x: __slots__=() import sys print(sys.getsizeof(x()))

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

32

[pliant tusk]

Hmm that's weird

[feral island]

are you on the same version?

[pliant tusk]

The screenshot is 3.6 I think

But I wonder what that space is used for now

Aha figured it out, the extra 16 bytes is from the garbage collector

The actual object still has the normal 16 byte footprint

There should probably be a check so that an object that has an empty slots doesn’t get the cycle gc because it doesn't need it

[feral island]

there could still be a cycle through the type object, right?

[raven ridge]

I believe it's because it makes handling overflow easier.

[pliant tusk]

Ah yea forgot about that

[boreal umbra]

@frigid shoal your question was off-topic for this channel; try #career-advice

[grave jolt]

@raw flicker If you want to ask something completely unrelated to Python, you can ask in the off-topic channels:

!ot

[fallen slate]

Off-topic channel: #ot2-never-nester’s-nightmare

Please read our off-topic etiquette before participating in conversations.

[dusk comet]

https://docs.python.org/3.11/library/ast.html?highlight=ast#ast.AST

[grave jolt]

the docs were written in Australia 😛

[west tinsel]

😂

[safe hedge]

Are turtle/tkinter the only stdlib modules that can/will throw errors about shared library files?

[feral island]

lots of them do, on linux you can run ldd to see what so files they depend on

most of them just depend on things like libc, but hashlib depends on libcrypto for example

[safe hedge]

How do I use ldd to see? Which files am I pointing it at?

[feral island]

the .so files for stdlib libraries. For example:

Python 3.9.12 (heads/3.9.12:6a8ccd4c69a, Apr  3 2022, 07:06:45) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _hashlib
>>> _hashlib
<module '_hashlib' from '/home/jelle/ans/venv3.9/lib/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so'>

so for me they live in that lib-dynload directory

then ```$ ldd /home/jelle/ans/venv3.9/lib/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so
linux-vdso.so.1 => (0x00007ffc62520000)
libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fcb4d99a000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcb4d77d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcb4d3b3000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fcb4d1af000)
/lib64/ld-linux-x86-64.so.2 (0x00007fcb4dfe9000)

[safe hedge]

Hmm so that library is packaged with Python itself right?

Basically I compiled python3.9 for Ubuntu and tried to package it, and basically everything seems fine when I install it except that libtk8.6.so cannot open shared object file: No such file or directory.

When I compiled it, importing turtle worked fine. But when you install it from our private apt then it’s broken. I am basically trying to find if any other modules/code is likely to be busted too? And whether the simple fix of apt-get install tk is sufficient

[quick snow]

No, Python doesn't ship with libc, for example, it just depends on it. If you compile it yourself, you usually need a -dev package (e.g. libtk-dev(?)), although the exact conventions differ per distro.

[feral island]

yes, I guess the difference is that libc is really always there (if not, not much is going to work), while tk might not be

[quick snow]

For Ubuntu, the package seems to be called tk-dev.

[safe hedge]

This is about after it’s been compiled and packaged though

I can’t work out whether I am supposed to package tk in with it somehow or whether I am just supposed to leave that up to the user to install separately?

[feral island]

is this for like apt installable packages? I think the normal thing there would be to depend on tk-dev or the equivalent

haven't ever tried to create such packages myself though

[safe hedge]

It is indeed for a apt-installable package

[old hearth]

if you package in tk make sure you check for the presence of the underlying tk runtime library(linux)

[safe hedge]

I think that’s my exact issue no?

[cyan blade]

how do u send code like that in a box

[old hearth]

oh lol...

read that wrong, thought you were talking about python-tk

[safe hedge]

When I compile on a system with tk-dev (so that it compiles with tkinter support) the import works fine. Then I package the Python using deb-pkg and apt install it on a new box. And boom, error because the shared libtk8.6.so is missing

I was just surprised that was the only thing that threw that error (ie hashlib noted above works fine) but it seems that the necessary libcrypto files are part of the package and the tk ones aren’t?

[old hearth]

yup. that is the runtime. i believe a sudo apt-get install python3-tk installs that...

[safe hedge]

Yeah, but I am trying to package my own Python3 haha

[feral island]

triple backticks

[safe hedge]

Because we don’t want the one from public apt

[old hearth]

i see

[safe hedge]

I think I can just make tk-dev a dependency, but I was concerned there were other underlying libs I would be missing and not know

I’m very much winging this and such packaging is not my area of expertise

[feral island]

the list of apt-get installs here is approximately what you'd need: https://devguide.python.org/setup/?highlight=lzma#linux

though those are the dev versions which you need for building, the runtime packages would be a little different I think

[safe hedge]

Right yes, that’s the thing. I used that list on the box I compiled on and built the deb pkg on. Then on a totally clean box (with only a very slim Ubuntu) I apt install and tkinter is broken. Given I don’t have several other of those dependencies from the build box installed either I am surprised nothing else appears broken

[old hearth]

you apt install what?

[safe hedge]

The Python3 pkg I built

[old hearth]

i believe you need to sudo apt-get install python3-tk for the tk runtime to get installed...

pip doesn't install the runtime?

[feral island]

tkinter is part of the stdlib, you can't pip install it

[safe hedge]

Yeah. I am not talking about installing a Python package. Rather a packaged Python. Maybe you misunderstood

[quick snow]

Then look into what the official package depends on

[unkempt rock]

Anyone who can help me pick a pc dm me

[halcyon galleon]

It would be a question to ask in on of the offtopic rooms.

[unkempt rock]

Where’s that

[safe hedge]

Is there anything (in a pep for example) that forbids or advises against multiline error message strings?

[rose schooner]

not that i know of

[visual shadow]

!ot rooms are under offtopic

[fallen slate]

Off-topic channel: #ot2-never-nester’s-nightmare

Please read our off-topic etiquette before participating in conversations.

[proven thistle]

7961684944-4949165165-4984941

what type of encoding is this (in site requests)

[boreal umbra]

@hoary coyote your question was off-topic. try #databases

[white nexus]

what was the point of this?

https://docs.python.org/3/library/functions.html#classmethod

Changed in version 3.9: Class methods can now wrap other descriptors such as property().

Changed in version 3.11: Class methods can no longer wrap other descriptors such as property().

[elder blade]

bugs 🥴

[white nexus]

what happened

[elder blade]

It looks like this issue has become a sort of management issue for it https://github.com/python/cpython/issues/89519

[young helm]

why i can't use a pip in my python program ?

[lusty scroll]

I've been looking for a list of all possible interpreter-recognized dunders. Does this look about right?

def __abs__(self) -> T_co:
def __add__(self, x):
async def __aenter__(self) -> "Timeout":
async def __aexit__(
def __aiter__():
def __and__(self, other):
async def __anext__(self):
def __await__(self):
def __bool__(self):
def __bytes__(self):
def __call__(self, ttype, tstring, stup, etup, line):
def __class__(self):
def __class_getitem__(cls, params):
def __complex__(self):
def __contains__(self, f):
def __copy__(self):
def __del__(_self):
def __delattr__(self, name):
def __delete__(self, obj):
def __delitem__(self, index):
def __dir__(self):
def __divmod__(self, other):
def __enter__(self):
def __eq__(self, other):
def __exit__(*args):
def __float__(self):
def __floordiv__(self, other):
def __format__(self, fmt):
def __fspath__(self):
def __ge__(self, other):
def __get__(self, obj, cls=None):
def __getattr__(self, name):
def __getattribute__(self, name):
def __getinitargs__(self):
def __getitem__(self, index):
def __getnewargs__(self):
def __getstate__(self):
def __gt__(self, other):
def __hash__(self):
def __iadd__(self, value):
def __iand__(self, other):
def __imul__(self, value):
def __index__(self) -> int:
def __init__(self, *args, **kwargs):
def __init_subclass__(cls):
def __instancecheck__(cls, instance):
def __int__(self):
def __invert__(self):
def __ior__(self, other):
def __isabstractmethod__(self):
def __isub__(self, other):
def __iter__(self) -> Iterator[List[str]]:
def __ixor__(self, other):
def __le__(self, other):
def __len__(self):
def __lshift__(self, other):
def __lt__(self, other):
def __missing__(self, b):
def __mod__(self, args):
def __mro_entries__(self, bases):
def __mul__(self, scalar):
def __ne__(self, other):
def __neg__(self):
def __new__(meta, name, bases, dict):
def __next__(self):
def __or__(self, other):
def __pos__(self):
def __pow__(self, exponent, modulus=None):
def __prepare__(metacls, cls, bases, **kwds):
def __radd__(self, other):
def __rand__(self, other):
def __rdivmod__(self, other):
def __reduce__(self):
def __reduce_ex__(self, proto):
def __repr__(self) -> str:
def __reversed__(self):
def __rfloordiv__(self, other):
def __rlshift__(self, other):
def __rmod__(self, template):
def __rmul__(self, other):
def __ror__(self, other):
def __round__(self, ndigits: int = 0) -> T_co:
def __rpow__(self, base):
def __rrshift__(self, other):
def __rshift__(self, other):
def __rsub__(x1, x2):
def __rtruediv__(self, key):
def __rxor__(self, other):
def __set__(self, obj, value):
def __set_name__(self, cls, name):
def __setattr__(self, name, value):
def __setitem__(self, index, element):
def __setstate__(self, state):
def __sizeof__(self):
def __str__(self):
def __sub__(self, other):
def __subclasscheck__(cls, other):
def __subclasshook__(cls, subclass):
def __truediv__(self) -> 'SimplePath':
def __trunc__(self):
def __typing_subst__(self, arg):
def __xor__(self, other):``` I started with the dunder names in `_PyRuntime.global_objects.singletons.strings.identifiers` and then searched for (at least one) corresponding member/function defined in cpython `.py` source files. thoughts?

[sacred yew]

signatures for __aenter__, __call__, __exit__, __iter__,__truediv__ are wrong

also aexit

[lusty scroll]

really? hmm, those are directly from source files... maybe some dont have the correct signature

which is a little oof

i guess other than that these are mostly legit

[elder blade]

If you have __setstate__(), __getstate__() and __copy__() you might as well add __post_init__()?

[elder blade]

__aenter__() is correct apart from the return type.

[lusty scroll]

hmm, so theres at least one missed

these are the names with dunders stripped off:
abc_tpflags abs abstractmethods add aenter aexit aiter all and anext annotations args await bases bool build_class builtins bytes call cantrace class class_getitem classcell complex contains copy del delattr delete delitem dict dir divmod doc enter eq exit file float floordiv format fspath ge get getattr getattribute getinitargs getitem getnewargs getnewargs_ex getstate gt hash iadd iand ifloordiv ilshift imatmul imod import imul index init init_subclass instancecheck int invert ior ipow irshift isabstractmethod isub iter itruediv ixor le len length_hint lltrace loader lshift lt main matmul missing mod module mro_entries mul name ne neg new newobj newobj_ex next notes or orig_class origin package parameters path pos pow prepare qualname radd rand rdivmod reduce reduce_ex repr reversed rfloordiv rlshift rmatmul rmod rmul ror round rpow rrshift rshift rsub rtruediv rxor set set_name setattr setitem setstate sizeof slotnames slots spec str sub subclasscheck subclasshook truediv trunc typing_subst warningregistry weakref xor

since post_init isnt there, guess it must use a slightly different mechanism

or there was no need to list it in identifiers 🤔

this is what gave me the clue to look in identifiers:

static PyObject * slot_nb_int(PyObject *self) {
    PyObject* stack[1] = {self};
    return vectorcall_method(
      &(
        _PyRuntime.global_objects.singletons
            .strings.identifiers.___int__
            ._ascii.ob_base
        ),
        stack, 1
    );
}```

that was the (macro expanded) code i saw running when int() was called on a UserString (which defines __int__)

[elder blade]

Is this tp_slots?

[lusty scroll]

slots yes, but i dont know what tp_slots is ?

[elder blade]

I think that's the struct where a type's dunder methods are stored

[grave jolt]

sort of

https://www.youtube.com/watch?v=qCGofLIzX6g

[lusty scroll]

leaky internals
say it not so ..

lmao

"no python programmer knows what happens when you add two objects together"

so there's two slots for __add__

[boreal umbra]

In [6]: class SubclassMe:
   ...:     _subclasses = []
   ...:     def __init_subclass__(cls, new_class):
   ...:         cls._subclasses.append(new_cls)
   ...:

In [7]: class NewClass(SubclassMe):
   ...:     pass
   ...:

TypeError: __init_subclass__() missing 1 required positional argument: 'new_class'

this is not what I expected.

[peak spoke]

subclasses is already a dunder fwiw

[quick snow]

Isn't __init_subclass__ called only with the subclass?

[fallen slate]

Hey @dusk comet!

It looks like you tried to attach a Python file - please use a code-pasting service such as https://paste.pythondiscord.com

[dusk comet]

https://paste.pythondiscord.com/yijitiqetu
list of all dunders mentioned in documentation

[brazen mesa]

hi

[pliant tusk]

!e ```py
class subclass_me:pass

class new_class(subclass_me):pass

print(subclass_me.subclasses())```

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

[<class '__main__.new_class'>]

[prime estuary]

Yep, the other arguments passed in should be keyword only, and they're set to keywords set in the bases section of the class definition (excluding metaclass).

[vestal sequoia]

Hey guys, not sure if this is the correct channel, but is there a way to check which of my 3.9 libraries have been ported to 3.10 and "auto install" them?

[feral island]

there are various levels of safety vs. effort. you can just pip install them; if that works, the library probably largely works too (though there could be runtime incompatibilities). you could download package metadata and check trove classifiers to see if they have declared support for 3.10 (but it's likely that many libraries that haven't declared such support actually work fine); you can download the source code and run the test suite yourself on 3.10 (but you have to figure out how to do that)

[timber dagger]

@wild ferry well the "usage point of view" there introduces an extra thing that's just not there and that in fact it is impossible for the interpreter to keep track of
so even if technically your way of looking at things leads to the same results, other than memory usage and usage of is/id, i don't think it's particularly useful, especially if you're correcting someone who was talking about the way it does actually work

[wild ferry]

Yeah ig, I wanted to explain the person in more simple terms what the actual implications are, without going into how Python actually works. I wanted to offer a practical view of how it seems to work, maybe in part because it helps me to look at it this way and different people use different "tools" or memory models how to approach things.

[timber dagger]

well to me the actual implications of what you said are that "primitive"/"immutable" are concepts that exist and can be observed, which actually they can't be observed (checking if an object is mutable is RE-hard)

[wild ferry]

Yes I used the wrong term "primitive", I apologize (was studying Rust yesterday and it got mixed up in my head)

[full jay]

Odd question, are Decimal objects arbitrarily large like the ints are? Or is it still restricted to the float limits

[hallow steeple]

hi

jow
Kkk kkk

[timber dagger]

the precision is something you can set ```py

from decimal import *
getcontext()
Context(prec=28, rounding=ROUND_HALF_EVEN, Emin=-999999, Emax=999999, capitals=1, clamp=0, flags=[], traps=[InvalidOperation, DivisionByZero, Overflow])
Decimal(1) / 3
Decimal('0.3333333333333333333333333333')
getcontext().prec = 100
Decimal(1) / 3
Decimal('0.3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333')
len('3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333')
100
getcontext().prec = 200
Decimal(1) / 3
Decimal('0.33333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333')```

[feral cedar]

oh, nevermind, the max precision is https://docs.python.org/3/library/decimal.html#decimal.MAX_PREC

999999999999999999
for 64-bit

[full jay]

So as long as you set it ahead of time, it can be essentially arbitrarily large?

[hallow steeple]

hi

[timber dagger]

yeah pretty much, although not really in the same way as ints since there are some very finite calculations (like 1/3) that produce infinite results that have to be truncated at some point

[full jay]

I just thought it stored it differently

Not sure how to explain what I mean

I might be thinking of fractions maybe?

[timber dagger]

fractions is infinite-precision rational arithmetic using integers

using stuff like a/b + c/d = (ad+bc)/bd

decimal is literally base 10, so it does stuff like "1/3 * 3 = 0.99999999999999999999"

[elder blade]

Decimal is for much higher precision decimals (floats). I think you were looking for fraction yeah

[feral cedar]

Decimal isn't a float actually, it's fixed precision

[elder blade]

Yes, that's the point. Floats are standardized in how they're laid out.

Sorry I made the comparison to float because that's the "default number type" when you need decimals

[feral island]

float is also fixed precision, just base 2 instead of base 10, right?

[peak spoke]

floats get more precise the smaller they are as they can use more of the data for the decimal part

[radiant garden]

fixed precision vs fixed point

they probably meant the latter

(though decimal also has floating point types)

[zenith briar]

does longer bytecode even matter?
so example if a line of bytecode instruction that does the same thing as 4 lines of bytecode, would it even matter

[radiant garden]

depends

different ops take different lengths

and 3.11 can specialize + optimize individual ones to be faster in certain uses

in general the amount of ops matters less than what ops you're executing (function calls have more overhead for example)

[elder blade]

Technically does for the GIL, but I do not think you should micro-optimize with bytecodes

[raven ridge]

The big difference between float and Decimal is that float is floating point in base 2, and Decimal is floating point in base 10

So in that way, float, Decimal, and Fraction all store things differently, but float and Decimal are more similar

[unkempt rock]

@charred terrace

@charred terrace

accept friend request

i saw your github

@charred terrace

@charred terrace

@charred terrace

[raven ridge]

!tempban 931623751802572913 30d Ping spamming

[fallen slate]

:incoming_envelope: :ok_hand: applied ban to @little valve until <t:1658159975:f> (29 days and 23 hours).

[unkempt rock]

How do I merge my master branch to main branch?
I'm a beginner and pushed my project onto github.

[quick snow]

Try #tools-and-devops or a regular help channel

[unkempt rock]

Ok

[vast saffron]

With the changes to the classmethod decorator coming

-> no more

@classmethod 
@property
def i_am_a_class_property(cls):
    …

There are some hints how it can be achieved via __wrapped__.

Anyone have some idea or maybe can point me to some description, how it can be done in the future and what the best practice should be?

||not talking about if classmethod properties are best practice, but if it is done, how it should be done. This may also be important in making existing code base work in the future||

[quick snow]

!e

class clmp:
    def __init__(self, fn):
        self.__wrapped__ = fn
    def __get__(self, instance, cls):
        return self.__wrapped__(cls)

class Foo:
    @clmp
    def bar(cls):
        print("cls is", cls)

foo = Foo()

foo.bar
Foo.bar

[fallen slate]

@quick snow :white_check_mark: Your eval job has completed with return code 0.

001 | cls is <class '__main__.Foo'>
002 | cls is <class '__main__.Foo'>

[quick snow]

See above. If you also want setters and deleters, implement __set__ and __delete__

[vast saffron]

Ahh, ok I had a brainfart moment and did not even think of descriptors and forgot that both previous decorators are also descriptors.

Thank you very much

[sacred yew]

hm can you link me to the pep?

[next lion]

I don't think there was ever a pep for that?

There is this issue that talked about some of the problems https://github.com/python/cpython/issues/89519

[vestal sequoia]

Okay, thanks 🙂

[zenith briar]

class as a decorator?

what

[grave jolt]

what's wrong?

[zenith briar]

never seen that

is it new?

[grave jolt]

@foo
def bar(...):
    ...
``` is just syntax sugar for ```py
def bar(...):
    ...

bar = foo(bar)
```, so there's no restriction on what `foo` is

in fact, property, classmethod and staticmethod are all classes

[zenith briar]

also why doesn't __get__ return <method Foo.bar>?

[zenith briar]

woah

[grave jolt]

Because it calls __wrapped__, it doesn't return it

class clmp:
    def __init__(self, fn):
        self._fn = fn

    def __get__(self, instance, cls):
        return self._fn(cls)

[zenith briar]

oh yea ye i see

[west tinsel]

https://github.com/python/cpython/issues/91742

Tackling this

Weirdest issue

cf345e945f48f54785799390c2e92c5310847bd4 is the breaking commit https://github.com/python/cpython/pull/31373

I think removing this line is the issue: https://github.com/python/cpython/commit/cf345e945f48f54785799390c2e92c5310847bd4#diff-c22186367cbe20233e843261998dc027ae5f1f8c0d2e778abfa454ae74cc59deL4482

call_shape.postcall_shrink = 1;

I went to look at main and I couldn't find anything like it

That was because postcall_shrink was removed entirely some commits later

My idea is that postcall_shrink is supposed to be 1 and execution moves somewhere that has STACK_SHRINK being called with 2 - is_meth instead of 1

This could be wrong, but my idea is to write a bash script to change each of the lines containing 2 - is_meth one-by-one with 1 and then see if it fixes anything 😂

If that doesn't work, I'll check out compile.c

I should actually be able to try cloning into a different directory, checking out the commit and re-insertingcall_shape.postcall_shrink = 1;

[west tinsel]

Definitely not it. And I'm very lost

[west tinsel]

A function might erroneously be erroneously treated as a method as one idea, another is that the issue is somewhere else entirely like in the pdb-specific code and this issues appeared as a side-effect

Really unsure

[red solar]

@west tinsel it says it got merged though?

[west tinsel]

@red solar What do you mean?

The merged commit introduced the problem

[red solar]

they don't make sure the build passes before they merge?

[west tinsel]

The build did pass, this wasn't tested for and bugs happen occasionally

[red solar]

ah

well that's beyond me :/ too much

[west tinsel]

It's beyond me too 😂

I want to say I'm close but I really don't understand what's going on

[rose schooner]

are you sure it's a "dereferencing null pointer" issue

the mention of _PyMem_DebugRawFree makes me think it's a deallocating issue

[west tinsel]

I'll send the gdb output

You're correct, I need to check a few things

Program received signal SIGSEGV, Segmentation fault.
frame_stack_pop (f=Frame 0x7ffff7f9b078, for file /home/sam/tmp/pdbcrash.py, line 3, in function ()) at Objects/frameobject.c:421
421        Py_DECREF(v);
(gdb) p v
$1 = 0x0
(gdb) bt

#0  frame_stack_pop (f=Frame 0x7ffff7f9b078, for file /home/sam/tmp/pdbcrash.py, line 3, in function ())
    at Objects/frameobject.c:421
#1  frame_setlineno (f=Frame 0x7ffff7f9b078, for file /home/sam/tmp/pdbcrash.py, line 3, in function (), 
    p_new_lineno=<optimized out>, _unused_ignored=<optimized out>) at Objects/frameobject.c:641
#2  0x00005555557023bf in _PyObject_GenericSetAttrWithDict (dict=0x0, value=1, name='f_lineno', 
    obj=Frame 0x7ffff7f9b078, for file /home/sam/tmp/pdbcrash.py, line 3, in function ()) at Objects/object.c:1387
#3  PyObject_GenericSetAttr (obj=Frame 0x7ffff7f9b078, for file /home/sam/tmp/pdbcrash.py, line 3, in function (), 
    name='f_lineno', value=1) at Objects/object.c:1446
#4  0x0000555555700dac in PyObject_SetAttr (
    v=v@entry=Frame 0x7ffff7f9b078, for file /home/sam/tmp/pdbcrash.py, line 3, in function (), 
    name=<optimized out>, value=value@entry=1) at Objects/object.c:1024
#5  0x000055555564324f in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, 
    throwflag=<optimized out>) at Python/ceval.c:2878

Backtrace

(gdb) l
416    
417    static void
418    frame_stack_pop(PyFrameObject *f)
419    {
420        PyObject *v = _PyFrame_StackPop(f->f_frame);
421        Py_DECREF(v);
422    }
423    
424    static PyFrameState
425    _PyFrame_GetState(PyFrameObject *frame)

[radiant garden]

double free?

[short hearth]

class Foo:
    pass

for name in ("bar", "baz", "qux"):

    def method(self):
        print(name)

    setattr(Foo, name, method)

foo = Foo()
foo.bar()
foo.baz()
foo.qux()

Output:

qux
qux
qux

Bug or feature? 🙂

[spark magnet]

neither, it's just the way Python works.

you can make it work as you want:

for name in ("bar", "baz", "qux"):
    def method(self, name=name):
        print(name)

[short hearth]

uh 🙂

[spark magnet]

success?

[short hearth]

do you know which PEP defines this behavior? or does it predates PEPs, maybe?

[spark magnet]

it's not a pep, it's just how scoping and default values work.

[short hearth]

nop, that won't help in my case

[spark magnet]

why not?

[short hearth]

well, most things that make Python work are specified in PEPs

[short hearth]

because I can't modify the function signature, the purpose is to monkey patch a class with proxy methods

[spark magnet]

but you are defining the methods. So you can add a default argument that will never be used

[feral island]

another way is to define the function nested in another function that you immediately execute

[short hearth]

I would prefer not and have probably more elegant ways to achieve my end goal / the reason I'm doing such hacks

like simply writing a decorator out of the closure and using it within

but tbh I'm not looking for help as to how to fix my stuff

I was just wondering what justified that late defined functions do not respect closures

[peak spoke]

closures capture names, not values

[short hearth]

right

thanks

[peak spoke]

at the time of execution the name refers to the last string in the loop

[short hearth]

indeed

any idea of the rationale?

(as to why closures use names instead of values)

[peak spoke]

probably because it works like everything else does

[short hearth]

everything else?

[peak spoke]

the values of all names are looked up from the relevant scope when the function is executed; doing it differently to closures would be an exception to that

[short hearth]

would you be able to share a simple example of code that's not using a closure and still exacerbates this design?

[peak spoke]

using globals or builtins; didn't even notice your code is not using a closure as it's referring to globals

[short hearth]

in fact yes, this is not a closure indeed, thanks for mentioning it 😅

it's funny that it feels at the same time very normal to me now and yet, somehow I was expecting the current value at definition time to be stored somewhere, as in method.__globals__ or something

[spark magnet]

Why isn't this a closure? You have a function using a non-local name.

[short hearth]

because it's not defined within another function

[timber dagger]

because the non-local name is actually global

[spark magnet]

ok, then the reason it works this way is because you are accessing a global, so of course you get the current value of the global. why is that surprising?

[short hearth]

as I said, it's not anymore

I would even self-flagellate as to saying that this is Python 101

for some reasons 10 years of Python development just went over my head while writing these lines

[timber dagger]

i'm fairly sure python 101 isn't out yet, we're still on python 3

[spark magnet]

Python 0b101 isn't that far off.... 🙂

[feral island]

but we'll never have Python 0101 🙂

[short hearth]

there's no dumb question but the more I look at this snippet, the more I feel that it actually was a dumb question

I'll blame my current COVID and get some sleep, thanks for the help pals 🙂

[elder blade]

I know how that is, take care!

[rich cradle]

would a question about how type checkers work with a certain feature be more appropriate for here or #type-hinting?

[feral island]

probably #type-hinting but I think either is fine

[static bluff]

Any exciting things coming down the pipe?

[rose schooner]

wdym by that

[static bluff]

I dunno — this a channel for the discussion on python internals

You guys probably have a finger on the pulse of the language

Know which way the winds are blowing. So whats up? 🙂

[rose schooner]

there's an issue about adding BINARY_SLICE and STORE_SLICE

[static bluff]

https://tenor.com/view/eating-popcorn-im-watching-interested-gif-14656217

[verbal escarp]

hey @feral island how much effort would it be to realize conditional in the for-loop head, symmetrical to how comps work? like

for item in mylist if item > 0:
  ...

there's a long, contrived and noxious debate on https://mail.python.org/archives/list/[e-mail]/thread/7IXPROG2ANAFZTHZC4G3HRVXSKRIPXDL/ about this, and i've wished for something like that before, so i'm genuinly curious

[rose schooner]

you have to:
change the for-loop node in Parser/Python.asdl to add a new field
change the grammar in Grammar/python.gram
add visitors for the new field in Python/ast.c, Python/ast_opt.c, and Python/symtable.c
add handling for the field in Python/compile.c
then regenerate and build

no need to add opcodes

[rose schooner]

it will probably have the same code generation as just doing ```py
for item in mylist:
if item > 0:
...

[verbal escarp]

sounds like it should be fairly simple

[rose schooner]

yeah well it may just be

[verbal escarp]

guess someone just needs to do it.. wished i had the time for it 😔

[vast cloud]

im curious. is "better to try and ask for forgiveness than ask before".. or whatever that python rule goes, still relevant?

I feel like it is kinda colliding with readability, now that newer python versions have brought new options

E.g.

my_dict = {}

if "key" in my_dict:
  print(my_dict["key"])

# VS

try:
  print(my_dict["key"])
except KeyError:
  pass

Or is that still the general consensus for more complex cases (which the example above is not)

[grave jolt]

I think it really depends on the use case

I generally find try-except a bit hard to reason about, but that's just me being weird

Sometimes LBYL is not desired because it can cause concurrency and security issues. In Python EAFP is implemented with exceptions, so that's what you'll have to use

[verbal escarp]

my_dict.get("key", sentinel) is more en vague i think

none-aware operators, sentinels..

basically what's needed to chain functions together

which is difficult to do with try/except or checking and be imperative

i think EAFP is most common for IO and networking

where you have the callback/errback pattern

i have yet to see a good approach though to branching to different exceptions

like, you try to pull a json from a websource, there are so many different things that can go wrong and how to handle things best depends on context, which can easily get very complicated

[feral island]

think you already got a reply, but yes it's pretty easy to implement

[south hearth]

is there a way in dis to identify if a opcode begins with the following :

18 LOAD_GLOBAL             12 (__armor_enter__)

the instruction width can be anything > 10
but the instruction offset will always be 18 and current instruction will always be LOAD_GLOBAL

[sturdy timber]

My first PR merged into CPython yay https://github.com/python/cpython/pull/92894 😄

[feral island]

nice, congrats!

[boreal umbra]

amazing

[boreal umbra]

@crude plover this server is not a hiring platform, so I removed your messages. Please review our #rules

[rose schooner]

wdym by "instruction width"

do you mean the co_names index or the length of the name

[south hearth]

i meant the offset width (12)

[rose schooner]

yeah that's the co_names index

[south hearth]

yep

[rose schooner]

so what do you want the output to look like

[south hearth]

i wanna detect if theres is an opcode as such and remove the opcode along with opcode till 30 if that exists

[rose schooner]

ok but why do you have to do that

that would break the code

[south hearth]

if i dont that would break the decompilation of the code lol, cuz the opcode is produced by pyarmor

[rose schooner]

ok

[rose schooner]

what does "along with opcode till 30" mean

[south hearth]

             18 LOAD_GLOBAL             10 (__armor_enter__)
             20 CALL_FUNCTION            0

257          22 POP_TOP
             24 NOP
             26 NOP
             28 NOP
             30 SETUP_FINALLY           86 (to 118)

till setup finally

[rose schooner]

yeah idk

it's just too specific it feels wrong

[south hearth]

then just tell me how do i detect invalid opcode?

that would do as well

[rose schooner]

from dis import opmap

a = code.co_code.find(opmap["LOAD_GLOBAL"])
if a != -1 and code.co_code[a + 1] > 10:
    # handle invalid opcode at index `a`

[heady mauve]

If ints have no max size, how does Python manage them getting larger and larger?

[timber dagger]

larger integers take up more memory

and then there's a number at the start saying how many digits it has

[heady mauve]

Ok, thank you

[timber dagger]

so technically there is a maximum, it's just that the maximum is something stupidly large like 2^2^64 that you're never going to run into

[heady mauve]

right

is it just one word containing the digit count?

[timber dagger]

although in practise the real limit is how much memory you have

[timber dagger]

it's a Py_ssize_t

which on my computer is a 64-bit int, hence a theoretical limit of somewhere around 2^2^64

[feral island]

your computer doesn't actually have that much memory though

[neat delta]

the wolfram-alpha feature of @neon trout doesn't seem to like me right now, but for those curious, 2^2^64 has 5_553_023_288_523_357_133 (~5e18) digits

[raw flicker]

For android app development. I heard some field of flutter. So when we need to start learning flutter after learning java ? Or can directly go and learn flutter?What difference we get if we not use flutter and just use java for making apps?

[rose schooner]

implemented ```py

for x in range(-10,10) if x > 0: print(x)
...
1
2
3
4
5
6
7
8
9

[verbal escarp]

😮

[verbal escarp]

guess i'll have to find someone now to help me write a PEP for it 😄

[spark magnet]

i think this idea has been floated before

[native flame]

i agree with the argument that even with reasonably long enough variable names that would have to be split over multiple lines anyway

[quick snow]

The one advantage it has is that you don't have to scan for an else block or something else following the if

But it doesn't pass my "enough of a gain for the additional complexity" threshold

[grave jolt]

I don't like cramming so much stuff in one line

I know people who would put a walrus operator into that if clause as soon as I look away 👀

[rose schooner]

||aw i was thinking of multi-statement headers :(||

[elder blade]

If you support that, then what's the point haha

[verbal escarp]

it would be simply symmetrical to a comprehension..

nice and simple

[verbal escarp]

yeah, there's a long and derailed thread on the ideas mailinglist

[peak spoke]

an inverted if with a continue at the top is also nice and simple

[verbal escarp]

but not symmetrical to a comprehension - symmetry = simplicity

and you wouldn't want to cram too much into a comprehension either, so same argument holds

[peak spoke]

Having the syntax in comprehensions is a necessity for filtering, it's not a necessity for a normal statement

I'd expect them being more similar to cause even more confusion for beginners that already try to mix them and find out it doesn't work

[verbal escarp]

example?

[peak spoke]

I don't have one I could just link to, but I've seen plenty of people here trying to use statements like the ones you'd use in a normal for loops in comprehensions (e.g. an if), or using comprehensions for something a normal for loop would be suited for more

[verbal escarp]

that's actually supporting my point - people expect symmetry just by how similar things look, so why not make it symmetrical?

[verbal escarp]

well.. i'm sure you could argue that filter with lambda would've been an adequate alternative for comprehensions, too, but now we've got the construct with the conditional, so..

[peak spoke]

Not really, the lambdas necessary for those look horrible

[verbal escarp]

totally agree 🙂

but just for the sake of argument, the conditional isn't necessary.. and changing how lambdas look could've solved the uglyness-issue

now that we have a beautiful conditional though, i don't think we need to make do with filter and other ugly constructs in for-loops either

[white nexus]

aight i know this isn't the best channel for this but I'm trying to connect to a python language server from python, are there any lsp clients written in python?

(i can't think of a different channel where the people who might know this actually frequent in large enough numbers for the above question to be noticed)

[round swallow]

#tools-and-devops is probably the channel you want @white nexus

maybe #editors-ides

[frigid bison]

how does python know the required stacksize

couldnt that be dynamic?

how can they statically analyze it

and also what is it used for

[peak spoke]

required stack size of what?

[frigid bison]

of a code object

co_stacksize

[timber dagger]

i don't think that's dynamic for any code that the interpreter ever actually produces

[frigid bison]

is it just to specify to c the length of the array?

[timber dagger]

it looks like it specifies the amount of memory to allocate

Objects/frameobject.c 873-874 c int slots = code->co_nlocalsplus + code->co_stacksize; PyFrameObject *f = PyObject_GC_NewVar(PyFrameObject, &PyFrame_Type, slots); Include/objimpl.h 186-187 c #define PyObject_GC_NewVar(type, typeobj, n) \ _Py_CAST(type*, _PyObject_GC_NewVar((typeobj), (n))) Modules/gcmodule.c 2306-2324 ```c
PyVarObject *
_PyObject_GC_NewVar(PyTypeObject *tp, Py_ssize_t nitems)
{
size_t size;
PyVarObject *op;

if (nitems < 0) {
    PyErr_BadInternalCall();
    return NULL;
}
size_t presize = _PyType_PreHeaderSize(tp);
size = _PyObject_VAR_SIZE(tp, nitems);
op = (PyVarObject *)gc_alloc(size, presize);
if (op == NULL) {
    return NULL;
}
_PyObject_InitVar(op, tp, nitems);
return op;

}Modules/gcmodule.c 2275-2292c
static PyObject *
gc_alloc(size_t basicsize, size_t presize)
{
PyThreadState *tstate = _PyThreadState_GET();
if (basicsize > PY_SSIZE_T_MAX - presize) {
return _PyErr_NoMemory(tstate);
}
size_t size = presize + basicsize;
char *mem = PyObject_Malloc(size);
if (mem == NULL) {
return _PyErr_NoMemory(tstate);
}
((PyObject **)mem)[0] = NULL;
((PyObject **)mem)[1] = NULL;
PyObject *op = (PyObject *)(mem + presize);
_PyObject_GC_Link(op);
return op;
}```

so basically it gets messed with a bit and eventually turns into the number of bytes to allocate

[frigid bison]

thanks

[rose schooner]

Python/compile.c computes the stacksize for a piece of code

[boreal umbra]

Do any other PSF members have anything to say about the PSF board of directors election? I don't really feel qualified to decide (even in a small way) on a bunch of people I haven't met.

[spark magnet]

the best you can do is read their statements to decide

[boreal umbra]

just tell me who to vote for /s

[rose schooner]

https://github.com/python/cpython/commit/c0453a40faaadb43d2e69767af6c9680f8689063 it's done now btw

[icy crater]

i like my internal self

[heady mauve]

As I understand it, CPython has a call stack full of frames, one for each function call in the program. Each frame contains the code object, a data stack, and a block stack. I believe the height of the call stack is limited by the amount of memory available, but is this also the case for the data stacks and block stacks, or is there a hard limit on them?

[raven ridge]

The height of the call stack isn't limited exclusively by available memory. The native stack size for a thread is fixed at the time when the thread is spawned, and Python calls (at least up till Python 3.11) always consume a frame on the native call stack, so you can hit a limit on the call stack size long before you hit the limit of available memory

[safe hedge]

Is there a good/correct way to bundle CLI scripts with a package without requiring them to be individually manually specified in setup.py/setup.cfg? Something like scripts="bin/*"?

Effectively if we have a bin with a handful of scripts that utilise our package code can we have them added to the path/made callable by e.g. script1 --options etc etc as part of the package install?

I'm a little confused on the disadvantages of scripts vs console_scripts too.

[heady mauve]

Interesting... What determines the limit on the call stacks?

[elder blade]

There is a recursion limit you can adjust in the sys module

[elder blade]

It's arbitrary right now. Just a number someone decided somewhere.

[verbal escarp]

a number someone decided with "we should probably increase it as machines get more powerful"

in mind

[spark magnet]

if you make it too large, your Python can crash hard

[elder blade]

Stack Overflow

[verbal escarp]

afaik they refrain from increasing the number to avoid weird incompatibilities between machines, but i guess the number could also be decided based on actual machine stats

[frigid bison]

What would be incompatible if the limit is increased?

[verbal escarp]

portability

you increase it to match current machines, it may not run on an older machine, just because of that

[spark magnet]

there is no check on the stack literally overflowing and corrupting memory. If you raise the limit too high, you could cause a real crash.

[heady mauve]

Oh yeah that number

[lofty rover]

Interesting issue I encountered today

I was generating a number of indexes from a list using Random

However, even with a set seed it was not reproducible

The list was generated from an OrderedDict, or just a Dict since python 3.7 makes sure the dict retains order

however, the generation of the list from the ordereddict was pseudorandom every time, so my indexes were getting messed up.

[timber dagger]

are you sure it wasn't the way you were creating the ordereddict in the first place

[lofty rover]

Quite sure, that's just reading lines in a json

I can try to reproduce it though with a test case.

[elder blade]

Random outputs a set of numbers right?

Just test that on its own?

[cursive wharf]

Hi. I wonder if Python's object system is ever used in C-only projects?

Is that a thing?

[feral island]

Never heard of it though I can see the appeal. I guess the GIL would be limiting.

[cursive wharf]

Like you know how Objective C is C with an object library+runtime, plus some minor language syntax additions?

[feral island]

hardly minor 🙂

[cursive wharf]

😉

[tough osprey]

Hi, I was looking for the "re" logic but I could not find it in the main branch

I then realised it's in some version specific branches like: https://github.com/python/cpython/commits/3.10/Lib/re.py

any idea why?

[quick snow]

Perhaps it was recently changed from a module to a whole package? I see a folder called re in the main branch: https://github.com/python/cpython/tree/main/Lib/re

Yes, see https://github.com/python/cpython/issues/91308

[tough osprey]

I see, I'm trying to wrap my head around cpython and started with the file which just got moved... LOL thanks!

[unkempt rock]

x, y = 5, 6

In here, a tuple containing 5 and 6 respectively is being created and then the values are being unpacked to x and y, correct? Or does the interpreter smartly directly assign x and y(that's what my HS CS textbook says, though I believe it could have been said that way for the understanding of the students)

[native flame]

there's a special bytecode instruction to do it the smart way

ROT_TWO

[unkempt rock]

Interesting!

[peak spoke]

in that case I believe it does just unpack

[native flame]

oh right yes i overlooked that

ROT_TWO is just for a, b = b, a

[feral island]

              0 RESUME                   0

  1           2 LOAD_CONST               0 ((5, 6))
              4 UNPACK_SEQUENCE          2
              8 STORE_NAME               0 (x)
             10 STORE_NAME               1 (y)
             12 LOAD_CONST               1 (None)
             14 RETURN_VALUE

(on a recent 3.11 build)

so it doesn't create the tuple at runtime, the tuple is in the co_consts

[dapper lily]

UNPACK is its own instruction?

[unkempt rock]

It seems I have touched on an interesting topic

[unkempt rock]

That makes more sense

[feral island]

yeah, I think it iterates two things off the object at the top of the stack and pushes them onto the stack

[unkempt rock]

However, what about this

x, _* = (1,)

[feral island]

note that this is only true if the tuple contains only constants

[unkempt rock]

_ is an empty tuple

[unkempt rock]

By constants do you mean immutable data types?

Like ints?

[feral island]

with non-constants it uses a SWAP instruction and avoids the tuple ```In [12]: dis.dis("x, y = 5, z")
0 RESUME 0

1 2 LOAD_CONST 0 (5)
4 LOAD_NAME 0 (z)
6 SWAP 2
8 STORE_NAME 1 (x)
10 STORE_NAME 2 (y)
12 LOAD_CONST 1 (None)
14 RETURN_VALUE

[dapper lily]

but there's no swap happening

[feral island]

yes, immutable primitives, likely just str/bytes/int/float/bool/None and tuples containing those

[unkempt rock]

But how is this even determined! Surely manually raising an exception on the setters of every attribute wouldnt be the deciding factor

[unkempt rock]

Oh! Primitives

But I did hear recently that Python has no real "primitives"

But yes, the message comes across

[peak spoke]

it has no primitives, but python does know the types of literals and their behaviour

[unkempt rock]

Anyways, what you are saying is

a, b = ([1, 2], [3, 4])

does create a tuple and unpack it

[feral island]

try it yourself with dis.dis 🙂

[timber dagger]

i would guess that a tuple is probably created if the things it would have been made from were all produced by LOAD_CONST

[feral island]

I believe that's right

[flat gazelle]

literals, immutable views of literals/immutable views, and all views of literals/immutable views, only when on the RHS of in both as a boolean op and as part of a for statement go into the constant pool

Well, immutable view is kind of silly, there is just a tuple view that fits that

Since frozenset doesn't get special syntax

[feral island]

the compiler can generate frozenset literals though

[timber dagger]

or if two constant tuples are added together, or if a constant tuple is multiplied by a constant integer and the result is less than 256 elements long
based on my testing (with python 3.9.2)

[feral island]

              0 RESUME                   0

  1           2 LOAD_NAME                0 (x)
              4 LOAD_CONST               0 (frozenset({1, 2, 3}))
              6 CONTAINS_OP              0
              8 RETURN_VALUE

[flat gazelle]

Yup, constant pooled set views become frozenset Constants

[west tinsel]

https://github.com/python/cpython/issues/91821

Can this be closed?

[timber dagger]

in terms of observable behaviour (just looking at what the code does without dising it) and how code is actually executed after it's compiled there aren't really any primitives
but some stuff like integer literals gets evaluated in advance and stored in the code object to be loaded by LOAD_CONST

but there definitely aren't any types that are passed by value or anything like that

[unkempt rock]

Got it! So ints come closest to real primitives

[timber dagger]

well... integers, strings, tuples, code objects, True/False/None, and frozensets are the stuff that the compiler generates LOAD_CONSTs for

(and possibly other things that i'm forgetting)

but honestly i think there isn't really much reason to use the word "primitive" with python

everything is objects, it just happens that some of the builtin objects collaborate quite closely with the internals of the interpreter

[neat cypress]

do cpython bigint ops hold the GIL?

(i.e. if I have a bigint-heavy workload, would I benefit from threading at all)

[feral island]

looking at longobject.c, doesn't seem like we drop the GIL anywhere

[dapper lily]

idk where to post this, but I had a question about pattern matching with classes

match some_instance:
    case SomeClass(attr1="some_value", attr2=matched_attr):
        print(matched_attr)

why does this create another instance of SomeClass?

the PEPs for patma seem to imply otherwise

https://peps.python.org/pep-0636/#adding-a-ui-matching-objects

A pattern like Click(position=(x, y)) only matches if the type of the event is a subclass of the Click class. It will also require that the event has a position attribute that matches the (x, y) pattern. If there’s a match, the locals x and y will get the expected values.

A pattern like KeyPress(), with no arguments will match any object which is an instance of the KeyPress class. Only the attributes you specify in the pattern are matched, and any other attributes are ignored.

[feral island]

what makes you think it creates an instance?

[dapper lily]

!e

from dataclasses import dataclass

@dataclass
class C:
    attr1: str
    attr2: int

    def __post_init__(self):
        print(self)

c = C("c", 42)
match c:
    case C(attr1="c", attr2=matched_attr):
        print(matched_attr)

[fallen slate]

@dapper lily :white_check_mark: Your eval job has completed with return code 0.

001 | C(attr1='c', attr2=42)
002 | 42

[dapper lily]

uh wait a sec

hmm. no it doesn't. that's actually reasonable

mb

[rose schooner]

you're on 3.10 so you don't see the instruction