#internals-and-peps

1 messages Β· Page 158 of 1

sour thistle
#

!d hash

fallen slateBOT
#
hash 

hash(object)```
Return the hash value of the object (if it has one). Hash values are integers. They are used to quickly compare dictionary keys during a dictionary lookup. Numeric values that compare equal have the same hash value (even if they are of different types, as is the case for 1 and 1.0).

Note

For objects with custom `__hash__()` methods, note that [`hash()`](https://docs.python.org/3/library/functions.html#hash "hash") truncates the return value based on the bit width of the host machine. See `__hash__()` for details.
feral island
#

that only affects strings, right?

sour thistle
#

I think so but you should know that better than me πŸ˜…

#

it does affects anything that contains strings though

naive adder
#

ohh, ty

feral cedar
#

maybe you could include a == b implies hash(a) == hash(b)

feral island
#

being a core dev doesn't mean I know everything about the runtime πŸ™‚

naive adder
feral island
naive adder
timber tendon
#

i have a good python idea and was thinking of submitting a PEP, can i get some general advice on how to do this?

raven ridge
timber tendon
#

okie dokie

rose schooner
#

oh wait it's already mentioned in the above link

paper echo
fallen slateBOT
#

Python/hashtable.c lines 105 to 116

/* makes sure the real size of the buckets array is a power of 2 */
static size_t
round_size(size_t s)
{
    size_t i;
    if (s < HASHTABLE_MIN_SIZE)
        return HASHTABLE_MIN_SIZE;
    i = 1;
    while (i < s)
        i <<= 1;
    return i;
}```
feral island
fallen slateBOT
#

Python/hashtable.c line 241

size_t index = entry->key_hash & (ht->nbuckets - 1);```
paper echo
#

i have no idea what it means to & two ints πŸ˜†

feral island
#

bitwise and

#

it's basically modulo, but much more efficient because it's a much simpler operation at the CPU level

paper echo
#

huh

feral island
#

(bitwise and in general is not like modulo, but anding with a power of two minus 1 is)

paper echo
#

wow that is stupidly clever

#

it took me a second

# 
x = 0b1010111010001110101
y = 0b0000000000001110101
z = x & y

wait, i actually don't think i get how it works

#

wouldn't that always just give 0b1110101?

feral island
#

let's assume 8-bit integers

#

and our hash table is size 8

#

so then nbuckets - 1 = 7 = 0b0000111

#

anything & 0b0000111 means we basically set the first five bits to 0

paper echo
#

ohhhhhh

feral island
#

so you get out a value that has only bits within the last three set, ergo it's within range(8)

paper echo
#

ok so i did get it right, i just picked a weird example

#

and the point of making it a power of 2 is that you get a number where all the trailing bits are 1

#

whereas in the y example above, it doesn't work as modulus because there's a 0 in the trailing bits

#

whereas

x = 0b1010111010001110101
y = 0b0000000000001111111
z = x & y

would work, basically masking off the higher bits

#

dang

#

c people are smart

#

ty for explaining

#

and how does one decide on numbers like this?

#define HASHTABLE_MIN_SIZE 16
#define HASHTABLE_HIGH 0.50
#define HASHTABLE_LOW 0.10
#define HASHTABLE_REHASH_FACTOR 2.0 / (HASHTABLE_LOW + HASHTABLE_HIGH)

experimentation? some rule of thumb in a textbook?

feral island
#

probably some of both, plus Tim Peters's intuition

nimble crystal
#

Looks like this wasn't Tim Peters but Don Owens or Victor Stinner.

feral island
#

oh, is this hashtable only used for tracemalloc?

nimble crystal
#

Yes.

#

Dictionaries are implemented in Objects/dictobject

fallen slateBOT
#

Objects/dictnotes.txt line 120

Results of Cache Locality Experiments```
paper echo
nimble crystal
#

Happy to help! :)

#

I usually lurk, but this one caught my eye. :)

west tinsel
#

How do I interact with issues while bpo is being migrated?

#

Do I just take a break? πŸ˜‚

feral island
#

you take a break πŸ™‚

timber tendon
#

blahh

#

how do people here motivate themselves to write for a long period of time

paper echo
#

caffeine and adhd

raven ridge
#

Money.

rose schooner
ancient jackal
#

a combination of the previous three responses

west tinsel
#

Interesting project or an impending deadline

thorn flume
#

honestly pep 9001 had some good stuff

#

like higher line limits

#

(before your ide starts yelling at you)

surreal sun
#

coding around 4-6 hours consecutively fries my brain though, so I'd watch out for burnout

verbal escarp
sand goblet
elder blade
#

Have you pulled the most recent changes?

#

I assume that you have a fork? Then the origin, which is what git pull usually synchronizes from, is out of date.

#

You need to update your fork

spice pecan
leaden robin
#

Hello guys, I have a task at university, which I think is basically to make some implementation of something with the use of SciPy (sparse matrices). Do you have any topic and resources on that which can be helpful for my task :D. Thank you in advance!

peak spoke
sand goblet
#

Or in the download?

peak spoke
#

both? The release is generated from the code on github

sand goblet
peak spoke
fallen slateBOT
#

Lib/string.py lines 37 to 48

def capwords(s, sep=None):
    """capwords(s [,sep]) -> string

    Split the argument into words using split, capitalize each
    word using capitalize, and join the capitalized words using
    join.  If the optional second argument sep is absent or None,
    runs of whitespace characters are replaced by a single space
    and leading and trailing whitespace are removed, otherwise
    sep is used to split and join the words.

    """
    return (sep or ' ').join(x.capitalize() for x in s.split(sep))```
sand goblet
#

Oh

#

So it might appear in 3.11?

raven ridge
#

it will appear in 3.11

#

Eventually a 3.11 branch will be cut off of the main branch, and work for stuff that will go into 3.12 or later will continue on main, and fixes that are specific to 3.11 will go into the 3.11 branch.

sand goblet
#

Alright, that’s good. Then I hope they don’t decide to change back before 3.11 is released.

unkempt rock
#

Funny to read whats going on here :)

grave jolt
#

sounds really minor

spice pecan
#

a bunch of these are left around as things that newer contributors can take care of IIRC

#

and in some cases they may not be left around intentionally, but a beneficial change may still be discovered, even if it's pretty minor

raven ridge
#

tiny improvements like that aren't necessarily encouraged, but as long as they have a clear benefit they'll be accepted

rose schooner
raven ridge
#

generally, changes will be accepted if they bring more benefit than cost.

#

if you make something that's a tiny improvement that's hard to verify or test, or that requires a lot of reviewer time, it might not get accepted

#

larger improvements with clearer benefits, or changes that are easier to prove correct, are more likely to be accepted.

rose schooner
prime estuary
#

So, if you're have a look at GitHub you can watch all the issues appearing in real time.

elder blade
candid grove
#

Why is it possible to assign a variable to function?

In [16]: def fun():
    ...:     x = 1
    ...:     return x
    ...: 

In [17]: fun.x
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [17], in <cell line: 1>()
----> 1 fun.x

AttributeError: 'function' object has no attribute 'x'

In [18]: fun.x = 2

In [19]: fun.x
Out[19]: 2
rose schooner
west tinsel
#

Because functions are first class objects too

# 
Python 3.11.0a6+ (heads/main:9e88b572fb, Apr 10 2022, 10:41:18) [GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def function():
...     return 1
... 
>>> dir(function)
['__annotations__', '__builtins__', '__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__get__', '__getattribute__', '__globals__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__kwdefaults__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']
>>>
native flame
#

eg x = object(); x.a = 1 fails

west tinsel
#

Then the answer is that it hasn't been disallowed

#

I don't know the specific implementation for functions, but for items implemented in C and python, there are explicit checks and specific errors that are written when setting attributes

rose schooner
west tinsel
#

Then that's the full answer. Function objects use the generic function for setting attributes which doesn't disallow setting attributes with arbitrary names

peak spoke
#

They have a __dict__ which allows arbitrary attributes as it can be convenient to store data on them

paper echo
#

i don't remember what the purpose was, but the author of the program actually used custom function attributes to pass information around

#

i remember at the time thinking it was confusing and unnecessarily obscure software design, but i don't remember what exactly they were doing

#

file under "just because you can, doesn't mean you should"

inland acorn
#

<@&831776746206265384> ^

high verge
#

@humble phoenix Please don't post random TikTok stuff. Respect the channel topics.

grave jolt
feral island
fallen slateBOT
#

Lib/typing.py line 2487

f.__final__ = True```
paper echo
#

interesting!

prime estuary
#

Another example is lru_cache, which adds a function to clear the cache and another to return statistics.

pliant tusk
#

Functools wrap uses it

#

So does Flask

native flame
#

mildly interesting discovery: the error messages for using break and continue outside loops are a bit different

#

SyntaxError: 'break' outside loop
SyntaxError: 'continue' not properly in loop

#

yield and return outside functions are similar to break

west tinsel
#

I'm listed as a mannequin on the github issues, is this permanent?

#

By BPO account is linked to my github account

white nexus
#

i believe you're listed as a mannequin because your github account didn't actually make those comments

#

if you make a comment on a github issue with your account, does that comment show up as a mannequin?

west tinsel
#

No, it never does

white nexus
#

essentially mannequins are used by github when importing data from other sites, so comments that are echoed to github will show up as a mannequin

west tinsel
#

Will the mannequin change to be my account?

white nexus
#

no

west tinsel
#

Ok, thank you

#

That's a shame

spark magnet
west tinsel
#

It's every issue I'm in

spark magnet
west tinsel
#

Can I un-mis-align it? πŸ˜‚ πŸ˜‚

spark magnet
west tinsel
#

It's not a problem and thanks!

west tinsel
spark magnet
feral island
#

That's why @spark magnet isn't a mannequin

white nexus
#

well

#

technically not permanent if they become a member of the org :P

feral island
#

not sure if things will get updated if they join the org, I'd expect not

spark magnet
#

@west tinsel https://discuss.python.org/t/github-issues-are-now-live/14967#known-issues-3

Discussions on Python.org
GitHub Issues are now live!

GitHub Issues are now live! The migration from bugs.python.org to GitHub is now officially complete, and all the issues have been successfully transferred. πŸ₯³ You can now browse, create, and comment on issues here: Issues Β· python/cpython Β· GitHub bugs.python.org will remain available, with the following changes: It will be read-only and it w...

glass robin
#

Hey, what would a be polite way to request a review? The cpython PR has been waiting for 2 months

rose schooner
glass robin
#

yes

rose schooner
#

i mean you could mention the person you want to be reviewing your code and ask them

glass robin
#

I'm not waiting for anyone in particular, it'd be an initial review

#

pinging someone at random doesn't seem great

rose schooner
glass robin
west tinsel
#

Fancy seeing you here! πŸ‘‹

quick night
# 
  6          16 SETUP_FINALLY           10 (to 38)

what is (to 38) here??
\

pliant tusk
#

It shows that the finally block goes from index 10 to 38

prime estuary
#

PEP 678 Enriching Exceptions with Notes has been accepted. It adds a way to add arbitrary string notes to exceptions, which are displayed after the exception message in tracebacks.
https://peps.python.org/pep-0678/

rose schooner
#

pretty cool new changes though

#

python 3.11 is probably gonna be big

verbal escarp
prime estuary
#

They mentioned examples in the PEP, but it's broadly going to be useful most for frameworks and other sorts of libraries which might want to catch an exception, add some data, then re-raise.

#

For instance Hypothesis generates sample data to pass to test functions. If the test raises an exception, it can go add a copy of the specific bad values to the exception for you to look at.

#

It needs to be built into BaseException because any exception could be raised, and you don't know how its __str__ behaves.

verbal escarp
#

ok

#

not sure why they need an extra dunder for that though

#

or why they need a PEP for that

west tinsel
#

Requires a PEP because it's a relatively significant addition to the language

#

I assume the dunder attribute comes from the need to store extra information while maintaining backwards compatibility

grave jolt
#

stringly exceptions πŸ˜”

#

aka the format-all-the-details-into-a-string-so-that-I-have-to-parse-it-with-regex pattern

verbal escarp
grave jolt
#

yay?

rose schooner
#

...yay?

quick night
#

¿ʎɐʎ

grave grail
paper echo
#

some libraries do a really good job of adding structured metadata to their exceptions, like jsonschema

grave jolt
#

fair

solemn sparrow
#

I'm looking for guidance on using entry points/console_scripts vs python -m <module> within user level packaging. It seems a few days ago (in GH #24003), the last of the docs direct users of cli's included in Lib to use the python -m <module> syntax. I was wondering is calling python -m <module> going to be recommended for user developed modules too, or should those people stick with entry points. Sadly I can't find much more context even from the bug at bpo-22295. It seems that the work around this started before the now common standard of python3 on unix like distributions, so python is often displayed via argparse, but isn't what was actually called. I'm happy to make a PR related to actually looking at sys.argv[0] if that's what makes sense, but I'm just not sure what the recommendation is for the future.

paper echo
#

i don't think there is going to be any general movement against console script entry points. too many programs use it and too many people rely on it

#

(and people want it)

solemn sparrow
# paper echo i don't have any sense that this is going to change in general. i think the docs...

Right now it's nice that strings like this https://github.com/python/cpython/blob/main/Lib/json/tool.py#L20 are static. However we could detect if python3 was used when the command was called. Leave all the docs the way they are, but allow for help text to match the invocation of the command aka:
me@comp$ python3 -m json.tool -h usage: python -m json.tool [-h] [--sort-keys] [--json-lines] [infile] [outfile] could be dynamic and display
me@comp$ python3 -m json.tool -h usage: python3 -m json.tool [-h] [--sort-keys] [--json-lines] [infile] [outfile]

fallen slateBOT
#

Lib/json/tool.py line 20

prog = 'python -m json.tool'```
solemn sparrow
#

It might just be nice for users to be able to copy the help/usage and use it directly

#

Also, note I'm not asking for a feature, I could do this easily enough if people think it makes sense.

paper echo
#

that would be nice. i think even click requires you to hard-code your app name? i don't remember off the top of my head

#

maybe it'd be a good snippet to post online or even on pypi?

#

it'd be nice to have it in the standard libraries

solemn sparrow
#

django does something like this I believe

#

I'll take a look in their source, and see if I can find an example.

solemn sparrow
fallen slateBOT
#

django/core/management/__init__.py lines 208 to 210

self.prog_name = os.path.basename(self.argv[0])
if self.prog_name == "__main__.py":
    self.prog_name = "python -m django"```
paper echo
#

apparently (and unfortunately) it's impossible to get the "real" module name when it's invoked as __main__, without manually parsing __file__, which would require you to manually figure out where the top-level package is, which would be horrible to do

solemn sparrow
paper echo
#

the real problem is that not all programs have identical behavior between foo and python -m foo

solemn sparrow
# paper echo the real problem is that not all programs have identical behavior between `foo` ...

That's kinda back to my original question since I think all things within the standard library are now invoked via python -m foo so no worries there. However for user written modules I would suggest choosing one way of invoking via either via python -m foo or just foo and stick with it. The docs just don't say that yet, and maybe they should. Or maybe my idea is wrong and having a simple pattern (similar to what you suggested above) to handle everything is a better way forward.

paper echo
#

imo it's a good idea to support both in general

#

if you won't support foo then at least support python -m foo

#

e.g. debugpy only supports the latter

#

i think the former should always be a synonym for the latter, there's no reason to do it otherwise except to confuse and punish your users

solemn sparrow
fallen slateBOT
#

src/pip/_internal/utils/misc.py line 109

def get_prog() -> str:```
solemn sparrow
#

I get Usage: /usr/bin/python3 -m pip <command> [options] when I call it with python3 -m pip

paper echo
#

that makes sense too

#

they are really trying to emphasize which python you invoked pip with

solemn sparrow
paper echo
solemn sparrow
paper echo
#

yeah that'd be nice! or just @ me here

glass robin
feral island
#

(not right now though)

glass robin
#

Thank you, no rush it was more about the process in those cases

feral island
#

I would like to get eventually to a state where cpython PRs are all reviewed promptly, but right now there is 1.6k open ones 😦

tawdry nexus
#

Hi, I'm not sure if this should be in a help channel, but it does relate to Python's implementations of floating-point arithmetic.
I'm writing a paper in a class of mine about how Python performs addition between two floating point numbers and analyzing its source code. I'm actually not quite sure where to look for the + operator method and I'm also not sure if this goes deeper into how C performs floating-point arithmetic. Is there any good resources or does anyone have pointers on where to start looking (beyond the GitHub page for python)

feral island
fallen slateBOT
#

Objects/floatobject.c line 577

float_add(PyObject *v, PyObject *w)```
feral island
#

you can see that in the end, it really does just do C-level float addition πŸ™‚

tawdry nexus
#

Thank you so much for the fast response. This should be enough for me to go off of, thank you so much!

paper echo
#

!e ```python
assert "\u1680".strip() == ""

fallen slateBOT
#

@paper echo :warning: Your eval job has completed with return code 0.

[No output]
sterile shale
#

Can sombody pls add me?

paper echo
fallen slateBOT
#

Objects/clinic/unicodeobject.c.h line 330

unicode_isspace_impl(PyObject *self);```
rose schooner
fallen slateBOT
#

Include/cpython/unicodeobject.h lines 24 to 25

#define Py_UNICODE_ISSPACE(ch) \
    ((Py_UCS4)(ch) < 128U ? _Py_ascii_whitespace[(ch)] : _PyUnicode_IsWhitespace(ch))```
paper echo
#

ahh ty

#

thanks @rose schooner !

rose schooner
#

basically a LUT for whitespace

rose schooner
rose schooner
feral island
rose schooner
feral island
#

though that file might be autogenerated

rose schooner
fallen slateBOT
#

Objects/unicodetype_db.h line 1

/* this file was generated by Tools/unicode/makeunicodedata.py 3.3 */```
paper echo
# 
import unicodedata

def is_whitespace(c: str) -> bool:
    if len(c) != 1:
        raise ValueError("Must be a length-one string.")
    return (
        c in {"\t", "\n", "\x0b", "\x0c", "\r", "\x1c", "\x1d", "\x1e", "\x1f", " "} or
        unicodedata.bidirectional(c) in {"WS", "B", "S"} or
        unicodedata.category == "Zs"
    )
#

that's the python translation of it, i think

rose schooner
#

translates to ```py
def _isspace(self: str) -> bool:
return (
c in {"\t", "\n", "\x0b", "\x0c", "\r", "\x1c", "\x1d", "\x1e", "\x1f", " "} or
unicodedata.bidirectional(c) in {"WS", "B", "S"} or
unicodedata.category == "Zs"
)

def isspace(self: str) -> bool:
length = len(self)
if length == 1:
return _isspace(c)
if length == 0:
return False
for i in range(length):
c = self[i]
if not _isspace(c):
return False
return True

paper echo
#

the logic is still the same though right?

rose schooner
timber tendon
#

merp

white nexus
#

I was looking at asyncio.Semaphore and i realise it would probably work for one of my needs, but i need to see how many are remaining, which is not part of the documented api

#

the other problem is subclassing doesn't seem like it would work since semaphore inherits from private asyncio classes

#

my question is, what is the practice for a private attribute that i want to use?

#

should I use it?

#

the problem is this is in a library

west tinsel
#

I have

@unittest.skipIf(sys.platform == "win32", "Requires file permissions")
class TestsPermissionExtraction(unittest.TestCase):
    ...
    @unittest.skipUnless(os.getuid() == 0, "requires root")
    def test_extractall_preserve_all(self):
        ...
#

It fails on windows because getuid isn't provided

#

It makes sense because to evaluate the class definition, they decorator funtion has to be executed

rose schooner
west tinsel
#

Thanks, I'll do that

rose schooner
#

ok but wait i think i see a problem

#

does unittest.skipUnless raise an error if the condition is false

west tinsel
#

No

#

It just skips the test

rose schooner
#

ok good

icy halo
#

wait where do i write these python exams?

quick night
#

is there a guide on writing good peps

boreal umbra
# quick night *is there a guide on writing good peps*

I think you need a cpython core dev to sponsor your PEP. if you want to suggest a change to the language as an "outsider", you can go to the python-ideas email list. PEPs are occasionally borne of discussions there, though PEPs in general are few as compared to the number of ideas thrown out.

quick night
#

0.o i didnt know about that, interesting

verbal escarp
#

anyone who wants to write a PEP with me to introduce the triple *** for arrow notation? :>

feral island
verbal escarp
feral island
#

ah yes

verbal escarp
#

* is multiplication, ** is exponentiation, so logically *** would be tetration or up-arrow

#

too bad it would require operating on operators.. or somesuch 😦

#

maybe it could be implemented as a recursive closure

#

like a *** b only returns a function, which can be stacked with another *** c - so all *** operators in a term form a single one

timid ibex
#

dafuq is this

quick night
timid ibex
#

never heard of ***

elder blade
#

That's the point

ruby willow
#

What about ^^ if *** isnt feasible.... at least it will be distinguishable from xor

ruby willow
quick snow
#

Not sure what the use of tetration would be if the result usually doesn't fit in a float :)

verbal escarp
feral island
verbal escarp
feral island
verbal escarp
#

python

#

well, i wrote a simple tetration function with caching, standard lib

feral island
#

oh right, tetration doesn't go up as fast as I thought it would so it's still representable

#

still, you'll have a hard sell convincing people that tetration is a common enough operation to merit a new operator

verbal escarp
#

yeah 😦

#
  1. not even sure where to use tetration in the wild 2) not sure how to calculate it efficiently, if ever
#

but it'd be cool, you have to admit

rose schooner
verbal escarp
rose schooner
#

but i'm pretty sure people only need the first 3 hyperoperations

#

then 4 and above (tetration+) doesn't seem to have any use

#

there's also the fact they're gonna be expensive

verbal escarp
#

yeah.. maybe there's an application in statistics or so? i could imagine that stochastics might deal with huge numbers

#

but they usually don't try to calculate the numbers but solve things algebraic

west tinsel
#

I wonder if anyone would find this useful

#

The use-case I gave is very niche πŸ˜‚

#

The above example matches files that import the ast library. Queries can already be quite complex but I want to work on making them easier to write and I'm going to add statistical queries so you can work out how often certain types of code are being written

ruby willow
#

Check in niche langs and see if they have the operator if not it may be even harder to justify

paper echo
#

i see that they were discussed in the mailing lists, but i guess i don't see the difference still

paper echo
#

does pip actually use the build backend to install a package when a wheel is available? or does it have its own mechanism to unpack and install the files in the wheel, without using any build backend like setuptools?

west tinsel
# paper echo this is what dataclasses, attrs, and to some extent pydantic are for

Dataclasses are slightly different (__init__ would be overritten entirely) and attrs isn't stdlib. The proposal I was looking at was similar to the one I linked but not the same, it's syntactic support to automatically initialise marked attributes in the __init__ method. Like the following

class ExampleClass:
    def __init__(self, these, are, example, parameters,  word):
        self.these = these
        self.are = are
        self.example = example
        self.parameters = parameters
        do_something(word)

Which would be written as the following

class ExampleClass:
    def __init__(self, @these, @are, @example, @parameters, word):
        do_something(word)

I'm not too concerned with the linked proposal itself I just wanted to show my new tangentially related toy project that the proposal inspired me to make πŸ˜‚

#

But I do like the new syntax, I am waiting for it to be posted to python-ideas, if it doesnt happen then I'll do it myself and see how others feel about it.

elder blade
rose schooner
#

!pep 654

fallen slateBOT
#
**PEP 654 - Exception Groups and except***

Link

Status

Accepted

Created

22-Feb-2021

Type

Standards Track

rose schooner
#

how does this pep work

paper echo
west tinsel
rose schooner
#

‫but how would you do it if it was ever accepted

west tinsel
#

There are some other considerations too like typing that need to be worked out

rose schooner
west tinsel
rose schooner
#

oh yeah these things also rely on another argument before it

west tinsel
#

Oh, yes. With that in mind, does what I'm saying now make sense?

rose schooner
#

i guess

west tinsel
#

Ok, at least the implementation is plausible πŸ˜‚

paper echo
#

i'm not convinced though

#

normally in this situation i require an alternate constructor to make an instance

#

the "main" __init__ is reserved for "internal" use only

#

so you'd do MyThing.new(...) instead of MyThing()

quick night
#

#bot-commands message

#

why is walrus faster?

sturdy timber
#

!timeit x=2

fallen slateBOT
#

@sturdy timber :white_check_mark: Your timeit job has completed with return code 0.

5000000 loops, best of 5: 52 nsec per loop
sturdy timber
#

!timeit (x:=2)

fallen slateBOT
#

@sturdy timber :white_check_mark: Your timeit job has completed with return code 0.

10000000 loops, best of 5: 31.7 nsec per loop
sturdy timber
#

!e ```python
import dis
print("x=2:")
dis.dis("x=2")
print("(x:=2):")
dis.dis("(x:=2)")

fallen slateBOT
#

@sturdy timber :white_check_mark: Your eval job has completed with return code 0.

001 | x=2:
002 |   1           0 LOAD_CONST               0 (2)
003 |               2 STORE_NAME               0 (x)
004 |               4 LOAD_CONST               1 (None)
005 |               6 RETURN_VALUE
006 | (x:=2):
007 |   1           0 LOAD_CONST               0 (2)
008 |               2 DUP_TOP
009 |               4 STORE_NAME               0 (x)
010 |               6 RETURN_VALUE
sturdy timber
#

I think it's the LOAD_CONST (None) and RETURN_VALUE for x=2 which only there because it's the end of the code? Whilst the walrus DUP_TOPed so didn't need to load None to return.

#

So if you did a sequence of instructions then x=2 would be faster

feral island
#

Agree, presumably DUP_TOP is faster than LOAD_CONST None

quick night
quick night
feral island
quick night
#

oh

#

ok

west tinsel
west tinsel
#

Oh, are you talking about when using attrs?

#

If I was going to use attrs, I would probably do the same

#

I've also improved my query tool

Screenshot_from_2022-04-14_19-54-48.png Screenshot_from_2022-04-14_19-55-24.png
#

I picked the top 4 downloaded packages from pypi and put them in the sources directory

paper echo
feral island
paper echo
#

great, problem solved then

#

im not sure if it allows you to have that without a default value

#

so ok, fair. there is a use case for this

#

but does it justify brand new syntax and a very very special case in the language semantics? im not sure

#

i'd rather see things like with and except suffix expressions, null-safe binary operators and attribute lookups, etc.

rose schooner
# quick night why is walrus faster?

@sturdy timber doesn't seem to be faster anymore for python 3.11.0a7: ```py

import dis
print("x=2:");dis.dis("x=2");print("(x:=2):");dis.dis("(x:=2)")
x=2:
0 RESUME 0

1 2 LOAD_CONST 0 (2)
4 STORE_NAME 0 (x)
6 LOAD_CONST 1 (None)
8 RETURN_VALUE
(x:=2):
0 RESUME 0

1 2 LOAD_CONST 0 (2)
4 COPY 1
6 STORE_NAME 0 (x)
8 RETURN_VALUE

# 
>>> from timeit import main
>>> main(["x=2"])
20000000 loops, best of 5: 15.8 nsec per loop
>>> main(["(x:=2)"])
10000000 loops, best of 5: 21.4 nsec per loop
#

or wait πŸ€”

#

how does the bot time it

rose schooner
fallen slateBOT
#

@rose schooner :white_check_mark: Your eval job has completed with return code 0.

001 | x=2:
002 |   1           0 LOAD_CONST               1 (2)
003 |               2 STORE_FAST               0 (x)
004 |               4 LOAD_CONST               0 (None)
005 |               6 RETURN_VALUE
006 | (x:=2):
007 |   1           0 LOAD_CONST               1 (2)
008 |               2 DUP_TOP
009 |               4 STORE_FAST               0 (x)
010 |               6 POP_TOP
011 |               8 LOAD_CONST               0 (None)
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/ilogipasic.txt?noredirect

rose schooner
#

the python bot has some shenanigans going on with timing stuff

west tinsel
#

For me, the walrus operator is consistently slower

#

!e

import timeit
print(timeit.timeit("x=2"))
print(timeit.timeit("(x:=2)"))
fallen slateBOT
#

@west tinsel :white_check_mark: Your eval job has completed with return code 0.

001 | 0.048418553080409765
002 | 0.03021063096821308
west tinsel
#

Interesting

#

I'm assuming there's no noticeable difference and it's just noise

rose schooner
#

(x:=2) has 2 more instructions when fairly compared to x=2

#

!e ```py
from timeit import main
def foo0():
x = 2
def foo1():
(x:=2)
main(['-s', "from main import foo0", "foo0()"])
main(['-s', "from main import foo1", "foo1()"])

fallen slateBOT
#

@rose schooner :white_check_mark: Your eval job has completed with return code 0.

001 | 2000000 loops, best of 5: 99.9 nsec per loop
002 | 2000000 loops, best of 5: 110 nsec per loop
west tinsel
#

It becomes quite likely when the standard deviation is high or the difference in expected run time is small

rose schooner
#

IPython also shows that the walrus operator is slower ```py
In [1]: def foo0():
...: x = 2
...: def foo1():
...: (x:=2)
...:

In [2]: %timeit foo0()
71.2 ns Β± 17.5 ps per loop (mean Β± std. dev. of 7 runs, 10000000 loops each)

In [3]: %timeit foo1()
75.8 ns Β± 2.79 ns per loop (mean Β± std. dev. of 7 runs, 10000000 loops each)

#

by "fairly compared" i mean "properly disposing of the value that (x:=2) returns and making it consistent with x = 2"

flat gazelle
#

!e

import timeit
print(timeit.timeit("x=2;"))
print(timeit.timeit("(x:=2);"))
```is also a fun one
fallen slateBOT
#

@flat gazelle :white_check_mark: Your eval job has completed with return code 0.

001 | 0.046923632035031915
002 | 0.0305294890422374
rose schooner
feral cedar
#

why?

rose schooner
#

LOAD_* + DUP_TOP (COPY 0 in python 3.11) + STORE_* is faster than LOAD_* + STORE_* + LOAD_* for some reason

sturdy timber
#

With

import timeit
print(timeit.timeit("(x:=2);", number=10000000))
print(timeit.timeit("x=2;", number=10000000))
print(timeit.timeit("(x:=2);", number=10000000))
print(timeit.timeit("x=2;", number=10000000))

I'm getting consistently different behaviour running it on the bot vs locally for multiple python versions and I have no clue why. Unless I'm misreading the code it seems the code is put in a loop to be run (https://github.com/python/cpython/blob/35fef2711033ce793ec1cb43dfbd95e2d06ab7bb/Lib/timeit.py#L69-L78) meaning that the walrus should probably be noticeably slower... but no.

fallen slateBOT
#

Lib/timeit.py lines 69 to 78

template = """
def inner(_it, _timer{init}):
    {setup}
    _t0 = _timer()
    for _i in _it:
        {stmt}
        pass
    _t1 = _timer()
    return _t1 - _t0
"""```
quick night
rose schooner
#

since the first constant is the function code object in that code

quick night
#

ah i c

frosty topaz
#

can we have a frozendict already

grave jolt
fallen slateBOT
grave jolt
#

πŸ™‚

#

If you don't want a HAMT buy just a read-only mapping, there's types.MappingProxyType

frosty topaz
#

too bad it doesn't have an efficient set and list implementations

#

@grave jolt also I am a bit saddened that x.update(y) is O(len(y)). Does that have to be the case?

grave jolt
fallen slateBOT
grave jolt
frosty topaz
#

pyrsistent is not efficient, I think x.add(y) is O(len(x))

grave jolt
#

From the readme on sets:

Random access and insert is log32(n) where n is the size of the set.

#

not sure why they specify 32 but yeah

frosty topaz
#

ah sorry, just slower, perhaps not asymptotically.
copying some work of a colleague:

immutables.Map:

m1 = immutables.Map({random.randint(0, 1000000): None for _ in range(1000000)}) # took 2.4s
m2 = m1.update({2000000: None, 2000001: None, 2000002: None}) # took 9.511799999017967e-05s

pyrsistent.s:

s1 = s(*[random.randint(0, 1000000) for _ in range(1000000)]) # took 1.51s
s3 = s2.update([2000000, 2000001, 2000002]) # took 0.24s
#

so it seems update is not O(1)

#

also add
if i run

x = [random.randint(0, 1000000) for _ in range(1000000)]
ps = pyrsistent.s(*x)
for i in range(10): ps2 = ps.add(1)

the last line takes about 4s

feral cedar
#

isn't the point that it's immutable? if you want efficient updates you probably shouldn't be using the immutable version

vast saffron
verbal escarp
#

they can also be frozen and obj attributes are pretty much the same as a dict, if your keys are proper names

vast saffron
#

frozendict is probably faster though

frosty topaz
dusk comet
sturdy timber
#

Yeah

#

I think they're planning to move to a new thing soon

feral island
#

Guess it still looks at BPO instead of GitHub

sturdy timber
#

remember seeing it mentioned in the migration issue

nova iris
#

hey guys, if a code object uses one byte for each instruction's argument, what happens when a code object is associated with more than 256 names/constants?

#

what happens when the argument byte needs to exceed 256?

unknown.png
feral island
sonic dagger
#

!pip chess

fallen slateBOT
#
chess v1.9.0

A chess library with move generation and validation, Polyglot opening book probing, PGN reading and writing, Gaviota tablebase probing, Syzygy tablebase probing, and XBoard/UCI engine communication.

nova iris
# feral island I think there's a special opcode for that, EXTENDED_ARG

EXTENDED_ARG(ext)
Prefixes any opcode which has an argument too big to fit into the default one byte. ext holds an additional byte which act as higher bits in the argument. For each opcode, at most three prefixal EXTENDED_ARG are allowed, forming an argument from two-byte to four-byte.

so now the real question... what happens when a code object is associated with more than 4294967296 names/constants? (\j, but if you do know the answer, i'd greatly appreciate you satisfying my curiosity lol)

#

also, tysm for answer the original question, this was extremely helpful!

rose schooner
#

the oparg should hold no more than 2147483647 because that's the maximum value

#

if you extend it may overflow

rose schooner
# rose schooner if you extend it may overflow

@nova iris it overflows ```py

import dis
f = lambda:None
f.code = f.code.replace(co_code=bytes([
... dis.opmap['EXTENDED_ARG'], 255,
... dis.opmap['EXTENDED_ARG'], 255,
... dis.opmap['EXTENDED_ARG'], 255,
... dis.opmap['UNPACK_EX'], 255,
... dis.opmap['RETURN_VALUE'], 0
... ]))
dis.dis(f)
1 0 EXTENDED_ARG 255
2 EXTENDED_ARG 65535
4 EXTENDED_ARG 16777215
6 UNPACK_EX -1
8 RETURN_VALUE

feral island
raven ridge
#

nasal demons

rose schooner
# feral island a black hole opens and swallows your code

is this supposed to happen ```py

import dis
f = lambda:None
f.code = f.code.replace(co_code=bytes([
... dis.opmap['EXTENDED_ARG'], 255,
... dis.opmap['EXTENDED_ARG'], 255,
... dis.opmap['EXTENDED_ARG'], 255,
... dis.opmap['EXTENDED_ARG'], 0,
... dis.opmap['EXTENDED_ARG'], 0,
... dis.opmap['EXTENDED_ARG'], 0,
... dis.opmap['EXTENDED_ARG'], 0,
... dis.opmap['EXTENDED_ARG'], 0,
... dis.opmap['EXTENDED_ARG'], 0,
... dis.opmap['EXTENDED_ARG'], 0,
... dis.opmap['UNPACK_EX'], 0,
... dis.opmap['RETURN_VALUE'], 0
... ]))
dis.dis(f)
1 0 EXTENDED_ARG 255
2 EXTENDED_ARG 65535
4 EXTENDED_ARG 16777215
6 EXTENDED_ARG -256
8 EXTENDED_ARG -65536
10 EXTENDED_ARG -16777216
12 EXTENDED_ARG -4294967296
14 EXTENDED_ARG -1099511627776
16 EXTENDED_ARG -281474976710656
18 EXTENDED_ARG -72057594037927936
20 UNPACK_EX -18446744073709551616
22 RETURN_VALUE

#

i'm pretty sure that's outside of the 32-bit signed limit

feral island
#

Not really sure. I know there was a recent bug report about negative extended args

raven ridge
#

one of the CPython core devs who owns the datetime module is looking for people to share feedback on a proposed change to datetime.datetime.isoformat() - picking a name for a new keyword argument that's being added to that function. If anyone has some time, it would be much appreciated if you could check out https://ganssle.io/s/isoformat-survey and help ensure we pick the best possible name for the new option

pliant tusk
lusty scroll
#

I have a question about the type system

#

it started out because I wanted to be able to obtain the signature fron all callable members of a BytesIO type

#

I found that this was not as easy as I thought it would be .. simply calling inspect.signature fails for about half the members.

#

the object in question is a _io.BufferedReader

#

for each member, I checked the descriptor defined in each MRO type and then called inspect.signature on that

#

this worked to get all signatures except write

#

I thought surely with argument clinic and everything, the signature of write must be part of some metadata somewhere

#

then I decided to check if all of these members had a docstring and they did, even for write

# 
>>> from _io import BufferedReader
>>> help(BufferedReader.write)
write(...)
    Write the given buffer to the IO stream.```
feral island
#

Funny, that method is not in io.BufferedReader.__dict__ and I don't see it in C

#

trying to figure out where it comes from

lusty scroll
#

well I searched its docstring and a couple places came up - one is a mysterious _pyio module

#

Lib/_pyio.py:

    def write(self, b):
        """Write the given buffer to the IO stream.

        Returns the number of bytes written, which may be less than the
        length of b in bytes.
        """
        self._unsupported("write")``` surely it can't be that
#

that type us not even in the mro

feral island
#

OK, found it. It's bufferediobase_write , which is METH_VARARGS

lusty scroll
#

but there's a weird registration thing going on with that _pyio one to

# 
io.RawIOBase.register(RawIOBase)
from _io import FileIO
RawIOBase.register(FileIO)```
feral island
#

It looks like BufferedReader doesn't inherit from _BufferedIOBase if you look at bufferedio.c, but elsewhere it does Modules/_io/_iomodule.c: PyBufferedReader_Type.tp_base = &PyBufferedIOBase_Type;

lusty scroll
feral island
#

The pyio stuff only gets used if the C implementation is unavailable, which should be basically never

lusty scroll
#

oh that's helpful to know

feral island
#

Actually in this case looks like it's never used

lusty scroll
#

I'm also curious where help() gets the write(...) from

lusty scroll
feral island
#

io.py unconditionally imports from _io which is the C version

lusty scroll
#

ahh ok

# 
PyDoc_STRVAR(bufferediobase_write_doc,
    "Write the given buffer to the IO stream.\n"
    "\n"
    "Returns the number of bytes written, which is always the length of b\n"
    "in bytes.\n"
    "\n"
    "Raises BlockingIOError if the buffer is full and the\n"
    "underlying raw stream cannot accept more data at the moment.\n");

static PyObject *
bufferediobase_write(PyObject *self, PyObject *args)
{
    return bufferediobase_unsupported("write");
}```
feral island
lusty scroll
feral island
lusty scroll
#

maybe I didn't miss anything then

#

is there any way to (from python) figure out which C function is associated with one of these members

#

member/descriptor/...

#

if I can find its address, I could theoretically figure out what symbol is there maybe

feral island
lusty scroll
#

so that bufferediobase_write should be in the PyType members for something with tp_name of _BufferedIOBase ?

feral island
#

yes

lusty scroll
#

I see in Modules/_io/bufferedio.c a static PyMethodDef bufferediobase_methods[7]; around line 2312

#

at least there's no magic

#

yeah so looking in gdb p bufferediobase_methods gives me

[5] = {
    ml_name = 0x7ff7ddcd88 "write",
    ml_meth = 0x7ff7d072d8 <bufferediobase_write at /data/media/0/src/python3/src/Modules/_io/bufferedio.c:181>,
    ml_flags = 1,
    ml_doc = 0x7ff7ddd6d0 <bufferediobase_write_doc> "Write the given buffer to the IO stream.\n\nReturns the number of bytes written, which is always the length of b\nin bytes.\n\nRaises BlockingIOError if the buffer is full and the\nunderlying raw stream cannot accept more data at the moment.\n"
  }```
#

theres also a static PyTypeObject *static_types[103]; in object.c and a separate one (static PyTypeObject *static_types[14];) in _iomodule.c lol

feral island
#

that's what static is for πŸ™‚

lusty scroll
#

hehe, indeed

feral island
#

I wonder what PySTEntry_Type is

rose schooner
lusty scroll
#

yes it's in the smaller one .. so I can easily get that type object from python

feral island
lusty scroll
# 
(gdb) p '/data/media/0/src/python3/src/Modules/_io/_iomodule.c'::static_types
$15 = {
  [0] = 0x7ff7eb3c10 <PyIOBase_Type>,
  [1] = 0x7ff7eb6478 <PyIncrementalNewlineDecoder_Type>,
  [2] = 0x7ff7eb4c20 <PyBufferedIOBase_Type>,
  [3] = 0x7ff7eb3e48 <PyRawIOBase_Type>,
  [4] = 0x7ff7eb61f0 <PyTextIOBase_Type>,
  [5] = 0x7ff7eb47c0 <PyBytesIO_Type>,
  [6] = 0x7ff7eb5130 <PyBufferedReader_Type>,
  [7] = 0x7ff7eb55a0 <PyBufferedWriter_Type>,
  [8] = 0x7ff7eb5908 <PyBufferedRWPair_Type>,
  [9] = 0x7ff7eb5e58 <PyBufferedRandom_Type>,
  [10] = 0x7ff7eb3fe0 <PyFileIO_Type>,
  [11] = 0x7ff7eb4968 <_PyBytesIOBuffer_Type>,
  [12] = 0x7ff7eb6ee8 <PyStringIO_Type>,
  [13] = 0x7ff7eb69f0 <PyTextIOWrapper_Type>
}```
lusty scroll
#

ok now I have a new question... is there any way to get the stub type (_pyio) from the real one?

#

I realized after some debugging that there is no signature nor any useful info in the bufferediobase one (except maybe the doc, which doesn't even mention the signature)

lusty scroll
jagged hamlet
#

Does Unity support Python ?
Or if it doesn't support Python, then which languages does it support?

lusty scroll
#

there is some possibility with

io.abc._get_dump(io.BufferedIOBase)```
jagged hamlet
#

Hey @lusty scroll , wanna help me with my question? I saw in your roles that you're a 'helper'.

rose schooner
lusty scroll
#

answered you there @jagged hamlet

jagged hamlet
feral island
compact saffron
#

Hi. I'm newbie. Please take care of me. Thanks !!!

verbal escarp
compact saffron
#

πŸ™‚

#

ok

west tinsel
#

When do differences in speed start to matter?

# 
[sam@samtop]: ~>$ python -m timeit "a, b, c =(1,2,3)"
20000000 loops, best of 5: 16.4 nsec per loop
[sam@samtop]: ~>$ python -m timeit "a, b, c =[1, 2, 3]"
5000000 loops, best of 5: 44.2 nsec per loop
[sam@samtop]: ~>$
elder blade
#

When it runs many times

west tinsel
#

I should have phrased my question better πŸ˜†

elder blade
#

πŸ˜… ah okay well what is your question then?

west tinsel
#

The two statements, do the same thing but the second one unnecessarily creates a list and is 3 times slower

#

Is that enough for it to be worth doing something about?

unkempt rock
#

Hello

feral cedar
elder blade
west tinsel
elder blade
#

With what Python 3.11 is doing, for sure! They would probably be happy knowing about this

feral island
#

This probably doesn't affect a lot of real code. When are you going to unpack a list/tuple containing only constants?

naive saddle
#

I'd imagine the extra buffer space allocated at the end to keep appends const time is the main cost IIRC?

#

this is just some random mypyc knowledge I have fwiw

feral island
#

The tuple version is faster because the whole thing gets compiled into a single co_consts entry in the bytecode, while the list has to be created at runtime

naive saddle
#

ah

feral island
# 
              0 RESUME                            0

  1           2 LOAD_CONST                        0 ((1, 2, 3))
              4 UNPACK_SEQUENCE                   3
              8 STORE_NAME                        0 (a)
             10 STORE_NAME                        1 (b)
             12 STORE_NAME                        2 (c)
             14 LOAD_CONST                        1 (None)
             16 RETURN_VALUE

In [24]: dis.dis("a, b, c = [1, 2, 3]")
              0 RESUME                            0

  1           2 BUILD_LIST                        0
              4 LOAD_CONST                        0 ((1, 2, 3))
              6 LIST_EXTEND                       1
              8 UNPACK_SEQUENCE                   3
             12 STORE_NAME                        0 (a)
             14 STORE_NAME                        1 (b)
             16 STORE_NAME                        2 (c)
             18 LOAD_CONST                        1 (None)
             20 RETURN_VALUE
#

I suppose in this case it could optimize away the list, but I feel like this is very rarely going to apply in real code

naive saddle
#

oh yeah apparently l = [*(...)] was a performance win, wasn't it lol

prime estuary
#

This is optimised with a ROT_THREE, ROT_TWO if they're not constant, but I guess it was decided that UNPACK_SEQUENCE is faster than 3 LOAD_CONST? The list should be optimised into a tuple regardless though like other similar usages...

rose schooner
# feral cedar no

i pretty much hate this answer

  • coming from a person who likes to make code fast
feral cedar
#

huh?

#

optimizing this will give you what, 5 ns faster?

rose schooner
feral cedar
#

ok

rose schooner
feral cedar
#

???

#

people focus on optimizations that will actually improve the speed of your code. this will do practically nothing

rose schooner
feral cedar
#

ok

rose schooner
#

i'm pretty sure that doesn't do "practically nothing"

raven ridge
#

The value of any change isn't just how much it speeds things up by, but also how much it costs to implement and maintain.

#

if it's a change that makes end user's code simpler and faster, removes code from CPython, and is cheap to maintain, that's a no-brainer. If it makes end users' code simpler and faster but it's hard to implement and makes the maintainers of CPython have to do a lot more work, then that may be a different story.

#

historically, a JIT for CPython has fallen into the latter category, which is why it has never been built - it's always been thought that the cost to build and maintain it would cost too much time and effort from core devs, crippling the development of the language.

rose schooner
raven ridge
#

yep.

#

wait, 3.12? I thought 3.13

rose schooner
rose schooner
raven ridge
#

that's what a JIT is

rose schooner
#

idk faster-cpython's plan for 3.12 had the word "JIT" in it

raven ridge
# rose schooner so why the change in mentality

There's a more concrete plan about how to minimize its cost and how to maintain it over time, I think. More core developers have stepped up to volunteer for it, and the Python ecosystem has grown to a point where more performance may be a more critical feature than more syntax or built-in libraries. I think it mostly just reflects a shift in priorities

rose schooner
raven ridge
#

hm, TIL

raven ridge
sturdy timber
#

How do you write a JIT for a language as dynamic as python. I'm not sure I'm really expecting an answer to that haha, it just sounds like a damn difficult task to get good results

#

Not that I know anything about JITs

white nexus
#

uhh pypy might have a thing about it

rose schooner
sturdy timber
#

I guess lol

boreal umbra
#

Well, how do you write PyPy?

white nexus
#

i'd imagine with a keyboard and IDE

#

i think it might described here

rose schooner
#

!pep 654

fallen slateBOT
#
**PEP 654 - Exception Groups and except***

Link

Status

Accepted

Created

22-Feb-2021

Type

Standards Track

rose schooner
#

anyone wanna help me understand this PEP

raven ridge
#

what don't you understand?

rose schooner
#

how exception groups work and what's their use compared to normal exceptions

raven ridge
#

they're a type of exception that holds other exceptions. You can throw a group of exceptions together, which can then be caught and analyzed to see all of the individual exceptions inside the group.

boreal umbra
#

It was accepted a while ago. Will it be in 3.11?

raven ridge
#

yep

rose schooner
#

3.11.0a7 has it

boreal umbra
#

I thought they were going to slow down with actual language changes, but this is pretty substantial. what are the chances that 3.12 will strictly be optimizations and the removal of antiquated stdlib content?

raven ridge
#

low to 0

#

there are a lot of core devs, and they have different interests and want to work on different things.

#

saying that no one can contribute features for a year would just alienate a lot of people.

boreal umbra
#

so, what else is slated for 3.11?

raven ridge
boreal umbra
# 
# re
Atomic grouping ((?>...)) and possessive quantifiers (*+, ++, ?+, {m,n}+) are now supported in regular expressions. (Contributed by Jeffrey C. Jacobs and Serhiy Storchaka in bpo-433030.)

How can I use this to make my code more esoteric?

raven ridge
#

how can't you

boreal umbra
#

idk. I just really like regex.

raven ridge
#

I mean, I suspect any use of those features will make your code more esoteric.

prime estuary
#

Exception groups aren't too big a change. The new syntax only takes effect with code that's actually raising them, otherwise you can still treat them like any other exception in generic code.

rose schooner
grave jolt
#

How can I use this to make my code more esoteric?

  1. use regex
#

πŸ™‚

verbal escarp
#

how can i write a decorator that acts as a transparent replacement of the class it decorates, like functool.wraps
so that

@decorator
class Foo:
  ...

has the same type etc. as the decorated class?

peak spoke
#

modify the existing type like dataclasses do?

verbal escarp
#

how does that work?

peak spoke
#

add methods or whatever to the class you got as the argument and then return it

verbal escarp
#

couldn't i turn it into a proxy?

#

but the proxy would also have to have the same type.. 😐

peak spoke
#

not sure if you can do anything for type() without keeping the obj

elder blade
peak spoke
#

but yeah also that, if it's decorated then only the decorator knows about it

elder blade
#

Unless, of course, it is called as other = decorator(original)

verbal escarp
#

how does type(thing) actually work? could i redirect the type lookup?

#

i'm keeping track of the original thing

#

so if i could redirect type() at the original thing, that could be a solution

elder blade
fallen slateBOT
#

@elder blade :white_check_mark: Your eval job has completed with return code 0.

<class '__main__.B'>
elder blade
#

It might be implemented inside of the interpreter to the point where you can't fake it. I think you can fake isinstance() though..

quick snow
#

There's no way of redirecting type() other than overwriting it

elder blade
#

Yeah it probably does a lookup on the PyObject* struct?

quick snow
west tinsel
#

I'm on my phone so I'll see how this goes πŸ˜‚

elder blade
#

Oh wait lmao, right.

west tinsel
#

!e

class A:
    pass

class B(A):
    def __init__(self):
        self.__class__ = A

print(type(B()))β€Šβ€Š
fallen slateBOT
#

@west tinsel :white_check_mark: Your eval job has completed with return code 0.

<class '__main__.A'>
west tinsel
#

Honestly, I need to be reminded not to type code on my phone

verbal escarp
# 
>>> foo.__mro__ = bar.__mro__
Traceback (most recent call last):
  File "<pyshell>", line 1, in <module>
AttributeError: readonly attribute

sigh

#

okay, so i need to replace the self.__class__ of the instance? oh my.

#

maybe i could return a class-factory instead

#

with __new__ returning the instance of the other class?

#

insanity 🀦

verbal escarp
#

i think i figured it out

# 
def decorator(thing):
    class wrapper(thing.__class__):
        def __new__(cls, *args, **kwargs):
            print(3434)
            return thing(*args, **kwargs)
    return wrapper
pliant tusk
verbal escarp
#

ah, well. it seems to work okay-ish this way

rose schooner
#

why does POP_JUMP_FORWARD_IF_FALSE and some instances of SWAP become CACHE when i try to create them using builtin code

feral island
#

a new optimization technique that mutates the bytecode in place to use fast paths

rose schooner
#

is there a way to like disable that?

feral island
rose schooner
#

i'm only testing out the most recent releases here because POP_JUMP_FORWARD_IF_FALSE was just added a few commits ago

feral island
#

well that's why we are in the alpha phase πŸ™‚

rose schooner
#

this must be the most thought-out thing i ever did

#

is it correct

nova iris
nova iris
#

also what happened to co_lnotab? the line to byte offset map or whatever

rose schooner
nova iris
#

so you don't need to put anything there?

rose schooner
#

i have to for proper line stuff but this is a demonstration and that's not important

modern thunder
#

HI GUYS

nova iris
rose schooner
lusty scroll
# 
Python 3.11.0a6+```
feral island
lusty scroll
#

probably.. it's been about a month since I updates / recompiled

rose schooner
#

or python 3.11.0a7+

#

it was just a few commits ago

rose schooner
#

like at least try not optimizing co_code when being replaced/made with the code object

lusty scroll
# rose schooner anyways to say again could someone see if this is correct https://paste.pythondi...

ok now I get:

          0 RESUME                            0
          2 LOAD_FAST                         0 (iterator)
          4 FOR_ITER                         26 (to 58)
          6 COPY                              1
          8 LOAD_CONST                        1 (63)
         10 BINARY_OP                         1 (&)
         12 CACHE
         14 LOAD_CONST                        2 (64)
         16 BINARY_OP                         1 (&)
         18 CACHE
         20 LOAD_CONST                        3 (6)
         22 BINARY_OP                        16 (<<=)
         24 CACHE
         26 FOR_ITER                          9 (to 46)
         28 COPY                              1
         30 LOAD_CONST                        1 (63)
         32 BINARY_OP                         1 (&)
         34 CACHE
         36 SWAP                              4
         38 SWAP                              2
         40 BINARY_OP                        20 (|=)
         42 CACHE
         44 SWAP                              3
    >>   46 LOAD_CONST                        2 (64)
         48 BINARY_OP                         1 (&)
         50 CACHE
         52 SWAP                              2
         54 POP_TOP
         56 RETURN_VALUE
    >>   58 LOAD_CONST                        0 (None)
         60 RETURN_VALUE

Fatal Python error: Segmentation fault

Current thread 0x0000007ff77f2dd0 (most recent call first):
  File "<stdin>", line ??? in c
  File "/data/media/0/src/py11code.py", line 68 in <module>
zsh: segmentation fault (core dumped)  PYTHONFAULTHANDLER=1 PYTHONDEVMODE=1
#

line 68 is c(iter([7,7,1,1,4])) πŸ˜„

#

where is co_code ? I'm not seeing it in the struct PyCodeObject ?

#

I see co_code_adaptive but it's only 1 byte?

rose schooner
rose schooner
# 
>>> from ctypes import *
>>> (c_char * 62).from_address(id(c.__code__) + type(c.__code__).__basicsize__)[18]
b'r'
``` seems to be fine when viewed raw
rose schooner
#

it should actually be 62 bytes long

# 
PyObject *
_PyCode_GetCode(PyCodeObject *co)
{
    PyObject *code = PyBytes_FromStringAndSize(NULL, _PyCode_NBYTES(co));
    if (code == NULL) {
        return NULL;
    }
    _Py_CODEUNIT *instructions = (_Py_CODEUNIT *)PyBytes_AS_STRING(code);
    for (int i = 0; i < Py_SIZE(co); i++) {
        _Py_CODEUNIT instruction = _PyCode_CODE(co)[i];
        int opcode = _PyOpcode_Deopt[_Py_OPCODE(instruction)];
        int caches = _PyOpcode_Caches[opcode];
        instructions[i] = _Py_MAKECODEUNIT(opcode, _Py_OPARG(instruction));
        while (caches--) {
            instructions[++i] = _Py_MAKECODEUNIT(CACHE, 0);
        }
    }
    return code;
}
#

that replacing thingy affects the interpreter ```py

c(iter([7,7,1,1,4]))

comments are added post-paste

RESUME visited
PUSH_NULL visited
LOAD_NAME visited # c
PUSH_NULL visited
LOAD_NAME visited # iter
BUILD_LIST visited # []
LOAD_CONST visited # (7, 7, 1, 1, 4)
LIST_EXTEND visited # [7, 7, 1, 1, 4]
PRECALL visited
CALL visited # iter([7, 7, 1, 1, 4])
PRECALL visited
CALL visited # c(iter([7, 7, 1, 1, 4]))

inside the c function

RESUME visited # 00 () []
LOAD_FAST visited # 02 (iterator) [iterator]
FOR_ITER visited # 04 (to 58 (LOAD_CONST 0) or X = next(iterator) = 7) [] or [iterator, X]
COPY visited # 06 (copy X) [iterator, X, X]
LOAD_CONST visited # 08 (63) [iterator, X, X, 63]
BINARY_OP visited # 10 ((&) => (Y = X & 63 = 7)) [iterator, X, Y]

missing vital instruction at position 12 (SWAP)

LOAD_CONST visited # 14 (64) [iterator, X, Y, 64]
BINARY_OP visited # 16 ((&) => Y & 64) [iterator, X, Y & 64] #(missing instruction at pos 12 causes wrong stack result but still correct values)

missing vital instruction at position 18 (POP_JUMP_FORWARD_IF_FALSE)

LOAD_CONST visited # 20 (6) [iterator, X, Y & 64, 6] #(missing instruction at pos 18 causes wrong stack result and doesn't branch which is fatal)
BINARY_OP visited # 22 ((<<=) => (Y & 64) <<= 6) [iterator, X, (Y & 64) << 6] #(all goes downhill from here, starting with the wrong value used)

missing vital instruction at position 24 (SWAP)

FOR_ITER visited # 26 (to 50 (RETURN_VALUE) or X = next((Y & 64) << 6) = !!) !! #(missing instruction at pos 24 puts the final nail in the coffin as FOR_ITER tries to tp_iternext an integer)
Windows fatal exception: access violation

Current thread 0x00000670 (most recent call first):
File "<stdin>", line ??? in c
File "<stdin>", line 1 in <module>

#

@lusty scroll breakdown

lusty scroll
#

just to clarify , it crashes for you too ?

feral island
elder blade
#

Could be worth it to open an issue, and make it clear that you're constructing the bytecode by yourself. If the fix is easy, then it may secretly fix some other bug that could be appearing but not yet noticed.

deft horizon
#

Any core devs around who would like to review a warnings PR? I've had a patch up to address user feedback (via pytest) for a while, and I'd really like to get it in before the PyCon/beta freeze rush πŸ˜…
https://github.com/python/cpython/pull/91435

feral island
rose schooner
rose schooner
# elder blade Could be worth it to open an issue, and make it clear that you're constructing t...

the function i linked (https://github.com/python/cpython/blob/main/Objects/codeobject.c#L1157-L1175) seems to do nothing in particular but replace a certain number of the opcodes in front of a position with CACHEs https://github.com/python/cpython/blob/main/Include/opcode.h#L219-L231

const uint8_t _PyOpcode_Caches[256] = {
    [BINARY_SUBSCR] = 4,
    [STORE_SUBSCR] = 1,
    [UNPACK_SEQUENCE] = 1,
    [STORE_ATTR] = 4,
    [LOAD_ATTR] = 4,
    [COMPARE_OP] = 2,
    [LOAD_GLOBAL] = 5,
    [BINARY_OP] = 1,
    [LOAD_METHOD] = 10,
    [PRECALL] = 1,
    [CALL] = 4,
};
# 
// Insert adaptive instructions and superinstructions. This cannot fail.
void
_PyCode_Quicken(PyCodeObject *code)
{
    _Py_QuickenedCount++;
    int previous_opcode = -1;
    _Py_CODEUNIT *instructions = _PyCode_CODE(code);
    for (int i = 0; i < Py_SIZE(code); i++) {
        int opcode = _Py_OPCODE(instructions[i]);
        uint8_t adaptive_opcode = _PyOpcode_Adaptive[opcode];
        if (adaptive_opcode) {
            _Py_SET_OPCODE(instructions[i], adaptive_opcode);
            // Make sure the adaptive counter is zero:
            assert(instructions[i + 1] == 0);
            previous_opcode = -1;
            i += _PyOpcode_Caches[opcode];
        }
#

i had gone through all of this trouble just to optimize dis.parse_varint from py def parse_varint(iterator): b = next(iterator) val = b & 63 while b&64: val <<= 6 b = next(iterator) val |= b&63 return val to py def parse_varint(iterator): val = (b := next(iterator)) & 63 while b & 64: val = (val << 6) | (b := next(iterator))&63 return val to manual bytecode that uses FOR_ITER instead of next() and replaces most of the variable names with stack-only stuff

feral island
#

sounds like you'd be better off using Cython or mypyc

elder blade
#

Yeah lol

pliant tusk
#

!e ```py
print >> "a"

fallen slateBOT
#

@pliant tusk :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | TypeError: unsupported operand type(s) for >>: 'builtin_function_or_method' and 'str'. Did you mean "print(<message>, file=<output_stream>)"?
pliant tusk
#

or is that because of how print worked in python2?

peak spoke
#

Yes

median palm
#

Is faster-cpython an official thing?

spark magnet
#

it's a testing ground for stuff that will go into CPython, iiuc

median palm
#

I see

#

So the changes it details will at some point be a part of cpython?

elder blade
#

Yeah

#

Unless it is only proposals

median palm
#

Alright, thanks :D

raven ridge
feral island
# median palm Is faster-cpython an official thing?

It's a group of developers (sponsored mostly by Microsoft). They use the faster-cpython GH project for ideas and coordination, but their changes are directly on CPython. Everything they've done will be in 3.11.

spark magnet
#

well, there are ideas they are working on that will have to wait for 3.12, 3.13, etc.

#

but faster-cpython isn't meant to ever be its own shipped project

median palm
#

I see, that's pretty neat. A JIT would be fun to see

paper echo
#

much better strategy than forking cpython only to abandon it and then kind of restart the fork but not with any specific stated goal or roadmap

#

(cf. dropbox)

rose schooner
#

cool new addition to dis ```py

from dis import Bytecode
def quicken(f, times=8):
... for _ in range(times):
... f()
...
co = compile("'a'+(b:='g')","","eval")
quicken(lambda:exec(co,{},{}))
print(Bytecode(co,show_caches=True).dis()) # before
0 RESUME 0

1 2 LOAD_CONST 0 ('a')
4 LOAD_CONST 1 ('g')
6 COPY 1
8 STORE_NAME 0 (b)
10 BINARY_OP 0 (+)
12 CACHE
14 RETURN_VALUE

print(Bytecode(co, show_caches=True, adaptive=True).dis()) # after
0 RESUME_QUICK 0

1 2 LOAD_CONST 0 ('a')
4 LOAD_CONST 1 ('g')
6 COPY 1
8 STORE_NAME 0 (b)
10 BINARY_OP_ADD_UNICODE 0 (+)
12 CACHE
14 RETURN_VALUE

rose schooner
vast saffron
#

sorry for kinda getting lost in all the dis statements, but what are you trying to achieve?

paper echo
#

what are the CACHE ops?

feral island
#

the general idea of the 3.11 specialization is that for code like a + b, the interpreter tracks what the types of a and b are on each execution of the same code. If it gets executed a bunch of times and a and b are both always ints, it mutates the code in place to a new op that is specialized for adding ints. There is a runtime check that makes it fall back to the general, slow path if they happen to not be ints the next time.

#

(haven't verified they did this specifically for +, but that's the general idea)

elder blade
#

Huh, how come it gets faster? thinkmon

#

I mean, you take the runtime hit of recording the types as well as the safety check

paper echo
#

ohh interesting

#

so adaptive=True shows the actual specialized versions?

#

or is that a different optimization?

feral island
feral island
prime estuary
#

The cache data is used to store stuff like the index inside a globals dict.

rose schooner
#

code has to be quickened though to actually have changes in co_code_adaptive

#

that's achieved by the quicken function

quick night
fallen slateBOT
#

Python/bltinmodule.c lines 1916 to 1922

static PyObject *
builtin_pow_impl(PyObject *module, PyObject *base, PyObject *exp,
                 PyObject *mod)
/*[clinic end generated code: output=3ca1538221bbf15f input=435dbd48a12efb23]*/
{
    return PyNumber_Power(base, exp, mod);
}```
rose schooner
quick night
#

Thanks

gentle ocean
#

hello guys, I started with python and I'm having this problem with pyinstaller, is there any way I can solve this?

Screenshot_2022-04-20_113338.png
zenith briar
lusty scroll
#

"timeout is too large for a C timeval" on Windows .. what could it be

#

its coming from some place in stdlib related to sockets

feral island
# 
Modules/_multiprocessing/semaphore.c:                            "timeout is too large");
Modules/selectmodule.c:            PyErr_SetString(PyExc_OverflowError, "timeout is too large");
Modules/selectmodule.c:            PyErr_SetString(PyExc_OverflowError, "timeout is too large");
Modules/selectmodule.c:            PyErr_SetString(PyExc_OverflowError, "timeout is too large");
#

looks like that exact error isn't from CPython. It may just be giving you the error from the underlying Windows API

lusty scroll
#

oh ok

#

hmm thanks

#

"timeout doesn't fit"

#

was the exact message

feral island
#

ah ```% git grep "timeout doesn't fit"
Modules/socketmodule.c: "timeout doesn't fit into C timeval");

lusty scroll
#

I'll do some digging around there, perhaps it will lead me to a clue

feral island
#

looks like this would happen if your timeout is bigger than 2**32? Not entirely sure

lusty scroll
#

perhaps it's meant to be -1 ?

rose schooner
pale furnace
#

Hi folks! Does anyone know how to determine when a module was added to the Python standard library? I'm trying to develop a timeline of several modules, but I'm not sure how to date things.

rose schooner
pale furnace
feral island
#

where "recently" is "during the lifetime of Python 3"

pale furnace
#

I'm specifically wondering about the select module. I'm researching the topic of asynchronous I/O support in various programming languages. It seems that select has been around for quite a while!

#

So, 1.0 it seems.

#

I'm shocked. Source versioning has been maintained from the outset? Wasn't Python just a bag of code being archived and shared? Was CVS used at the time?

#

Well done, Guido!

rose schooner
#

so that makes it easier i guess

white nexus
lusty scroll
#

I posted a PR to pyreadline assuming it was just that way because I was using 3.11. could it be it's been broken since 3.9

native flame
#

you're supposed to import it from collections.abc instead yeah

lusty scroll
#

ahh. wow ..

lusty scroll
native flame
lusty scroll
#

like instead people are using IPython or vscode or such things

#

just guessing as to why nobody would have noticed all this time

dusk comet
#

how to implement callable(x) in python?

def callable(x: object) -> TypeGuard[Callable[..., Any]]:
    return any('__call__' in cls.__dict__ for cls in type(x).__mro__)

is this correct?

#

hasattr(x, '__call__') and hasattr(type(x), '__call__') isnt correct

pliant tusk
#

afaik you could techically make a callable type in C that sets tp_call without setting __call__

dusk comet
#

its weird πŸ€”

feral island
pliant tusk
#

ah that makes sense. although if thats something you'd call really crazy you should see some of the stuff i put in #esoteric-python

feral island
#

πŸ™ˆ

quick snow
#

:P

pliant tusk
dusk comet
quick snow
#

Could inspect the traceback to see if the error comes from within the function of from callable

grave jolt
#

!e

print(callable(int))
print(callable(42))
fallen slateBOT
#

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

001 | True
002 | False
verbal escarp
#

i'm doing ```py
def is_callable(thing):
try:
object.getattribute(thing, "call")
return True
except AttributeError:
return False

#

which works quite well

#

haven't come across any callable that didn't work with that so far

quick snow
# 
class NotCallable:
    __call__ = 23
verbal escarp
# 
>>> is_callable(NotCallable)
True
quick snow
#

Yes, but it isn't

feral island
quick snow
#

!e

class NotCallable:
    __call__ = 23
NotCallable()()
fallen slateBOT
#

@quick snow :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 3, in <module>
003 | TypeError: 'int' object is not callable
verbal escarp
#

but it is πŸ˜‰

quick snow
#

The class is, the instance isn't

verbal escarp
#

that's the whole point

#

the thing that is passed in, is callable

quick snow
#

!e

def is_callable(thing):
    try:
        object.__getattribute__(thing, "__call__")
        return True
    except AttributeError:
        return False

class NotCallable:
    __call__ = 23

not_callable = NotCallable()
print("callable?", is_callable(not_callable))
not_callable()
fallen slateBOT
#

@quick snow :x: Your eval job has completed with return code 1.

001 | callable? True
002 | Traceback (most recent call last):
003 |   File "<string>", line 13, in <module>
004 | TypeError: 'int' object is not callable
verbal escarp
#

😐

#

you broke python, well done :p

quick snow
#

to be fair, builtins.callable() also believes it is callable

verbal escarp
#

okay, this is interesting

# 
def is_callable(thing):
    try:
        return callable(object.__getattribute__(thing, "__call__"))
    except AttributeError:
        return False
#

it actually can smell your int stink

#

but it's wrong-ish

# 
>>> is_callable(NotCallable())
False
>>> is_callable(NotCallable)
False
>>> NotCallable()
<__main__.NotCallable object at 0x0000023E33FFF250>
>>> class Foo:
             pass
             
>>> is_callable(Foo)
True
quick snow
# 
def is_callable(thing):
    try:
        return callable(object.__getattribute__(type(thing), "__call__"))
    except AttributeError:
        return False
feral island
#

hm why the __getattribute__? why are you bypassing descriptors?

verbal escarp
#

descriptors.. are weird

flat gazelle
#

as the docs for callable say, not callable(f) implies that f() will fail, but callable(f) doesn't mean f() will work

#

and you can't really do better than that

verbal escarp
#

and properties are tricky to decorate

feral island
#

Not really sure what your function adds over builtins.callable

quick snow
#

Yeah, if I just do getattr(type(thing), "__call__") it still seems to work the same, except it catches descriptors bound to the name __call__ :D

quick snow
verbal escarp
quick snow
#

It only does one indirection, though. If __call__ is an object that itself has a __call__ attribute (on the class) that's an int or something, it doesn't notice it

verbal escarp
#

i've stumbled over a number of cases where callable didn't work properly

flat gazelle
#

it always works properly, it's just that it's impossible to check whether calling an object will succeed without actually calling it

radiant garden
#

while x is not x:x=getattr(x,"__call__")

feral island
#

Yes. Curious to hear of non-contrived cases where callable() doesn't do the job

radiant garden
#

You can also check the tp_call slot pointer a la C
(I think that's what callable does?)

feral island
verbal escarp
#

and callable() doesn't work on properties

feral island
#

what do you mean by "work"?

verbal escarp
#

it can't tell if something is a property or not

radiant garden
#

how does that relate to whether it's callable

verbal escarp
#

as i said, i built the check to find callables for mass-decoration. the very first application was indeed to decorate properties

#

and properties are call-by-implication so to say

#

which reminds me, i need to put property-decoration back in there

radiant garden
#

properties and other descriptors are invoked with __get__ so I don't see how it would be feasible to detect whether the method calls something internally

feral island
#

there are some inspect helpers for finding descriptors

verbal escarp
#

you can check for the descr.. what Jelle says

#

they have a special type

radiant garden
#

and where does callable come in play here, just as a bonus?

verbal escarp
#

you mean how it fails?

#

or what i'm doing with it?

#

confused :p

#

fascinating. callable is qualitatively much less strict than the modified is_callable check. using my check, only 1455 things are decoratored in numpy, with callable as check it's 2777 things

#

i'll have to look into that some other time

rose schooner
dusk comet
lusty scroll
#

come on, stop messing around with callable

#

it's pretty limited

#

just accept you'll have to special case every type of callable you're interested in

#

!e

def really_callable(thing):
  try:
    thing.__get__
    thing.__call__
    return True
  except Exception:
    return False
print(really_callable(...))
print(really_callable(lambda i: None))
print(really_callable(really_callable))```
fallen slateBOT
#

@lusty scroll :white_check_mark: Your eval job has completed with return code 0.

001 | False
002 | True
003 | True
lusty scroll
#

@verbal escarp

ruby pilot
white nexus
fossil blade
#

Wait wut str.istitle vs float.is_integer

verbal escarp
# 
>>> numpy.zeros((2,2))
array([[0., 0.],
       [0., 0.]])
#

false negative

white nexus
#

hm

white nexus
#

!e import numpy ; print(type(numpy.zeros))

fallen slateBOT
#

@white nexus :white_check_mark: Your eval job has completed with return code 0.

<class 'builtin_function_or_method'>
white nexus
#

shouldn't this be a different type

#

as its not a built in

verbal escarp
#

c-extension

white nexus
#

welp

#

it discovered a bug with inspect.getsourcefile

#

or no wait

#

nope that's with my code

rose schooner
#

iter() and probably aiter() relies on callable

white nexus
verbal escarp
white nexus
#

long story short is i wasn't properly catching c-extensions

verbal escarp
#

oh

quick night
#

!e

args = map(int, (10e10, 10e20, 10e30))


print(pow(*args))
print(pow(*args))
fallen slateBOT
#

@quick night :x: Your eval job has completed with return code 1.

001 | 7063394790938602034297680429056
002 | Traceback (most recent call last):
003 |   File "<string>", line 5, in <module>
004 | TypeError: pow() missing required argument 'base' (pos 1)
quick night
#

why is this happening

#

this doesn't happen without the map, what is map doing here?

#

!e

args = (int(10e10), int(10e20), int(10e30))


print(pow(*args))
print(pow(*args))
fallen slateBOT
#

@quick night :white_check_mark: Your eval job has completed with return code 0.

001 | 7063394790938602034297680429056
002 | 7063394790938602034297680429056
quick night
#

!e

args = map(int, (10e10, 10e20, 10e30))

for arg in args:
    print(arg)
    
for arg in args:
    print(arg)
fallen slateBOT
#

@quick night :white_check_mark: Your eval job has completed with return code 0.

001 | 100000000000
002 | 1000000000000000000000
003 | 9999999999999999635896294965248
quick night
#

huh, map can be iterated over only once?

native flame
#

yes

quick night
#

why

native flame
#

generally speaking an iterator will work just once

#

when you loop over a list twice it creates two separate iterators for each

#

an iterator only needs to have a __next__ method, which should either provide the next value to be taken up by the loop variable, or raise StopIteration to signal that the loop should end

#

it doesn't have to support a mechanism for "resetting"

quick night
#

ah ok, i didnt know it creates two iterators in case of lists( and others alike)

#

thanks πŸ‘

dusk comet
dusk comet
wide shuttle
#

It would also mean that something like this is not guaranteed to work for all iterators:

for element in iterator:
    ...
native flame
#

yeah. i remember there was an github issue on this

#

oop you're in the thread too

lusty scroll
#

but any exception something would throw in there

#

I guarantee you don't want

#

fixed @white nexus πŸ˜„

#

now Ctrl+C and sys.exit() won' fail :p

west tinsel
#

I'm quite lost πŸ˜‚

#

At the surface, the CI tests pass. But it fails when compiled with optimisations?

feral island
#

huh it still fails when I revert your commit

west tinsel
west tinsel
#

It's weird because it seems to just be one test

feral island
#

weirdly this passes ./python.exe -m test test.test_decimal -m '*test_from_tuple*'

#

but this fails ./python.exe -m unittest test.test_decimal.CWhitebox.test_from_tuple

#

doing some printf debugging scan_dpoint_exp in the C code is behaving differently

#

no it's not, I printed the address instead of the value

#

the value of prec in the global context is different

feral island
west tinsel
#

Oh, weird

hazy linden
#

Is the yield keyword in Python the same as a yield system call on an OS?

median palm
#

Same as in yield invokes that system call? If so, no

hazy linden
#

Ah gotcha. I am taking a systems class in C and saw the similar keyword

median palm
#

The yield keyword in python sort of "pauses" a function in place, returning from it, and when you call that function again, it resumes execution at the point where you used yield

west tinsel
#

What's the command for serving documentation?

#

I normally use php -S or open the html files with my browser. I remember seeing a tool being discussed somewhere but forgot what it was

raven ridge
west tinsel
#

It doesn't do exactly what I wanted it to do but I can see myself using it in the future

#

I'm talking about the rst files in the Doc folder

feral island
#

I usually just do make html in the Doc folder and open the generated files in my browser

#

the pydoc thing had a security vulnerability recently

west tinsel
#

Oh, is there a link to information about that?

feral island
west tinsel
#

Thanks!

elder blade
#

Hmm, for the new forward class and continue class statements does anyone know any other languages that has something similar?

native flame
#

what are you referring to?

elder blade
swift imp
#

Seems convoluted if the purpose if purely for typing. I agree with the premise and it pisses me the hell off that I cannot do circular references but at the same time, I don't know if I would go through the trouble

native flame
#

dont like it

#

what issue does it solve that automatically stringifying annotations won't

swift imp
#

Like I've totally been in a situation (specifically when making pathlib.Path subclasses that had properties referring to parent/child object directly. I couldn't type annotate it, and it bothers the hell out of me, but this is just like so much work.

swift imp
native flame
#

they can use get_type_hints then to evaluate the annotations

swift imp
#

I think you would still hit the same issue, you could be trying to evaluate it and get a class that doesn't exist yet

#

Like I have no idea at what stage in class creation say pydantic needs the run time type of the type annotation

#

It could be in at the end of __init__ after its already created or even in __call__

#

No promise that the type being annotated will exist

#

I still don't understand what the problem with the pep titled deferred type annotations using descriptors or something like that. An alternative to the stringification of type hints pep from Lukas

west tinsel
#

So, there are merge conflicts

unknown.png
#

But even when resolving and removing the markers, I can't mark it as resolved

unknown.png
dusk comet
feral island
west tinsel
#

One thing to note is that versionchanged applies to the function above

#

And was introduced in a different commit

#

It tripped me up, I thought someone added it to set_start_method

feral island
#

I get the same thing 😦

#

maybe fix it on the command line instead (git fetch upstream; git merge upstream/main, depending on your remote name)

#

wait I did it

#

removed the "disabled" class from the button in the browser inspector πŸ˜„

west tinsel
#

πŸ˜‚

elder blade
raven ridge
#

I think I like this proto PEP. It seems pretty easy to implement, it seems reasonable easy to explain how it works, and it seems like it would thoroughly solve the problem

#

And it certainly helps that it's analogous to how declarations work in C and C++, so lots of developers would already have a frame of reference for understanding it

civic wasp
#

Whats a PEP

raven ridge
#

!pep 1

fallen slateBOT
#
**PEP 1 - PEP Purpose and Guidelines**

Link

Status

Active

Created

13-Jun-2000

Type

Process

raven ridge
#

PEPs are formal proposals for changes to the Python language. If you click the Link in the embed above, it explains PEPs in detail

radiant garden
#

pointers to incomplete types are fine because their size is known

#

(or, er, it might not just be the mutual recursion since declarations should be properly ordered and so the size of X can't be known at Y's declaration site)

deft horizon
feral island
deft horizon
#

I've had my own experience with ideas-I-thought-were-good meeting Guido and Raymond πŸ˜…

rich cradle
#

!unsilence

fallen slateBOT
#

βœ… unsilenced current channel.

median palm
#

πŸ‘€

west tinsel
#

Is there a method to get the default mode of created files?

#

My idea is creating a file and checking what mode it has

quick snow
#

You mean the umask?

# 
import os

umask = os.umask(0o777)
os.umask(umask)

For some reason you need to set the umask in order to get its current value. Note that the umask is what's forbidden, so it needs to be xored with 0o777 to get the default permissions, I think.

fallen slateBOT
#
Bad argument

Converting to "int" failed for parameter "pep_number".

#
Command Help

!pep <pep_number>
Can also use: get_pep, p

Fetches information about a PEP and sends it to the channel.

west tinsel