internals-and-peps

[flat gazelle]

I sometimes put them in places just to get better error reporting.

but ye, they are quite rare.

[pseudo cradle]

Oddly enough I see them all the time in C

[flat gazelle]

checking preconditions and postconditions is probably more common in C where failing one results in a segfault, whereas in python you get a sentry alert with a stack trace/a logged exception, which is less bad

[halcyon trail]

It's because in C/C++ you're writing high performance software, so there's many many many cases where you'd like to do a check but it's too expensive to be in the final binary

that's basically the whole point of asserts in those languages; something that by default is present in debug builds but not release builds

in languages that don't really do this debug/release distinction the same way, and languages where you are incurring a zillion branches anyway (e.g. bounds checking on list), it doesn't seem like there's much point

[sturdy timber]

I sometimes use them to check pre/postconditions whilst implementing an algorithm or something, partly just as a way to help me understand the problem more as I write it, like a comment

[visual shadow]

I wrote an assert once that came in extremely handy

It was essentially a sanity check, i had a complex algo and I knew that the result should meet some condition

Guess what, i had forgotten an edge case, but the assert caught it

I like asserts for those kind of niche situations in real code.

[halcyon trail]

I guess in python I'd just typically be more inclined to throw

[frigid trout]

I usually only use assert for type narrowing

[halcyon trail]

Right, that's a good example as well

but those asserts will never fire unless you ignore your mypy though, or you make a mistake

[halcyon trail]

the thing is that in python in most cases rather than make something a true pre-condition, you may as well just check it and throw if it doesn't hold

[quick snow]

I use asserts as power-comments (they can not lie)

(yeah, I know, -O; I don't use it)

[sturdy timber]

True, that would work, although asserts are slightly more concise, and slightly more explicitly say "this thing should not be possible"

[halcyon trail]

they are different though, asserts you're not really allowed/supposed to catch

so they are truly for pre-conditions

whereas users can catch exceptions, and that can be useful in many cases, and they can rely upon it

Suppose for example your function is called foo; I have some code that is taking arguments from somewhere and using them to call foo. my code doesn't relaly know anything about the arguments. If your code promises to check argument validity and throw, I can just do the call and catch, and try to do something sensible in that case. If your code says its a pre-condition and just asserts, then I should be doing the checks myself, which is more annoying and brittle

[peak spoke]

but why should the function check the validity of arguments

[sturdy timber]

If your function might run without error with that input but return a wrong, surprising, or undocumented result it would make sense.

[halcyon trail]

to help ensure that errors are caught in an earlier, more understandable way, to aid in debugging; also to allow for error recovery like in the example I gave.... lots of reasons? I'm kind of confused because functions checking their arguments for validity is such a common and basic thing

[peak spoke]

it can use asserts for clearer errors that don't trigger somewhere down the line in a random operation, but usually I don't think it should validate in any way. e.g. I used an assert for a getter that fetched an object that had to be set first, with the assert checking if it was actually set. Then if someone runs it with optimize that check won't execute every time as it is expected the caller already properly sets it

[halcyon trail]

I mean if you can design things in such a way that an error is not reasonably/easily possible, yes, that's definitely better than checking

often not possible though

and in a lot of cases in python the "checks" are implicit because performing illegal operations generally returns well defined exceptions

true preconditions though are quite rare in the python standard library, compared to e.g. C++

[peak spoke]

from my experience most functions raise something down the line instead of explicitly checking, which can be made a bit nicer with an assert that won't impact the production code

[halcyon trail]

the only "Impact" on production code though is performance, and most asserts would anyway be negligible performance costs. I don't even think most people bother disabling asserts in "production" code

[peak spoke]

the checks are usually negligible compared to other things, but they're also trivial to disable

[halcyon trail]

sure, I've just almost never heard of anyone bothering to disable them personally

[peak spoke]

assert is in a bit of a cycle because of that

people doesn't use it because the optimize flags are not used, and optimize flags are not used because the don't impact much without people using asserts

[halcyon trail]

it's already happened to me a few times that I asserted something because the function wasn't designed to handle such and such case, i.e. it was effectively a pre-condition.
Then people were dealing with data were they weren't easily able to verify that pre-condition, so they wanted to simply call my function, and if it failed just do some other thing.

and they didn't understand that catching assertions isn't great. So they just caught the assertions.

If I'd just thrown some kind of exception instead all would be well.

By throwing you gain more functionality for users basically, and there's not much benefit I can find to the assert (in python)

optimize flags wouldn't matter much even if people did use asserts, is my point

[flat gazelle]

yeah, they are quite rare

[halcyon trail]

only thing I really like about python asserts is saving a line 🙂

[bronze steeple]

I am getting the following error:

‘ValueError … was created by a different ContextVar’
The exception shows it happening when I call .reset(original_val) on the context var.
I am using the ThreadPoolExecutor and I think it has something to do with the exception but I don’t understand what the root cause is or how to fix it.

Would appreciate any guidance

[lusty scroll]

oh geez

I only ever have really used them with pytest

it's very common to have precondition "guards" in java code like

static int doSomething(String a, Object b) {
    Objects.assertNonNull(a)
    Objects.assertNotEmpty(a)
    Objects.assertInstance(a, List.class)
}```  (this code is not really meant to do something useful)

and kotlin has built that into the code the compiler generates to make non-null types work, I think

[halcyon trail]

I guess in Kotlin since it assumes all types are not null by default it probably doesn't need explicit guards

But maybe it does, I guess you could try subverting the Kotlin type system with some Java and try

[heady mauve]

assert is super handy when you need to get a quick-and-dirty solution up and going for things like preconditions

[halcyon trail]

well, yeah, I mean it saves one line of code

[cloud crypt]

usually you'd want to raise TypeError or ValueError depending on what conditions are not met

[spice pecan]

I don't really see a reason for assert to be a keyword over a built-in function

def assert(condition, message='', exc_type=AssertionError):
    if not condition:
        raise exc_type(message) 

Something like this would be pretty much equally as readable and convey the same intention without being a keyword

[flat gazelle]

well, assert doesn't evaluate its second argument

!e

assert True, 1/0

[fallen slate]

@flat gazelle :warning: Your eval job has completed with return code 0.

[No output]

[spice pecan]

Hmm, that's fair

[elder blade]

Aren't asserts also removed on the bytecode level?

[white nexus]

they are

!d -O

[fallen slate]

-O

-O```
Remove assert statements and any code conditional on the value of [`__debug__`](https://docs.python.org/3/library/constants.html#debug__ "__debug__"). Augment the filename for compiled ([bytecode](https://docs.python.org/3/glossary.html#term-bytecode)) files by adding `.opt-1` before the `.pyc` extension (see [**PEP 488**](https://www.python.org/dev/peps/pep-0488)). See also [`PYTHONOPTIMIZE`](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONOPTIMIZE).

Changed in version 3.5: Modify `.pyc` filenames according to [**PEP 488**](https://www.python.org/dev/peps/pep-0488).

[dusk comet]

no

>>> dis('assert a, b')
  1           0 LOAD_NAME                0 (a)
              2 POP_JUMP_IF_TRUE         6 (to 12)
              4 LOAD_ASSERTION_ERROR
              6 LOAD_NAME                1 (b)
              8 CALL_FUNCTION            1
             10 RAISE_VARARGS            1
        >>   12 LOAD_CONST               0 (None)
             14 RETURN_VALUE

[elder blade]

Do you have the optimization that removes it enabled?

[dusk comet]

with optimization it will be equal to pass

[dusk comet]

no, there are no optimizations it this example

[spice pecan]

-O is kind of a questionable option IIRC

doesn't it also remove docstrings and things alike?

[flat gazelle]

I think that's -OO

[spice pecan]

Yeah, so as far as I could gather, -O ignores asserts, doesn't generate bytecode for if __debug__ branches and -OO also removes docstrings

And that should be all it does

[dusk comet]

>py -m pprint
Traceback (most recent call last):
  File "D:\Programs\Python\310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "D:\Programs\Python\310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "D:\Programs\Python\310\lib\pprint.py", line 670, in <module>
    _perfcheck()
  File "D:\Programs\Python\310\lib\pprint.py", line 645, in _perfcheck
    p._safe_repr(object, {}, None, 0, True)
TypeError: PrettyPrinter._safe_repr() takes 5 positional arguments but 6 were given

[jovial flame]

Are you trying to run pprint as a module? I think it is normally imported then used as a print statement

[dusk comet]

pprint can be executed directly

https://github.com/python/cpython/blob/3.10/Lib/pprint.py#L669-L670

[fallen slate]

Lib/pprint.py lines 669 to 670

if __name__ == "__main__":
    _perfcheck()```

[jovial flame]

Huh, TIL

[peak spoke]

Is it documented anywhere?

[dusk comet]

no

[peak spoke]

not much of a reason to expect it to work then

[paper echo]

!e import pprint; pprint._perfcheck()

[fallen slate]

@paper echo :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 |   File "/usr/local/lib/python3.10/pprint.py", line 645, in _perfcheck
004 |     p._safe_repr(object, {}, None, 0, True)
005 | TypeError: PrettyPrinter._safe_repr() takes 5 positional arguments but 6 were given

[paper echo]

% python -m pprint
_safe_repr: 1.103822235
pformat: 2.26268847

works on 3.8.12

i wonder if something is weird about the py runner

[prime estuary]

Here's the problem, in this commit the sort_dict parameter at the end was removed, but the call there wasn't fixed:
https://github.com/python/cpython/commit/ff420f0e08a2443339da0df7ace95e14177bac53

[severe snow]

I can't find much discussion about this. big loop simple profiles like this are consistently a good 30-50% slower on 32-bit than 64-bit standard python builds for me (on 64-bit os)

def x():
    pass

if __name__ == '__main__':
    t0=time.time()
    for i in range(100000000):
        x()
    t1=time.time()
    print(f'time diff: {t1-t0}')

I'm curious, are compiler settings a factor here and why is python, far more than other things, so much slower? is it just due to the implementation of unbounded integers?
does stuff like faster-cpython care about 32-bit? of course, 32-bit is fading, but as python gets faster, it becomes a more viable extension/embedded language so its 32-bit base could even grow a lot

[swift imp]

There's a lot of overhead there from calling next

[halcyon trail]

i'm not sure why exactly but 32 bit builds can easily be faster than 64 bit builds, if they're using big arrays of pointers/integers that are 32 bit instead of 64 bit

because now your big array is half the size, and things fit better in cache, fewer cache misses, etc

I remember complaining in the Kotlin slack that their default integer, used everywhere, is a 32 bit integer, which is really annoying on occasion

and apparently it's like that because the standard JVM integer is 32 bit, and that's still the way it is because if you make the standard integer and internal pointer on the JVM, 64 bit instead of 32 bit, it slows things down quite a bit and consumes more memory

[sacred yew]

but they said 32 bit is slower than 64

[halcyon trail]

Ah ok lol

Hard to say for sure, there's rarely penalties for using a smaller type that I can think of. Maybe some extra work that the python has to do to "translate" the pointers?

Not sure really

[severe snow]

that's a good point in that it kind of invalidates my original intent for the test (estimate the cost of a python callback from my C++ code) -- but still, then, why would any of it be so much slower on 32?

I would have expected 32 to be faster in a contrived non-data-heavy case like this too

[dusk comet]

https://www.python.org/downloads/release/python-3102/
https://www.python.org/downloads/release/python-3110a4/
where is installer for windows?

[frigid trout]

It's probably not out yet, but you can always just build locally from the source

[dusk comet]

im too dumb to build it from source

[dusk comet]

yes, this works! thank you
last time i tried to build autogenerated VS projects, but I couldn't

[peak wasp]

guys no one is helping me in help servers

i have assignment due tommorow

can i post my query here, please?

[dusk comet]

post it in #python-discussion

[feral cedar]

now that dictionaries maintain insertion order, is there a reason to still raise a RuntimeError when adding/removing keys while iterating over one? my understanding is that because the iteration order was arbitrary, adding a key could cause the table to resize and potentially loop over a value twice

[peak spoke]

can't the entry array indices move around on a resize, even if the relative order is maintained?

Would need to reiterate up to that point and keep track of which item it was on if that is the case

[feral cedar]

ah I see what you mean. you wouldn't know where in the array you were. ok thanks

[tiny whale]

lmk if wrong channel. From https://pip.pypa.io/en/stable/topics/caching/#http-responses:

While this cache attempts to minimize network activity, it does not prevent network access altogether. If you want a local install solution that circumvents accessing PyPI, see Installing from local packages.

when monitoring IO on my system I see that pip is still doing a lot even if all wheels are cached (proven by "Collecting..." and "Using cached...")

and it takes a minute just for pip to collect all the cached wheels before installing, presumably because of this network IO

why can't 100% cache hits cause 0 network activity? is pip still checking something from PyPI even though the hashes and wheel files are already stored offline?

[naive saddle]

this is probably the wrong channel but I can partially explain, pip still has to reach out to the index (in this case being PyPI) to check there isn't a newer version available. Even if you pinned to the package version level pip still needs to check there isn't a release with a higher build tag (effectively a way in the wheel spec to "replace" wheels). Although yeah I'm not sure why pip would reach to the network if you have hashes pinned too, perhaps the cache-control header requests the client always revalidates the cache before using it?

[nocturne pasture]

hey

[unkempt rock]

Hello

[nocturne pasture]

help please

[lost sequoia]

Hi, not sure it is the correct channel for this but do you know a way to check where the sources in site-packages comes from ?

I thought they would be namespaced by the package's name but it seems not. Like a pip install something would install the lib in something_else directory.

Makes it difficult to know where things comes from and adds security issues (just a wild guess)

Any insight?

[white nexus]

!e ```py
import this

https://github.com/python/cpython/commit/63cd9bf4887cd4603ead4db29c772fa370e68a25

[fallen slate]

@white nexus :white_check_mark: Your eval job has completed with return code 0.

001 | The Zen of Python, by Tim Peters
002 | 
003 | Beautiful is better than ugly.
004 | Explicit is better than implicit.
005 | Simple is better than complex.
006 | Complex is better than complicated.
007 | Flat is better than nested.
008 | Sparse is better than dense.
009 | Readability counts.
010 | Special cases aren't special enough to break the rules.
011 | Although practicality beats purity.
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/wubaqelega.txt?noredirect

[lost sequoia]

Hum is that answering anything?

[boreal umbra]

It would appear that you can't use f strings in docstrings.

In [1]: foo = "lalalalala"

In [2]: def func():
   ...:     f"""hello {foo}"""
   ...:     return 1
   ...:

In [3]: func.__doc__

In [4]: print(func.__doc__)
None

In [5]: help(func)
Help on function func in module __main__:

func()

very disappointing.

[tiny whale]

is there a way to disable these network checks, since I know that hashes protect me from packages where a version is replaced in-place? or is this internal behavior not modifiable easy

[naive saddle]

I have no idea, you might want to ask in the pip text channel under the PyPA server: https://discord.gg/w2BV5UqV

[tiny whale]

thank you, I'll check this out

[flat gazelle]

docstrings are stored in the ast IIRC, so it does kind of make sense.

[peak spoke]

and well, they are there mostly for static tools and the readers so interpolating like that doesn't make much sense

can still overwrite the doc attr if it is necessary for something, though an explicit attribute or a wrapper should be more suited imo

[quick snow]

How would that even work

def foo(x):
    f"{x}"

print(foo.__doc__)

[flat gazelle]

NameError IG

[quick snow]

When? At lookup of __doc__? At function definition time?

[raven ridge]

If interpolation were allowed, that would have to fail at function definition time.

[fallen slate]

@unkempt rock :white_check_mark: Your eval job has completed with return code 0.

None

[raven ridge]

Right. Only regular string literals can be docstrings. The conversation is about the implications if that were to change to also allow f-strings

Such as, when would the interpolation occur? And, what would happen if it falls?

[covert nova]

Tons of docstrings out there with code examples in them. The thought of possible f-string docstrings makes me shudder a bit. Do tools like Sphinx allow you to do variable substitutions already?

[raven ridge]

Sphinx gives you a hook where you can edit the docstring after it's been extracted and before Sphinx has generated docs from it - so, you could use that to do substitutions if you wanted to, I suppose

though I can't imagine why you'd want to

[frozen harbor]

random question, but has there been any more discussion since PEP 463 on except expressions?

[elder blade]

I remember seeing a post in #mailing-lists about it.

[boreal umbra]

Can anyone think of a way to have cached class properties?

[peak spoke]

could make your own descriptor

[boreal umbra]

I think I'd have to know how the classmethod decorator works

[peak spoke]

it looks like cached_property from functools almost works, but doesn't like the __dict__ of types

if that's adjusted to use getattr/setattr with a mangled name it should work

[boreal umbra]

mangled. what do you mean?

[peak spoke]

something added to it so it doesn't get in the way of the user's names, as you can't use the attribute name from __set_name__ itself or the descriptor would trigger itself

https://paste.fuelrats.com/esupowijid.py for example, where it's just https://github.com/python/cpython/blob/7a822c92782ffda8fa32a4b30a95b9de7cc1b8e6/Lib/functools.py#L938-L992 but done through the attribute and with removed dict checks

[boreal umbra]

interesting; did you make that?

[peak spoke]

Oh the classmethod messes up the descriptor's setname so looks like that'd have to be done manually through the func's name in the init instead of the setname magic

I made the setattr/getattr changes to the class from functools if that's what you mean

[jolly depot]

Hi

[swift imp]

Can you be more descriptive on this

Like if I access through an instance what happens

[swift imp]

Yes it does. When you decorate multiple descriptors only the most outer gets __set_name__ called. That's because __set_name__ gets called at end of __new__ of the classes meta for anything in the namespace that has __get__ attribute

[boreal umbra]

it would be the same as if you had accessed it through the class; same as a class method.

[swift imp]

So like

[swift imp]

There is this types.DynamicClassAttribute it may solve your problem. Other than that you would have to make a custom type and add the descriptor there

A descriptor that does caching is pretty simple

[boreal umbra]

@swift imp thanks

[swift imp]

https://stackoverflow.com/a/36062031/11871687

Youll have to do type programming in either case

[lusty scroll]

classmethod strikes again

[lusty scroll]

do you know the properties in advance?

[lusty scroll]

perhaps it might be better to edit the generated HTML, at least that way it would be semi-structured?

[boreal umbra]

I'm not sure I understand what you're asking. The point is to have a cached property that's shared by the whole class

[visual shadow]

Easier to make a method that accesses a cached class variable, but if you must, i think this requires a metaclass to achieve.

So it's much easier to just have a private (sic) class variable that's being accessed by a "get_var" style of method.

[peak spoke]

the cached property copy I posted above should work fine with a classmethod, just needs to create the name from the function obj or create it itself instead of relying on setname

or doing the classmethod handling inside of its get

[lusty scroll]

OK, I guess I misunderstood your goal. Don't worry about it 🙂

[boreal umbra]

👻

[swift imp]

What's the benefit of it being a class attribute

[boreal umbra]

subclass-aware behavior

[swift imp]

Why not make it a special namespace object

I'll explain in a second

[swift imp]

So like is a situation where you compute something on an instance and it's set on all other instances and instances of subclass?

I'm confused why this needs to be cached

Or you just don't want subclasses to be able to change the value of the attribute ?

[raven ridge]

is the goal lazy computation, or are you OK with it being computed at the point where the class is defined? If the latter, why not just use a regular class attribute?

that is, what do you want that you wouldn't get from: ```py
class MyClass:
attibute = 42

[boreal umbra]

I want it to be recalculated once automatically for each subclass, without having to rewrite it.

[raven ridge]

define an __init_subclass__ on MyClass that sets it?

[boreal umbra]

could work

[swift imp]

This

If you just need it to be different for each subclass

[raven ridge]

!e ```py
class MyClass:
attribute = 42

def __init_subclass__(cls):
    cls.attribute = 24

class SubClass(MyClass):
pass

print(MyClass.attribute)
print(SubClass.attribute)

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | 42
002 | 24

[white nexus]

hey what is the most efficient class, say I have several strings that are being passed as keyword arguments, presumably tuples would be the most efficient and then go to dict.fromkeys, right?

[surreal sun]

is there a way you can make a logical expression lazy? i would imagine no but if there's some hacky way I'll take it

I'm trying to make a filter method for a class I made (I know filter exists, just wanted to try something out) and for fun I thought I'd make an attempt to make logical expressions lazy, so something like instance.filter(x or y or z), and I wouldn't evaluate x, y, or z till I'm in the method

i could do something like
Class(x) | Class(y) | Class(z), but atp you could just use the regular filter method

[feral cedar]

pass a string and eval it inside

[surreal sun]

I was gonna do that but tbh it looks kind of ugly

I guess the only way that looks nicer is to just do what filter does normally with a callable

[quick snow]

https://github.com/L3viathan/lazex

[surreal sun]

Ooh this is cool, thank you

@quick snow out of curiosity, how does this work? i looked at the source code but i got lost 😅

[white nexus]

Are function attribute getters slow?
someone asked me this and now I'm wondering.. what is the slowest getter in python?

[quick snow]

Basically patching these calls in decorated functions and replacing foo(x + y) with foo(lazy("x + y"))

[surreal sun]

ohh that's cool, but given that you're decorating it wouldn't the arguments already be evaluated before the wrapper itself is called per se

foo(2 + 3) would be simplified to foo(5) before foo is called, I would think so at least right?

[quick snow]

Correct, that's why the function that makes that call needs to be decorated. It's not perfect.

[surreal sun]

but even if it's decorated, with wrapper, the *args and **kwargs would already be evaluated though right?

[feral cedar]

getters can execute arbitrary code, so there isn't really a cap if you include user defined ones

[quick snow]

but x+y becomes a string and doesn't get evaluated at all

[white nexus]

true... what's the slowest/fastest built in getter

[quick snow]

(I think. It's been a while)

[white nexus]

like, is a class attribute getter faster than a function getter?

[surreal sun]

sorry about all the questions, I'm still confused, but my question is how do you prevent them from being evaluated even though the function that wraps it gets called after the evaluation of the arguments?

[quick snow]

I'll explain later, in a meeting right now

[surreal sun]

ah alright, thanks!

oh wait, I understand nevermind

[surreal sun]

the way you did it tho looks a lot better than how I would do it.

I would probably just inspect the call stack, get the source code and then take those arguments passed in and put them into a string, but that wouldn't work in the REPL so it's not a good way

[quick snow]

Yes, and also it wouldn't work if there's side effects. Think my_function(os.system("rm -rf /*"))

Alright, so: Let's say there's two functions a and b, where a calls b with some argument(s) that should not be evaluated immediately when calling.
Both a and b are decorated with lazex.me: This basically walks through the function and looks for function calls. Whenever one is found, b(foo, bar + bar) is replaced with b(Expression("foo"), Expression("bar + bat")) (with the use of astunparse) by replacing the function's code object. Note that this happens before any of those functions is executed (hopefully). It also only happens if the function that is being called is also decorated with lazex.me, otherwise you could never call any normal functions from within a decorated function (because most functions don't know what to do with an Expression object). When a is executed and it calls b, the Expression objects get instantiated, but their values are just strings. The Expression objects look up the calling frame at this point to get a reference to the locals and globals where it is called from. Later, when evaluate() is called, the expression is recursively evaluated with those globals and locals.

@surreal sun ^

[surreal sun]

Ohhh

[swift imp]

I really wish python had a way of adding docstrings on class attributes

Or just for variables in general

This should be possible even if the thing has no __doc__

[peak spoke]

something like __annotations__ would be nice for that, but it is something that is usually only needed for the reader and static tools which can already do it

[swift imp]

But that doesn't work with help

[peak spoke]

ah, to be honest I can't remember the last time I used help for anything other than the signature of functions, I usually jump to the source and fold the code

[swift imp]

I use it all the time

I'm a vim + tmux person

So I just have a repl open all the time

Be able to see the docstring and repr of a descriptor/class attribute would be nice and for properties and property like descriptors see the docstrings/signatures of each get/set/delete

[dusk comet]

python on windows has a bug related to printing long lines: strings longer than 16kb are truncated to 16kb
this happens because cmd.exe has a print buffer of 16kb (echo also cuts lines to 16kb)
I propose to change the behavior of the output streams so that they output long strings in parts of 16kb

size = 2 ** 14
print('<' + (size - 2) * '-' + '>') # truncates
print('<' + (size - 3) * '-' + '>') # ok

[flat gazelle]

adding intermittent flushes to an output stream is a breaking change, so I doubt that's going to happen.

[spark magnet]

why would an extra flush be a breaking change? There's no guarantee about when things get flushed.

[halcyon trail]

yeah, what ned said

[raven ridge]

What happens if you:

size = 2 ** 14
print('<' + (size - 3) * '-' + '>', end="")
print('!')

Does the ! show up?

[dusk comet]

yes

every call to stdout.write prints up to 16k chars

i tried wrap stdout in custom object and write long lines in several calls to stdout.write
in this case, all strings are printed correctly

[boreal umbra]

I was just reading fluent python, as an exciting person does on a Friday night, and this is the first time I had it explained why subclassing builtin types is a bad idea. Namely that methods which would in principle involve calling other methods often operate on the underlying C data directly, so methods that are overridden in the subclass may not be called.

[elder blade]

Yeah definitely, you need to use it for the right purpose

Subclassing built-in types (or any kind of C-implemented type) is always a bad idea if the plan was to override magic methods

[swift imp]

Only builtin I've subclasses is dict and that was for __missing__.

[native flame]

there's a collections.UserDict

[deft pagoda]

userdict is deprecated i think, or at least no longer encouraged

[white nexus]

according to the docs not deprecated

yes i legit have the collections docs open rn and was already reading them for something

[deft pagoda]

oh this:

The need for this class has been partially supplanted by the ability to subclass directly from dict;

[dull jungle]

class Immortal:
    def __del__(self):
        globals()['reference']=Immortal()
        print('Attempt has been made to destroy Immortal object')
        print(f'Deleted {self}')

Immortal()

how does this work

it prints the messages twice and prints nothing else and also the program doesnt end

[raven ridge]

!d object.__del__

[fallen slate]

object.\_\_del\_\_

object.__del__(self)```
Called when the instance is about to be destroyed. This is also called a finalizer or (improperly) a destructor. If a base class has a [`__del__()`](https://docs.python.org/3/reference/datamodel.html#object.__del__ "object.__del__") method, the derived class’s [`__del__()`](https://docs.python.org/3/reference/datamodel.html#object.__del__ "object.__del__") method, if any, must explicitly call it to ensure proper deletion of the base class part of the instance.

It is possible (though not recommended!) for the [`__del__()`](https://docs.python.org/3/reference/datamodel.html#object.__del__ "object.__del__") method to postpone destruction of the instance by creating a new reference to it. This is called object *resurrection*. It is implementation-dependent whether [`__del__()`](https://docs.python.org/3/reference/datamodel.html#object.__del__ "object.__del__") is called a second time when a resurrected object is about to be destroyed; the current [CPython](https://docs.python.org/3/glossary.html#term-CPython) implementation only calls it once.

[raven ridge]

the docs specifically cover that - the last sentence of the docs snippet there.

[dull jungle]

i read that already after that only i tried it

my question is why its printing only twice not infinite

[raven ridge]

hm, actually, I'm not sure...

I didn't notice you were creating a global reference to a new instance, I thought you were resurrecting the existing one

[unkempt rock]

It is not guaranteed that del() methods are called for objects that still exist when the interpreter exits.

Ah, the process doesn't end either 🤔

[dull jungle]

yea

then what process is still running that makes the program not stop

[unkempt rock]

the process does end when you add an extra call to print lol

class Immortal:
    def __del__(self):
        print('called del for', self)
        globals()['reference']=Immortal()
        print('Attempt has been made to destroy Immortal object')
        print(f'Deleted {self}')

[dull jungle]

what

ya it ends but why how

[unkempt rock]

Well, i'm guessing the current object gets resurrected for just a tiny bit because im referencing it

as to why that would change the behaviour - not sure

[dull jungle]

but there is a print that uses self already

maybe because del doesnt delete the object it just decreases the count of references by 1

[unkempt rock]

well yes, but youre resurrecting after the creation of the new object

class Immortal:
    def __del__(self):
        print(f'Deleting {self}')
        globals()['reference']=Immortal()
        print('Attempt has been made to destroy Immortal object')

Immortal()
``` has the same effect

[dull jungle]

how does that make any difference ahh

also im not resurrecting

after printing self it gets deleted

[unkempt rock]

i dont even think this counts as resurrection, we're not really postponing the destruction but clearly increasing the refcount does something lol

[dull jungle]

refcount is not increased because im creating a new Immortal() object so not creating reference of old one

i guess its some 'flawly made way' to prevent creating infinite objects

[unkempt rock]

passing the object to print adds a new reference

[dull jungle]

but it will be deleted then and there right

[unkempt rock]

yes, the refcount will go back down

the actual deletion is not until the end of del though

since we still have a reference to the object as self

[dull jungle]

how does changing the position of print statement change the output

class Immortal:
    def __del__(self):
        print(self)
        globals()['reference']=Immortal()
        print('Attempt has been made to destroy Immortal object')

Immortal()

program ends

class Immortal:
    def __del__(self):
        self
        globals()['reference']=Immortal()
        print('Attempt has been made to destroy Immortal object')

Immortal()

program doesnt end

why is that so

maybe because self is active inside print function?

[unkempt rock]

passing any object to any function will at the minimum temporarily increase its refcount

[raven ridge]

Ah. Actually, I know.

[unkempt rock]

its the same reason sys.refcount will seem to have one extra reference to the object

[raven ridge]

It's creating an infinite number of instances of this class, destroying each one, and creating a new one each time. Of course that's an infinite loop

[dull jungle]

but putting a print statement stops it?

[unkempt rock]

But then we'd see the prints executing right

[dull jungle]

also if its infinitely deleting it should print each time

but it prints only twice

[raven ridge]

No, because print has already been destroyed

[dull jungle]

what?!

[unkempt rock]

oh that makes sense lol

[dull jungle]

wdym print gets destroyed

[unkempt rock]

del() can be executed during interpreter shutdown. As a consequence, the global variables it needs to access (including other modules) may already have been deleted or set to None. Python guarantees that globals whose name begins with a single underscore are deleted from their module before other globals are deleted; if no other references to such globals exist, this may help in assuring that imported modules are still available at the time when the del() method is called.

[dull jungle]

also this is a lie

i tested it

it doesnt give priority to underscores

[raven ridge]

The point where this is getting cleaned up is the point where module globals are being set to None. After your second print, the Immortal class no longer exists, and print probably doesn't either

Or maybe print exists but sys.stdout doesn't

[dull jungle]

!e

class test():
  def __del__(self):
    print(self)
_obj=test()
obj=test()
print(_obj,obj)

[fallen slate]

@dull jungle :white_check_mark: Your eval job has completed with return code 0.

001 | <__main__.test object at 0x7fb63c5a5b70> <__main__.test object at 0x7fb63c5a5b40>
002 | <__main__.test object at 0x7fb63c5a5b70>
003 | <__main__.test object at 0x7fb63c5a5b40>

[dull jungle]

!e

class test():
  def __del__(self):
    print(self)
obj=test()
_obj=test()
print(_obj,obj)

[fallen slate]

@dull jungle :white_check_mark: Your eval job has completed with return code 0.

001 | <__main__.test object at 0x7f1f309bdb40> <__main__.test object at 0x7f1f309bdb70>
002 | <__main__.test object at 0x7f1f309bdb70>
003 | <__main__.test object at 0x7f1f309bdb40>

[dull jungle]

see the order is first created deleted first

the underscore did nothing

[raven ridge]

But either way, the impact is the same. When the __del__ tried to execute print(), it fails with an exception that can't be printed. If that happens after you create a new Immortal object, you infinite loop. If it happens before you create the new Immortal object, the __del__ dies without ever creating another instance and the program ends

[unkempt rock]

yeah that seems to be it

[dull jungle]

ooh

wow this makes sense

btw can you explain this scam

[unkempt rock]

It's talking about globals from modules

[raven ridge]

In both of those cases you evaled above, the _ one was deleted first, right?

[unkempt rock]

class Foo:
    def __del__(self):
        print(self)
import math
math.a = Foo()
math._a = Foo()
print(math.a, math._a)

<__main__.Foo object at 0x7fd2f0cd58e0> <__main__.Foo object at 0x7fd2f0cd5040>
<__main__.Foo object at 0x7fd2f0cd5040>
<__main__.Foo object at 0x7fd2f0cd58e0>

[dull jungle]

not rly

the normal obj was deleted first

[raven ridge]

Ooh, you're right. Hm.

[raven ridge]

__main__ is a module

[unkempt rock]

hmm

[dull jungle]

either the doc scammed or they meant something else

[unkempt rock]

it does seem to hold true for assigning to modules other than dunder main

[raven ridge]

Yeah. Most likely there's something special about dunder main, but I'm not sure what or why.

[dull jungle]

hmm is there a way to get output without print

[raven ridge]

Write directly to sys.stdout, or call os.write to fd 1

[dull jungle]

from async_timeout import sys


import os
class Immortal:
    def __del__(self):
        try:
            print(self)
            globals()['reference']=Immortal()
            print('Attempt has been made to destroy Immortal object')
        except:
            os.write('print is deleted :(')


Immortal()

[unkempt rock]

oh one second

[dull jungle]

this prints nothing 😮

[raven ridge]

Since references to sys or os might have been dropped by the point a dunder del runs, you'd need to save a reference to them.

!d os.write

[fallen slate]

os.write

os.write(fd, str)```
Write the bytestring in *str* to file descriptor *fd*.

Return the number of bytes actually written.

Note

This function is intended for low-level I/O and must be applied to a file descriptor as returned by [`os.open()`](https://docs.python.org/3/library/os.html#os.open "os.open") or [`pipe()`](https://docs.python.org/3/library/os.html#os.pipe "os.pipe"). To write a “file object” returned by the built-in function [`open()`](https://docs.python.org/3/library/functions.html#open "open") or by [`popen()`](https://docs.python.org/3/library/os.html#os.popen "os.popen") or [`fdopen()`](https://docs.python.org/3/library/os.html#os.fdopen "os.fdopen"), or [`sys.stdout`](https://docs.python.org/3/library/sys.html#sys.stdout "sys.stdout") or [`sys.stderr`](https://docs.python.org/3/library/sys.html#sys.stderr "sys.stderr"), use its `write()` method.

Changed in version 3.5: If the system call is interrupted and the signal handler does not raise an exception, the function now retries the system call instead of raising an [`InterruptedError`](https://docs.python.org/3/library/exceptions.html#InterruptedError "InterruptedError") exception (see [**PEP 475**](https://www.python.org/dev/peps/pep-0475) for the rationale).

[raven ridge]

Try:

import os

class Immortal:
    def __del__(self, write=os.write):
        write(1, (str(self) + "\n").encode())
        globals()['reference']=Immortal()

reference = Immortal() 

I'm on mobile, can't try it myself.

[dull jungle]

a byte like object is required

[raven ridge]

Fixed

[dull jungle]

it prints only once

then stops

this prints twice and program doesnt end

[raven ridge]

Then at the point where it would fire a second time, stdout has likely already been closed.

[dull jungle]

but if i dont use output and i use print directly, it ends

this is so confusing

your theory is partially correct i guess

[raven ridge]

Hm, ok. So the builtins have already been removed from your module namespace, and so calling print fails

[dull jungle]

but why cant i capture the exception

[raven ridge]

You can, you just can't do anything much with it since stdout has already been closed

[dull jungle]

oh

what if i import print inside the del

but print is built in how can i import it

[unkempt rock]

prefix it with an underscore?

[raven ridge]

Actually .. Try: ```py
import os

class Immortal:
def del(self, system=os.system):
system("echo " + str(id(self)))
globals()['reference']=Immortal()

reference = Immortal()

[dull jungle]

???

system argument must be string not byte

[raven ridge]

lol really

[dull jungle]

i remove the encode

[raven ridge]

Fixed

[dull jungle]

and it prints a number

[raven ridge]

But only once. Makes sense, fd's are inherited from the parent process... Ok...

[dull jungle]

and program ends

how can i import print

[raven ridge]

Oh, because str is gone...

[dull jungle]

nvm i found it from builtin import print

[raven ridge]

Try: ```py
import os

class Immortal:
def del(self, system=os.system, str=str, id=id):
system("echo " + str(id(self)))
globals()['reference']=Immortal()

reference = Immortal()

[dull jungle]

whoa new exception: acheivement unlocked

strange... del is supposed to ignore errors

[raven ridge]

Yeah, importing stuff while the interpreter is shutting down is unwise.

[raven ridge]

It did, that's why that message says "exception ignored in"

[unkempt rock]

that works

and.. doesn't let you use ^C to exit out

[raven ridge]

As in, prints an infinite number of numbers? Sweet.

[raven ridge]

Uh, on Unix ctrl-z should suspend it

[dull jungle]

my only doubt now is why is this not ending

[raven ridge]

And then you can kill %%

[unkempt rock]

lolol im p new to unix, just rebooted my vps instead

[raven ridge]

lol

[dull jungle]

how is printing on top and bottom make a difference

[raven ridge]

Why would it? Python loops, destroying module globals. Every pass, you create a new global. Ergo, the loop never terminates.

[dull jungle]

it should get error print doesnt exist right

[raven ridge]

It doesn't, you can move that system call wherever.

[dull jungle]

why the error isnt raised this time

dont say raise got deleted 🤣

[raven ridge]

Ah, exceptions in dunder del are normally ignored silently, I think. They might show up if you run the interpreter with -Xdev as a command line argument, though I'm not certain.

[raven ridge]

Because we saved a local reference to every object we need. We have a saved reference to str and id and system inside our function object as default arguments, so we can refer to those even though the globals have been destroyed

Which was the mistake I made in the write attempt

[dull jungle]

class Immortal:
    def __del__(self):
        print(self)
        globals()['reference']=Immortal()
        print('Attempt has been made to destroy Immortal object')
        

Immortal()
>>>program ends

class Immortal:
    def __del__(self):
        globals()['reference']=Immortal()
        print('Attempt has been made to destroy Immortal object')
        print(self)
        

Immortal()
>>>program doesnt end

how are these two different

[raven ridge]

Try:

import os

class Immortal:
    def __del__(self, write=os.write, str=str, globals=globals):
        write(1, (str(self) + "\n").encode())
        globals()['reference']=Immortal()

reference = Immortal() 

[dull jungle]

it prints thrice and the program ends wth

[raven ridge]

In the first case, the lookup for the name print fails, terminating the __del__ before creating a new Immortal object. In the latter case the new object gets created before the __del__ fails

[dull jungle]

wow that explains everything

you are genius thanks

[raven ridge]

That was an interesting puzzle 😄

[verbal escarp]

Finally able to get back here, at least via mobile.. gotta read up 🍿

[sudden plaza]

Sudo airodump-ng Start wlan0

😵‍💫😵‍💫😵‍💫

[verbal escarp]

Haha.. I did a little warwalking the other day.. only closed networks and a single open one, on the other side of town.. so hard to live without internet as a netizen

[sudden plaza]

Yeah I can say the same thing

I can’t live without Internet

[verbal escarp]

Paid for mobil data to be able to order stuff from Amazon, ended up paying for all thos video ads

[sudden plaza]

Yeah I have a future with my own

I love running parrot OS because I am a beginner

Kali Linux it’s not hard it’s just not in my favor

[verbal escarp]

Never bothered about those little video ad popups, but once you realize the GB of data they make up and you have to pay by GB..

[sudden plaza]

Packages in injection monitor mode and everything

Just download a ad blocker to your fire fox

Or tor

[verbal escarp]

I have, but some pages don't work without

[sudden plaza]

Mines this

Adblock

[verbal escarp]

I use Fitefox Clear on my mobile as default, which blocks everything, but some pages require you to click on stuff etc

[sudden plaza]

Don’t use your phone use a laptop

[verbal escarp]

Sorry about typos

[sudden plaza]

It’s koo

We all miss up even developers mess up

We’re here to learn and fix are errors in our python/anything that we do

Error to be made as a lesson is

If there was no errors we will never learn

😵‍💫

[verbal escarp]

Well, I'm on my neighbors wlan by grace.. been waiting for 3 weeks on internet technician to set up my connection

[sudden plaza]

So what are you and four on this server

[verbal escarp]

Me and four?

I've come on discord originally from IRC when freenode collapsed, mostly needing feedback and help with justuse. Learned a lot in here about python internals I needed to realize some features

[verbal escarp]

Finally read up tha last 30 days of logs.. a little disappointed tbh. Expected more crazyness

[haughty fern]

Hi

I need to get voice verification

Can anyone help

Pls help

[grave jolt]

@haughty fern have you read #voice-verification?

[boreal umbra]

!otn a godly's immortal instance

[fallen slate]

:ok_hand: Added godly’s-immortal-instance to the names list.

[grave jolt]

!e
Easy tail recursion!

class Fibonacci:
    def __init__(self, n, a, b, on_done):
        self.n = n
        self.a = a
        self.b = b
        self.on_done = on_done

    def __del__(self):
        if self.n == 0:
            self.on_done(self.a)
        else:
            Fibonacci(self.n - 1, self.b, self.a + self.b, self.on_done)

def fibonacci(n):
    result = None

    def on_done(x):
        nonlocal result
        result = x

    def _run():
        Fibonacci(n, 0, 1, on_done)

    _run()
    return result

print(fibonacci(2000))

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

4224696333392304878706725602341482782579852840250681098010280137314308584370130707224123599639141511088446087538909603607640194711643596029271983312598737326253555802606991585915229492453904998722256795316982874482472992263901833716778060607011615497886719879858311468870876264597369086722884023654422295243347964480139515349562972087652656069529806499841977448720155612802665404554171717881930324025204312082516817125

[sudden plaza]

I here u

[gusty marsh]

Is it possible to detect unicode escape sequences in a string?
eg: "AAAAAAAAAA \u200B A", I want to get the index of the unicode escape sequence.

Would regex be the only option?

[quick snow]

That string contains no unicode escape sequence. Do you mean "AAAA \\u200B A"?

Unless you mean you want to find "non-printable characters, except those in a whitelist"

[gusty marsh]

Yeah meant \\, but even \ fits my use case

[quick snow]

You don't mean "finding zero-width joiners" (if I guessed the codepoint correctly?), but find sequences of backslash, u, four hex digits. In that case yes, regex would be a good option

[gusty marsh]

no, basically i want to switch the uppercased unicode characters like B here to lowercased (related to #black-formatter)

[quick snow]

Nevertheless, good fit for regex. You can hand re.sub a function instead of a replacement string, then the lowercasing should be easy.

[gusty marsh]

Yeah that's what my current solution looks like, but its really the best, wanted to check if there are any python internal methods/functions doing the same task

[sturdy timber]

A regex wouldn't work right? As you need to match \u200B (unicode escape) but not \\u200B (backslash followed by arbitrary characters), and by extension you need to match \\\u200B but not \\\\u200B, so you'd need to parse it properly somehow?

[gusty marsh]

maybe I could check for an even number of back-slashes

[flat gazelle]

it could be done with a regex, but it would be quite annoying

[gusty marsh]

yeah, would probably have many edge cases 😔

[flat gazelle]

it may be easier to have a regex match both \\ and \uFFFF and then just findall and filter.

[gusty marsh]

r"[\\u00C0-\\u1FFF\\u2C00-\\uD7FF\w]" this is what I currently have, I am just checking for a \ initially, and then split out the \ and check if there is an even number of them

[gusty marsh]

should actually be the opposite since when we get the unicode escaped, it would \\

"\\UU0001f977" -> "\UU0001f977" escaping the \U

so would "\\\\UU0001f977" be a valid unicode escape sequence? since it ends up being "\\UU0001f977"?

[quick snow]

(\\\\)*?(\\?)(\\u\d) should work. Then do something or nothing depending on the truthiness of the second capture group

[digital panther]

hi , i need help , i have this string and i need to take from this string only things behind ,name=(names), I know how to get extract nickname from that string , but i need o it multiple times how many times i have ,,name= ,,there

[vast saffron]

for match in re.finditer(r"name\=(?<name>.*)"):
    name = match.group("name")

depends on what characters can be in name, so best to replace that .* with something more specific, or if those entries are delimited by something, maybe split the string first by that delimiter. Can help you if you have an example string.

[white nexus]

TIL docstrings for attributes actually exist in python

!e ```py
import inspect

class Huh():
slots = {'wut':'So you can actually document attributes?!'}

h = Huh()
print(inspect.getdoc(Huh.wut))

[fallen slate]

@white nexus :white_check_mark: Your eval job has completed with return code 0.

So you can actually document attributes?!

[surreal sun]

Woah

[grave jolt]

[deft pagoda]

can you do it without slots

[frigid trout]

only with descriptors

[peak spoke]

Just noticed shutil.get_terminal_size is a thing, why is it a thing in shutil?

[naive saddle]

I ask the same question every time I use it

[raven ridge]

hm. Is there any other place it would fit better?

[white nexus]

os?

sys?

[raven ridge]

there is an os.get_terminal_size(), but it's lower level and not quite the same

shutil.get_terminal_size() respects os.environ["LINES"] and ["COLUMNS"]

it's a bit of a strange place for it, but it seems to be sticking with the metaphor that shutil is for high level things that have shell command counterparts

[peak spoke]

I guess just os, even with it being a higher level operation. Feels like a more natural place to look for it

[raven ridge]

hm. I would never have thought to look in os for it - though, I didn't know there was an os.get_terminal_size() until today, so 🤷

[verbal escarp]

Putting logic in __del_ is so very ingeniously mad, I like it 👨‍🔬🥜

But I somewhat doubt there is a performance advantage 🤣

[grave jolt]

yeah it's not great tbh

A normal tail-recursive function is about 215 us

hmm. with __slots__ it's about 700us

[fallen slate]

**PEP 673 - Self Type**

Link

Status

Draft

Python-Version

3.11

Created

10-Nov-2021

Type

Standards Track

[heady mauve]

How does exec handle globals and locals? Interestingly enough, when I pass a blank dict for them, imports in the executed file are discarded.

[peak spoke]

the generated code obj uses the dicts you passed instead of inheriting the scopes you called it from

[grave jolt]

!e

g = {}
l = {}
exec("import math", g, l)
print(g.get("math"), l.get("math"))

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

None <module 'math' from '/usr/local/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so'>

[heady mauve]

yeah, but if you exec "import random\nrandom.randint(0, 4)" what happens to random?

[frigid trout]

!e

exec("import random\nrandom.randint(0, 4)")

[fallen slate]

@frigid trout :warning: Your eval job has completed with return code 0.

[No output]

[grave jolt]

!e

g = {}
l = {}
exec('import random\nprint(random.randint(0, 4))', g, l)

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

0

[grave jolt]

It gets put into l

[heady mauve]

what...

[grave jolt]

what's wrong?

If you pass in the dicts, it uses those dicts for the global and local scope, and it mutates them

[heady mauve]

Well I had a script I execed and it imported, but couldn't find the imports two lines later oh I get it

[high wolf]

weird pep, I tried the code and didn't encounter the problem it describes
the type of Circle().set_scale(0.5) is Circle, not Shape, as they claim

[heady mauve]

this makes a lot of sense

[peak spoke]

in what checker?

[high wolf]

the pep doesn't mention what checker, it's referred to as "the type checker"
and python has no built-in type checker afaik, so I assume they meant the way classes function in general
I did type(Circle().set_scale(0.5)) and it was Circle, and got no error

[grave jolt]

The PEP is talking about type hints

not runtime behaviour. Of course at runtime it will be a circle

[high wolf]

yes
but why make a pep when the problem is third party checkers

[grave jolt]

If you're not familiar with type hints, we have some resources in the pins of #type-hinting 🙂

[high wolf]

I've used them

[peak spoke]

the problem is what python provides as a language to the type checkers

some type checkers have supported Self for a while even if it's not in python

[grave jolt]

The PEPs are trying to come up with a consistent set of primitives for type checkers to understand. It would be a nightmare if a library had to provide different stubs for mypy, pyright, pytype, pyre, pyanalyze, PyCharms' own thing and whatever comes next

Or if a big application only worked with PyCharm's type checker

||i may or may not have had one like that at work...||

[heady mauve]

So back to the whole exec thing... Why does this code raise a NameError for 'element' is not defined on element.__init__(self), but not on the class definition that also references element?

from logical_utils.logic.core import pin, element
from logical_utils.ui import vec2, widget

class pyElement(element):
    def __init__(self, colorR, colorG, colorB, posX, posY):
        element.__init__(self)
        ...

[high wolf]

but wdym worked? the interpreter ignores type hints

it will work

[heady mauve]

For reference this is how I'm execing code:

...
globalVars = {}
localVars = {}
exec(compile(source, filePath, 'exec'), globalVars, localVars)
...

[peak spoke]

Not all peps are concerned with runtime behaviour, they're there as enhancements to the language

[grave jolt]

Type hints provide three main things:

• formally checkable documentation
• catching mistakes (such as passing an int | None where an int is expected)
• providing autocompletion and hints in IDEs.

If all type checkers used incompatible primitives (like if a library uses an imaginary mypy.Union, but want to use a different type checker, like pyright), you only get those benefits on one particular type checker.

[high wolf]

ok, so I guess this means some type checkers don't work, and others have different custom ways of detecting the type
and this aims to standardize it
makes sense then

[grave jolt]

There are different ways of evolving the ecosystem, of course. For example, in JavaScript there were several incompatible typed extensions like Flow and TypeScript. In the end TypeScript won because it was the solution most people liked ||and because it was backed by microsoft||, not because it was decided by a committee.

And actually, I think the Python typing thing could be defined better. Stuff like type inference is not really defined at all

or e.g. pyright assigns some meaningful semantics to making a TypedDict @final, but mypy prohibits a @final on a TypedDict

[raven ridge]

The point of type hints is to explain to static analysis tools what types are expected for the parameters of a function, and what type is returned by that function. Up until now, there hasn't been a good way to explain that a function returns self, and this PEP is adding a nice way to declare that.

it's not about a defect in any particular type checker, it's about a feature that was missing from the entire type hint ecosystem.

the type hint "language" didn't give a good way to describe this.

that is, the method signature py def set_scale(self, scale: float) -> Scale: and the method signature py def set_scale(self, scale: float) -> Self: mean something different. Type checkers aren't wrong for what they interpret -> Scale to mean; it just means a different thing.

[bronze steeple]

I can’t seem to find the source code for ‘deque’. Is it not in cpython GitHub ?

[white nexus]

https://github.com/python/cpython/blob/eaeb99468045b863d2dd3da3e3d1c3c9c78e1254/Lib/collections/__init__.py#L41-L46

[fallen slate]

Lib/collections/__init__.py lines 41 to 46

try:
    from _collections import deque
except ImportError:
    pass
else:
    _collections_abc.MutableSequence.register(deque)```

[white nexus]

seems like its in c by this?

[raven ridge]

yep.

[white nexus]

!e ```py
import collections, inspect

print(inspect.getsource(collections.deque))

[peak spoke]

most of the code is for the deque implementation https://github.com/python/cpython/blob/3.10/Modules/_collectionsmodule.c

[fallen slate]

@white nexus :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 3, in <module>
003 |   File "/usr/local/lib/python3.10/inspect.py", line 1147, in getsource
004 |     lines, lnum = getsourcelines(object)
005 |   File "/usr/local/lib/python3.10/inspect.py", line 1129, in getsourcelines
006 |     lines, lnum = findsource(object)
007 |   File "/usr/local/lib/python3.10/inspect.py", line 974, in findsource
008 |     raise OSError('could not find class definition')
009 | OSError: could not find class definition

[bronze steeple]

Ok so it’s all in c. Thanks folks

[glass robin]

If a PR is stalled due because its contributor not signing the CLA, is it possible to reuse its code in another PR, or do they effectively prevent the issue from being solved the way they did?

[verbal escarp]

Is there any way to serialize closures?

[elder blade]

They basically hold the copyright to the changes they made. It's not that say the exact sorting algorithm they used is claimed I'd say.. but you can't commit those exact same changes.

Of course I am not a lawyer, but there has to be a line to draw. For example, someone not signing the CLA to fix a typo probably doesn't mean you can't commit that same typo if you made the change yourself.

[elder blade]

What is the behaviour expected? You can serialize the function that creates closures.

[verbal escarp]

I mean the function 'instance' that got returned by calling the outer function with params

[elder blade]

I think that pickle doesn't support it

[verbal escarp]

Outer function.. I wouldn't be asking in here, would I 🤣

[elder blade]

How would you even create a closure function without a function factory like that (so from pickle data)..

[verbal escarp]

One sec

https://github.com/amogorkon/fuzzylogic/blob/master/src/fuzzylogic/functions.py

All those functions work in two steps, setting up and work

I've been pondering for some time how to put 'primed' functions to disk, but I guess I'd need to work with explicit representations

[jolly depot]

👍

[oblique raven]

Why does asyncio's ProactorEventLoop always raise a RuntimeError: Event loop is closed on exit (I don't know if this is relevant, but using aiohttp)? I'm not doing anything incorrectly in my code afaik, and the same code with a Selector policy does not error in the same way.

If this is a bug, why has it stuck around for so long, and why was the proactor made default if it has that bug

Here's a minimal reproduction:

import aiohttp
import asyncio
async def main():
    session = aiohttp.ClientSession()
    (await session.get("https://google.com")).close()
    await session.close()

# Uncomment this to get rid of the error
# asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
asyncio.run(main())

If it wasn't clear so far, this is a windows-specific issue

[pliant tusk]

this would be a breaking change (~~~a right now calls a.__invert__ 3 times)

[feral cedar]

that's unary invert, not unary not

[sacred yew]

yeah not doesn't have a dunder

[verbal escarp]

De Morgan is waving from beyond the grave

[pliant tusk]

oh shoot you're right, nevermind

[warm junco]

There’s a package called dill which probably should be able to do it

!pypi dill

[fallen slate]

dill v0.3.4

serialize all of python

[verbal escarp]

Really? Woah

e! ```
import dill

def func(y):
def foo(x):
return x * y
return foo

f = func(2)

print(dill.dumps(f))```

Holy moly

How???

[pliant tusk]

it works by constructing the code and function objects i think

[verbal escarp]

That's some serious crazyness

Thanks!

[pliant tusk]

its also possible to register code and function objects in such a way that normal pickle will handle them

need to find my poc of doing that

[quick snow]

Basically fishhook.hook the __getstate__/__setstate__, right?

[pliant tusk]

__reduce__ not __getstate__ afaik but there i found my code

it had way more to get around some hardcoded bits in pickle (it tries to save all functions as globals)

[paper echo]

also https://pypi.org/project/cloudpickle/ but heed the warnings on that page

cloudpickle is what joblib uses

[sacred yew]

tbf that warning applies to any usage of pickle, not just cloudpickle specifically

[verbal escarp]

Okay, cloudpickle also can handle the closure

But wheras dill's str is 194, cloudpickle needs 560

Optimization potential

[pliant tusk]

i have yet to see a pickle extension library that doesnt require itself for unpickling i was hoping that was the reason for the extra chars

but nope

[rich cradle]

Is bugs.python.org purely for bugs, or is it also used for suggestions?

I see https://bugs.python.org/issue40280, and that doesn't exactly seem like a bug.

[native flame]

bpo is equivalent to github issues, afaik

[rich cradle]

got it got it

The name was misleading

[gusty marsh]

suggestions are mainly filed in the mailing list iirc

[elder blade]

If you think about it, raising a suggestion as "issue" doesn't make sense either 😅

[rich cradle]

Fair enough

[magic nova]

a lot of the time suggestions bubble up from the mailing lists like python-ideas before being turned into bpo tickets

if you make a bugs.python.org ticket for a really big change you'll likely be directed there

[vast saffron]

I want to do a suggestion at python-ideas for some time now.

I guess I run it through here first to see what people actually think and maybe reality check it:

A new protocol like __fspath__ just for urls as __url__ that returns a string representation of the url. This is to be able to convert and use the multitude of libraries that have url-objects, where whe have to turn them into strings before anyway if we want ro mix currently.

Example:
Using yarls URL and request/httpx get. Especially httpx only accepts its own URL object or a string.

[magic nova]

makes sense theoretically but you might be better off going to the popular http libraries and coming up with a convention like numpy's __array__ instead of going to python-ideas

[vast saffron]

I just think that something so universal used as an url should have a stdlib protocol just like file paths have. Especially when one of the biggest use of python is networking and web-stuff

It does not enforce use, but it at least gives a centralized hint (if that is the right word)

[magic nova]

sure but in those cases most people are using stuff like requests, not the stdlib modules like urllib unlike for IO/filepaths. Demonstrating a need/popularity for this feature outside will make it a lot easier to standardize than just cpython doing it out of the blue

[vast saffron]

Hm I understand what you are saying and yes the success of __fspath__ probably comes down to the stdlib also probividing pathlib

[paper echo]

don't forget about https://pypi.org/project/hyperlink

but i do like the proposal. it's indeed a little frustrating that httpx doesn't support yarl or hyperlink urls

[vast saffron]

and just casting anything to str can lead to shit bugs, especially in scripts. It is one of my big gripes currently, the other is parts of pathlib.

Thank you for that library though, did not know it, will check it out!

[verbal escarp]

We should have a good url representation/implementation in the stdlib. <fullstop>

Anything other, trying to work around this vacuum is exactly that - just a workaround

Not wanting to put stuff in the stdlib due to maintenance is understandable, but then there should be a mechanism to officially endorse mature packages and include them in meta-packages for installation

[quick snow]

Wasn't the point of __fspath__ that different file systems have different restrictions, details, etc., something that isn't true with URLs? Why doesn't str(some_link_type) work?

[grave jolt]

https://docs.python.org/3/library/stdtypes.html#text-sequence-type-str
🙂

[vast saffron]

Why do all the libraries then use an URL object?

[paper echo]

1. nice object-oriented interface
2. you can store the URL in a kind of pre-parsed format, either decoded or encoded as needed

reading the hyperlink docs is very enlightening imo

they try to justify their design decisions in terms of balancing correctness and usability

i think it's good reading for anyone interested in library and api design, even if you don't care about urls

[verbal escarp]

Same argument could be made for pathlib

And nobody argues its use

[grave jolt]

I was not being serious 😛

[verbal escarp]

Oh

Forgive me for being the humorless German of the conversation

[radiant garden]

(that would be bytes)

[verbal escarp]

Pff. Ints. Mapping to positions in pi.

Of course

[radiant garden]

Simply because path contents are not guaranteed to encode valid strings

[verbal escarp]

Windows paths?

[radiant garden]

I believe Unix paths only restrict the null and '/' bytes (within a path component), anything else goes

[verbal escarp]

So bytes for unix, str for windows, everyone's happy

Jokes aside, the topic was url representation

Not filepaths

And urls are escaped, iirc

On the other hand, bytes are default on the wire.. not sure which would be best in that case

[vast saffron]

I also took it as face value, not german, but german speaking. 🙂

[unkempt rock]

Hello

Question:
How python C implementation stores names (locals, globals, imported names, ....)?
In an array, some sort of mapping, sorted array ?

[dusk comet]

globals in dict, locals in array

[split wasp]

Hello. I'm trying to compile Python3.8.9 from source on Windows 10 but getting compiling errors. I've tried a couple of version of python so far and got the same error below:

fatal error RC1116: RC terminating after preprocessor errors

Can anyone here help?

There is an issue for this https://bugs.python.org/issue45220

[raven ridge]

the globals aren't in localsplus

[sick junco]

!rep

[unkempt rock]

Thank u all

[unkempt rock]

hii

I have a doubt

I programmed 2 bots

to chat with each other

but there a problem arises

what should I do

[quick snow]

You should go to #❓｜how-to-get-help

[livid mason]

Why does python have only one memory heap? Wouldn't it be better to split it up and have some sort of a router layer (essentially shard it)?

[flat gazelle]

It more or less does that with arena allocations already. The problem is you still have to manage the shards.

[livid mason]

I also heard that python 3.11 will be 2x faster at least. Is that true?

[prime estuary]

I don't think it'll be that extreme a difference overall, but there's a lot of work being put into speeding up the interpreter - this site has graphs of various benchmarks on different versions: https://speed.python.org/

[verbal escarp]

Improving the performance at least is a goal of 3.11 with quite a few people working full time on it

It's probably a question of which part you're looking at. Some pieces are already reasonably fast, some will benefit more from TLC

[surreal sun]

I've heard it'll be 1.2x faster with the addition of one feature (Adaptive Bytecode), but I can't confirm that

I thought I saw it somewhere though

[verbal escarp]

If you're only working with the scientific stack, there probably won't be big jumps

With ops mostly being outside of python anyways most of the time

[livid mason]

I'm doing web dev with python

[verbal escarp]

That could be an area with big jumps

[livid mason]

Yeah, and I'm excited! Thanks a lot everyone for the responses!

[verbal escarp]

Welcome

[surreal sun]

speaking of web dev w/ Python, isnt CPython able to be compiled to WASM? Something along the lines of that

I wonder what the implications of that will be in the sense that the uses of it

[livid mason]

We could see browsers running native python code

[elder blade]

The only issue is the size of Python

[surreal sun]

Yea maybe it might be something similar to Rust's support where they have multiple libraries for supporting WASM but no direct browser support

[verbal escarp]

I think I've heard a podcast a while ago about this and that it's doable

Talk Python To Me: #212: Python in Web Assembly with Pyodide https://talkpython.fm/episodes/show/212/python-in-web-assembly-with-pyodide

I haven't personally experimented with that though

[stoic igloo]

Hello, I hope you are well, can you help me with this, can you do me a favor, please

in this
This is why we must choose to make the best possible code. Therefore:

Research good programming practices in Python.

Create a summary document, where these good practices are consolidated.

Create a software program that can analyze the Python code of other programs and identify if best practices are being followed.

A summary should be identified and presented at the end indicating which good practices were identified and which were not.

It should give a suggestion on how to apply the good practice.

I already have the document of good practice and bad practice, but I don't know how to do it so that I can identify the practice

I already know how to make him read it line by line

[flat gazelle]

this may be a topic better suited to #type-hinting

[unkempt rock]

oh yeah, thanks

[white nexus]

well this is extremely nice

!e ```py
import pkgutil
print(pkgutil.resolve_name('datetime.datetime.today'))

[fallen slate]

@white nexus :white_check_mark: Your eval job has completed with return code 0.

<built-in method today of type object at 0x7f23ae695d20>

[verbal escarp]

What is it good for?

[tacit hawk]

is http.client useful as http library base?

[white nexus]

finding attributes from just a string, like string annotations but without context, or other cases

i used it to make a feature for my bot to get the source of near anything

[prisma forge]

happy black history month

[vast saffron]

any danger in using this? like with exec? or does it not execute the code when called?

also can it resolve stuff that is not in globals/imported? (guess yes from that snippet, but I want to be sure)

[white nexus]

yes, it uses importlib and getattr to remove anything

danger would be anything that does something when imported

[tender bluff]

When do I use

smth.smth
Vs
from smth import smth

[acoustic tangle]

import x.y will allow you to use y like x.y() whereas from x import y allows you to just say y()

@tender bluff

Sorry didn't ping on reply

[tender bluff]

Ik that but "when"

[acoustic tangle]

If you want to import a bunch of stuff from x then you will want to use import x.y so that the code is more readable; if it's just one or two things then from x import y

[steel jasper]

this happened when I tried to do pip install kivy

how do I download it without it doing this?

[vast saffron]

if you are on python 3.10, I read that kivy does not have a release for python 3.10 (not 100% sure) and this could be the problem

[prisma urchin]

d

[quiet crane]

I have never used import a.y, I think it's something I would only use if there was a name clash on y, i.e. I also wanted to do import b.y

Does anyone know of discussions on range syntax like 0..10 appearing in python?

[quiet crane]

Ok I might be lying, they appear in our code, but most often times in the form of `import a.LongCoolName as lc_name"

[quick snow]

https://www.python.org/dev/peps/pep-0204/

[white nexus]

where is a module's repr located?!

[spark magnet]

the code would be in C someplace. what's up?

[white nexus]

found it-- it was the __spec__ attribute

what's up?
its probably best if i don't explain 🙃

[dull flower]

how do I do files and exceptions in python

[maiden sonnet]

you mean open, try and except? wrong channel btw

[unkempt rock]

How come this attempted emulation of zip doesn't work? (it yields () eventually which won't unpack to a, b)

!e ```py
def my_zip1(*iterables):
iters = [iter(i) for i in iterables]
while True:
try:
yield tuple(map(next, iters))
except StopIteration:
return

for a, b in my_zip1(range(3), range(5)):
print(a, b)

[fallen slate]

@unkempt rock :x: Your eval job has completed with return code 1.

001 | 0 0
002 | 1 1
003 | 2 2
004 | Traceback (most recent call last):
005 |   File "<string>", line 10, in <module>
006 | ValueError: not enough values to unpack (expected 2, got 0)

[unkempt rock]

Shouldn't the StopIteration happen before it yields and thus the for loop stop before anything weird starts happening

def my_zip2(*iterables):
    iters = [iter(i) for i in iterables]
    while True:
        t = []
        for i in iters:
            try:
                t.append(next(i))
            except StopIteration:
                return
        yield tuple(t)
``` this works but is uglier

[white nexus]

figure y'all might be curious what I was doing.

I have a package that I want to mirror the namespace to a different namespace and have it be the same package be imported in both cases. I finally ended up doing this with importlib and import hooks and messing with sys.modules. It works, which is great.

[dry patio]

https://www.python.org/dev/peps/pep-0479/

maybe this is relevant

somehow in this case the pep doesn't apply, so it comes to the weird error.

the StopIteration is caught by map and it just stops mapping.

[unkempt rock]

aha

Oh that's weird it fails as a generator too yield tuple(next(i) for i in iters)

but works as a list yield tuple([next(i) for i in iters])

I guess that makes sense. The list comp has it's own stop check

[dry patio]

as a generator it should be handled by the pep. transforming the StopIteration into a RuntimeError (so it cannot be caught by tuple to create an empty tuple)

[unkempt rock]

ty for your pep finding wizardry

[dry patio]

found it in this https://stackoverflow.com/questions/67444910/python-cannot-catch-stopiteration-raised-from-generator

because i was searching for a "cleaner way". but i think you already found the cleanest way. (not using next inside a generator)

[unkempt rock]

Wonder if it looks better with the try outside the loop ```py
def my_zip(*iterables):
iters = [*map(iter, iterables)]
try:
while True: yield tuple([next(i) for i in iters])
except StopIteration:
pass

cleanest way to replace something already built in

Wait has that ever happened? Rewriting Python in Python like Rust does

[spice pecan]

Sort of

[spice pecan]

That's what PyPy originally was

Before they introduced jit

[deft pagoda]

add strict kwarg

[elder blade]

The simple answer is no.

[unkempt rock]

Sorry, I don't know what this means

[deft pagoda]

!e

for i, j in zip(range(3), range(6), strict=True):
    print(i, j)

[fallen slate]

@deft pagoda :x: Your eval job has completed with return code 1.

001 | 0 0
002 | 1 1
003 | 2 2
004 | Traceback (most recent call last):
005 |   File "<string>", line 1, in <module>
006 | ValueError: zip() argument 2 is longer than argument 1

[deft pagoda]

@unkempt rock this was added in 3.10

raises an error if the iterables aren't the same length

[unkempt rock]

oh, I thought you were talking about rust

but interesting

[lusty scroll]

you don't have your very own magic pep finder?

[boreal umbra]

What if you have zip in strict mode, but one of your iterator's __next__ method does file IO for some reason?

I can see that causing unexpected behavior pretty quickly. though I'm not sure why someone would do that.

[fallen slate]

:incoming_envelope: :ok_hand: applied mute to @plain solstice until <t:1644096324:f> (9 minutes and 59 seconds) (reason: chars rule: sent 6000 characters in 5s).

[native flame]

Why would it be a problem?

[raven ridge]

It'll consume an extra element that is never returned to you.

[quick snow]

Already the case when you zip without strict and the second item is longer, isn't it?

[verbal escarp]

Anything interesting going on in PEPland?

[grave jolt]

!pep 681

[fallen slate]

**PEP 681 - Data Class Transforms**

Link

Status

Draft

Python-Version

3.11

Created

02-Dec-2021

Type

Standards Track

[verbal escarp]

Did rhettinger already try to shoot it down?🤣

It seems microsoft is heavily invested in typing.. are those eriks part of guidos team?

[native flame]

that looks oddly specific

[verbal escarp]

Well, I just came across https://www.infoworld.com/article/3648539/faster-python-made-easier-with-cythons-pure-python-mode.html which also also benefits from typing improvements greatly

We're talking about speed improvements of 10-100x without extra syntax and only decorators

I'm hoping I can find some time to work in justuse again, I'm dreaming of programmatically decorating packages and running them in a subprocess-interpreter in cython without lots of user intervention

Are there any developments on the subprocess-interpreter front that could help in that venture?

PEP or otherwise?

[grave jolt]

yeah, I have the same feeling

[flat gazelle]

I think starting from the the object struct and the type struct makes the most sense

this header file explains the objects pretty well https://github.com/python/cpython/blob/main/Include/object.h
the type objects are here, but they are a bit less explained
https://github.com/python/cpython/blob/main/Objects/typeobject.c

[fallen slate]

Include/object.h lines 14 to 18

Objects are never allocated statically or on the stack; they must be
accessed through special macros and functions only.  (Type objects are
exceptions to the first rule; the standard types are represented by
statically initialized type objects, although work on type/class unification
for Python 2.2 made it possible to have heap-allocated type objects too).```

[flat gazelle]

static allocation means the bytes of the value are embedded in the compiled binary

yes, like a rust const

or well, to be strict in terms of ISO C, the value is allocated such that it is accessible for the entire duration of the program

[flat gazelle]

this isn't quite right, they get copied out of the binary into ram sometime before main() runs, so that the statically allocated object can be mutable.

[raven ridge]

global variables are statically allocated - there's only one of them, and it is not dynamically allocated, but rather is known to be at a specific location from the very start of the program.

global variables are allocated statically, local variables are allocated on the stack, things that are neither local nor global are allocated on the heap. More or less.

Have you seen the CPython Internals book?

Well, a kindle copy of that book is only $20

the resources here might be helpful, too: https://devguide.python.org/exploring/

there's definitely much more to CPython's internals than could be covered in 30 minutes - what do you want the talk to focus on?

I mean, people have given 30 minute talks on nothing more than how a Python dict works internally...

then you may be more interested in things like how the ceval loop works

[peak spoke]

30 minutes is a surprisingly small amount of time

the interpreter loop

[raven ridge]

https://devguide.python.org/compiler/ explains how the bytecode compiler works

at a high level

and https://tech.blog.aknin.name/category/my-projects/pythons-innards/ explains how the bytecode is evaluated

[white nexus]

always a good day to find an NameError in asyncio 🥲 ```py
Traceback (most recent call last):
File "/home/aru/.pyenv/versions/3.9.9/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/home/aru/.pyenv/versions/3.9.9/lib/python3.9/asyncio/tasks.py", line 775, in _done_callback
if outer.done():
NameError: free variable 'outer' referenced before assignment in enclosing scope

[naive saddle]

damn good job

[white nexus]

ahahaha I figured out what caused it

I passed a dict to asyncio gather

... for something that simple why does it cause such a bad error

hm

[raven ridge]

you should report the bug, if it hasn't already been fixed

looks like it still exists on main

and I don't think it's been reported yet

[paper echo]

it'd be pretty cool if you could pass a dict and get a dict back, though

well it's *args so nvm

i wish asyncio.wait wan't so cumbersome

there's nothing between wait and gather in terms of complexity

also having to pass return_when=asyncio.FIRST_COMPLETED blech

[paper echo]

@overload
def wait_first(
    coroutines: Sequence[Awaitable[V]],
    raise_on_error: bool = False,
    timeout: Optional[float] = None
) -> list[V]: ...

@overload
def wait_first(
    coroutines: Mapping[K, Awaitable[V]],
    raise_on_error: bool = False,
    timeout: Optional[float] = None
) -> dict[K, V]: ...

[white nexus]

.bm

wait how do I report a bug?

[native flame]

bugs.python.org ig

[raven ridge]

Yep, exactly. Register an account, and craft a small reproducer

Seems like it should be pretty trivial to reproduce.

[white nexus]

!e ```py
import asyncio
async def main():
coros = [asyncio.sleep(1), {"a": 1}]
await asyncio.gather(*coros)
asyncio.run(main())

[fallen slate]

@white nexus :x: Your eval job has completed with return code 1.

001 | Exception in callback gather.<locals>._done_callback(<Task cancell...tasks.py:593>>) at /usr/local/lib/python3.10/asyncio/tasks.py:714
002 | handle: <Handle gather.<locals>._done_callback(<Task cancell...tasks.py:593>>) at /usr/local/lib/python3.10/asyncio/tasks.py:714>
003 | Traceback (most recent call last):
004 |   File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
005 |     return loop.run_until_complete(main)
006 |   File "/usr/local/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
007 |     return future.result()
008 |   File "<string>", line 4, in main
009 |   File "/usr/local/lib/python3.10/asyncio/tasks.py", line 775, in gather
010 |     if arg not in arg_to_fut:
011 | TypeError: unhashable type: 'dict'
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/ovatecabox.txt?noredirect

[white nexus]

what's a good name for this issue? NameError in asyncio.gather when passing a invalid type as an arg with multiple awaitables?

i think i answered my question

[raven ridge]

Sounds reasonable to me

[white nexus]

er, how is a codeblock added on bpo, or is there not a rich markup?

[raven ridge]

There isn't

[white nexus]

oka thx

actually

is there a guide or tldr on submitting issues so i do it right?

[raven ridge]

There's is... 1 sec

[white nexus]

ty ty

do ping when you have it since i gotta fix the code that caused this

[livid dove]

oh hey its arl

[raven ridge]

Well, there's https://devguide.python.org/triaging/ but that's more from the pov of the people who will handle the bug report, not the submitter. It at least shows what the different fields are for...

[livid dove]

https://docs.python.org/3/bugs.html#using-the-python-issue-tracker

here is also a small paragraph on issues

[raven ridge]

Ah, perfect

I looked on devguide and bpo, didn't think to check the regular docs 😄

[livid dove]

i always do a quick google search

usually gets me what i need

i'd be screwed if google was down

[white nexus]

i should have just used my own bot 😅

[livid dove]

lol

[raven ridge]

It's also worth noting that it's not a big deal if you have a field wrong or leave it blank or whatever. Whoever triages the bug report can fix it up a bit

[white nexus]

• makes tools to help devs
• does not use own tools

[livid dove]

sounds an awful lot like me

[white nexus]

oh i meant to send the link here oops

https://bugs.python.org/issue46672

[verbal escarp]

I'm wondering why dis.dis only prints but doesn't return anything to iterate over

And a code object returned by compile also is quite opaque

Ah, I guess I need get_instructions..

Funky to see ast and bytecode next to each other

[elder blade]

Oooh, let me see?

[verbal escarp]

Don't have internet on my machine, if I remember tomorrow I can post it

[grave charm]

I am currently working on a flask project which has a package structure
src/_init_.py
src/admin/...
the problem is that I have a global variable named admin at src. ||admin = Admin()||
when importing admin It could mean the sub package or the variable. Another person told me that renaming the variable was the best solution. what would be a good idea to name this?

I coulnd't find any pep8 guide on this

[boreal umbra]

While the python spec isn't purported to be semantically versioned, my understanding is that most of the breaking changes within 3 versions are deprecating or changing the behavior of stdlib content. Are there any breaking grammar changes within 3? The only one I can think of are async and await becoming proper keywords.

[spark magnet]

those are the only breaking syntax changes within 3.x I know about.

[white nexus]

match didn't stop match being used as a variable?

!d match

[fallen slate]

match

8.6. The match statement

New in version 3.10.

The match statement is used for pattern matching. Syntax:

match_stmt   ::=  'match' subject_expr ":" NEWLINE INDENT case_block+ DEDENT
subject_expr ::=  star_named_expression "," star_named_expressions?
                  | named_expression
case_block   ::=  'case' patterns [guard] ":" block
```...

[boreal umbra]

No, that would break tons of regex code, but having it as an allowable name and as a keyword doesn't introduce any ambiguous cases.

[white nexus]

true

[raven ridge]

there've been some minor ones. In Python 3.7 and earlier you could do ```py
print((end)="\n")

In 3.8 and earlier you could do ```py
[x for x in [] if lambda: x]

[elder blade]

The rest is just forwards incompatible yeah

[verbal escarp]

How can I shuffle the nodes of an ast?

(Yes, I know it will often lead to broken code)

Ah, I linearize it, then pick the nodes from the list and transform them via reference on the tree.. hmm

[spark magnet]

wouldn't it work to shuffle the nodes at each level? shuffle each nodes' children?

[verbal escarp]

I fear that'd be too little mutation for actual genetic programming

My idea was to combine two asts at the module level, then shuffle the nodes across the tree ( since I cant guarantee same-length codes for 'sexual' reproduction)

Inspired by how bacteria recombine their circular DNA

[spark magnet]

it will be a wild ride, that's for sure

[verbal escarp]

Thinking about automatically cythonizing 'champion' code for performance

[verbal escarp]

Definitely, but getting the hang of how ast uses the visitor pattern is a bit more involved than I expected

@spark magnet you got some experience with ast transforms?

[true ridge]

@verbal escarp do you mean something like this?

import ast
import random

def recursive_shuffle(tree: ast.Module) -> None:
    for node in ast.walk(tree):
        for field, field_value in ast.iter_fields(node):
            if isinstance(field_value, list):
                random.shuffle(field_value)

tree = ast.parse("""
print(a or b or c or d or e)
try:
    raise Something
except NameError:
    print(1)
    print(2)
except ValueError:
    print(3)
    print(4)
print(a > b > c > d)
""")
recursive_shuffle(tree)

print(ast.unparse(tree))

[verbal escarp]

Hmm.. I'm not sure, I need to run it to tell.. (hope my router gets here today, so finally internet on my pc again) is there a chance that your code swaps the first and last node of the tree?

[true ridge]

!e

import ast
import random

def recursive_shuffle(tree: ast.Module) -> None:
    for node in ast.walk(tree):
        for field, field_value in ast.iter_fields(node):
            if isinstance(field_value, list):
                random.shuffle(field_value)

tree = ast.parse("""
print(a or b or c or d or e)
try:
    raise Something
except NameError:
    print(1)
    print(2)
except ValueError:
    print(3)
    print(4)
print(a > b > c > d)
""")
recursive_shuffle(tree)

print(ast.unparse(tree))

[fallen slate]

@true ridge :white_check_mark: Your eval job has completed with return code 0.

001 | try:
002 |     raise Something
003 | except ValueError:
004 |     print(4)
005 |     print(3)
006 | except NameError:
007 |     print(2)
008 |     print(1)
009 | print(a > c > b > d)
010 | print(e or d or a or c or b)

[verbal escarp]

Very nice, I think that could work indeed

[paper echo]

https://mail.python.org/archives/list/[e-mail]/message/XXWLARRYM2D4R6RHMP5CTQ7WN2WXU6TK/ interesting proposal

[verbal escarp]

Since I haven't really got much exposure to metaclasses, could you explain the ramifications?

[paper echo]

every class now can act like an abstract base class and use @abstractmethod, abstract base classes become vestigial and only used for signaling intent, but don't add functionality

presumably class instantiation becomes a bit slower everywhere? i don't think it's ever going to be accepted btw, would probably involve hacking up cpython internals too much, for very questionable benefit

[elder blade]

Hm? I've never run into the issue where I instantiate an ABC

...so it fails?

Interesting 🤔

[verbal escarp]

@true ridge there are two other mutations I'm thinking of, one is randomly deleting nodes, the other is randomly disabling nodes (possibly with an added if False: ...), how would you realize those?

I've tried implementing them with Transformers, but getting weird errors about missing values in Expr..

[paper echo]

i always hated how it only checked for abstractmethod implementations upon instantiation and not upon class definition

seems like it doesn't really solve the problem that it is meant to solve

but i guess it allows for some level of dynamism

[elder blade]

Mmm.. yeah but you could have an ABC that inherits another ABC

[paper echo]

fair

[elder blade]

Only when you instantiate it do you know that it's meant to be used

[paper echo]

but then you could omit that check as a special case

collect all the abstract methods and only check once you hit a non-ABC in the MRO

should be possible... i think?

[peak spoke]

If you don't implement some abstract method it's just an another abc, how would it know to raise an error?

[true ridge]

Probably with something like this

!e

import ast
import random
from typing import Any, List

random.seed(100)
def remove_or_guard(nodes: List[Any]) -> None:
    if len(nodes) == 0 or any(
        not isinstance(node, ast.stmt)
        for node in nodes
    ):
        return None

    action = random.choice(['remove', 'guard', None])
    if action == 'remove':
        nodes.pop()
    elif action == 'guard':
        index = random.randrange(len(nodes))
        nodes[index] = ast.If(
            test=ast.Constant(False),
            body=[nodes[index]],
            orelse=[]
        )


def recursive_shuffle(tree: ast.Module) -> None:
    for node in ast.walk(tree):
        for field, field_value in ast.iter_fields(node):
            if isinstance(field_value, list):
                random.shuffle(field_value)
                remove_or_guard(field_value)


tree = ast.parse("""
print(a or b or c or d or e)
try:
    raise Something
except NameError:
    print(1)
    print(2)
except ValueError:
    print(3)
    print(4)
print(a > b > c > d)
""")
recursive_shuffle(tree)

print(ast.unparse(tree))

[fallen slate]

@true ridge :white_check_mark: Your eval job has completed with return code 0.

001 | if False:
002 |     print(a > d > b > c)
003 | try:
004 |     raise Something
005 | except NameError:
006 |     if False:
007 |         print(1)
008 |     print(2)
009 | except ValueError:
010 |     print(4)
011 | print(b or e or a or c or d)

[true ridge]

NodeVisitor/NodeTransformer is generally useful for simple stuff; but for modifications like this, I like ast.walk/ast.iter_child_nodes better.

[spark magnet]

@verbal escarp i've done a lot with reading ASTs, but not transforming them.

[ruby willow]

Ah yes the plasmids

Bacterial conjugation ....an exchange of plasmids...maybe represent a whole node with that is a module as that unit ...the unit must be atomic ...no partial transfers perhaps but variation in that unit is allowed as mutations.

https://www.nature.com/scitable/definition/conjugation-prokaryotes-290/#:~:text=Conjugation is the process by,factor%2C or F-factor.

[grave charm]

class Extension:
    admin = Admin()
    bcrypt = Bcrypt()
    db = SQLAlchemy()

    @classmethod
    def init_app(cls, app: Flask):                                                                                             cls.admin.init_app(app)
        cls.bcrypt.init_app(app)
        cls.db.init_app(app)```

I made this to prevent my code from using global vars because someone said to me that using global variables is bad in general, but is this a good idea though?

this is a flask app btw

[verbal escarp]

Thanks! Yeah, I'm starting to see why

[verbal escarp]

Hehe.. yep. Microbiology is fun

[verbal escarp]

Global vars are bad for some things, but not in general. Go with whatever makes your code simpler and easier to read.

[verbal escarp]

Actually, microbio also advised me not to give into the idea of randomly creating nodes but instead create 'junk' by disabling functional parts

[ruby willow]

Yep

That is epigenetics

https://medlineplus.gov/genetics/understanding/howgeneswork/epigenome/

[grave jolt]

!e
Why is this invalid?

def f():
    ((yield)(yield))

[fallen slate]

@grave jolt :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 2
002 |     ((yield)(yield))
003 |       ^^^^^^^^^^^^
004 | SyntaxError: invalid syntax. Perhaps you forgot a comma?

[grave jolt]

!e
But this is valid?

def f():
    ((yield))((yield))

[fallen slate]

@grave jolt :warning: Your eval job has completed with return code 0.

[No output]

[unkempt rock]

I'd assume that one is assuming the second (yeild) is a function call with the argument being a keyword, effectively being an invalid syntax

The others are only made of tuples (yeild) which weirdly enough seem valid

[grave jolt]

No tuples involved

[grave jolt]

But (yield) is a valid expression, why can't I call it 🤔

[unkempt rock]

Right it isn't a tuple

It is just some syntax thing

I think the issue is one parenthesis group is interpreted as a function call and doesn't expect yield to be there

I assume if you look at the DRF it should make sense

[grave jolt]

Ohhh right, (yield) is an expression, but yield is not

So if I want to pass it as an argument, I need to call f((yield))

Thank you

[verbal escarp]

Oh man, reminds me of that one time the other day some guy insisted on asking about the theoretical difference between statement and expression in the main chan

Didn't want to believe that its all about implementation details

[unkempt rock]

aren't there some languages that don't make the distinction between the two

maybe it is lua?

I don't remember

[peak spoke]

how is it implementation details? If you switch out a statement for an expression the allowed syntax changes completely and will no longer be python

[unkempt rock]

their point was totally flawed, yeah

[verbal escarp]

Mine?

[unkempt rock]

nah, the person saying it was an implementation detail

[verbal escarp]

Okay, again. There was a guy in #python-discussion who wanted to know the theoretical definition/difference between expression and statement in order to understand things from the ground up, refusing to dig into the details because the theoretical understanding would be more important

But the actual definition what counts as expression is in the code, not a theoretical thing to understand

What actually counts as Expr or statement is defined in the parser rules, which may or may not follow some external conventions

And since python doesn't have a formal definition, only cpython as reference implementation...

[deft pagoda]

that's not really true, python is independent of its implementation

https://docs.python.org/3/reference/grammar.html

[feral cedar]

yes but that grammar comes directly from cpython

[visual shadow]

im not sure why that matters. i think the fallacy is in assuming that implementation can never determine theoretical differences, but they very well can. What came first may very well be irrelevant, as long as the implementation is determined to be used as a property of the language itself

ultimately, even if implementation came first, if the language specifies that all subsequent implementations "must" follow something, then at that point, it's not a property of just the first implementation, but also of the language.

as such, talking about differences between expression and statement, i think we're all in agreement that saying "theres no theoretical difference between statement and expression" would be grossly incorrect. If that's acceptable, then clearly, such a difference exists.

[rich cradle]

How does Python define expression vs statement? Is it the classic definition of "if it can return something, it's an expression"?

[flat gazelle]

I believe so, expressions have a result, statements don't. (on the grammar level, statements also have an indentation level, which expressions don't)

[rich cradle]

cool cool, thanks

[raven ridge]

An expression can be a statement.

print()

That's both an expression and a statement.

[austere field]

!e
print("Hello world!")

[fallen slate]

@austere field :white_check_mark: Your eval job has completed with return code 0.

Hello world!

[austere field]

!e

while x == x:
  print(x)```

[fallen slate]

@austere field :x: Your eval job has completed with return code 143 (SIGTERM).

001 | wpsi
002 | wpsi
003 | wpsi
004 | wpsi
005 | wpsi
006 | wpsi
007 | wpsi
008 | wpsi
009 | wpsi
010 | wpsi
011 | wpsi
... (truncated - too many lines)

Full output: too long to upload

[austere field]

bot broken

[feral cedar]

?

[verbal escarp]

I'd say via ast. There's a table:

!d ast

[fallen slate]

ast

Source code: Lib/ast.py

The ast module helps Python applications to process trees of the Python abstract syntax grammar. The abstract syntax itself might change with each Python release; this module helps to find out programmatically what the current grammar looks like.

An abstract syntax tree can be generated by passing ast.PyCF_ONLY_AST as a flag to the compile() built-in function, or using the parse() helper provided in this module. The result will be a tree of objects whose classes all inherit from ast.AST. An abstract syntax tree can be compiled into a Python code object using the built-in compile() function.

[rich cradle]

Oh no I didn't mean how does the source differentiate, I just meant how is that differentiation defined

[verbal escarp]

Eh. I didn't mean the source but the docs.. stupid me

[rich cradle]

I'm not sure I understand what you mean

[verbal escarp]

https://docs.python.org/3/library/ast.html#abstract-grammar

There

Although I'm not sure where this abstract definition is actually, formally defined besides this page in the docs

I don't think I've seen it in a pep or some other official thing

It would be nice if this grammar definition actually was official and more pinned down, dunno

[magic nova]

the official thing is the Python Language Reference https://docs.python.org/3/reference/index.html

[grave charm]

https://sadafnoor.me/2016/09/21/how-to-add-flask-admin-to-a-blueprint/
I'm trying to redesign this code because there are hard coded codes there
do I need to copy flask.Blueprints _init_ parameters?

[white nexus]

so, say you were worried about speed (ikr?)
You have an integer, and want to check if its another integer.

Would you use is (checking the memory address is the same) or would you use ==

[native flame]

is wont work outside the -5, 256 range

[verbal escarp]

back on the internet finally \o/

[verbal escarp]

i wouldn't call that "official". who can edit/change it? is there an actual process to approve changes to this document? who cares if there's a mistake (as in, are there real consequences for other python interpreter implementations)?

i think the answer to those questions are "anyone who has access, who cares?", "the process is like 'ey, someone should update the docs sometime', "people who want to quickly check something but don't care about the details, because those check the code on github, so no real consequences"

[flat bear]

peps

[grave jolt]

is just won't work

[white nexus]

okay, assume the numbers are from 1 to 15

[grave jolt]

If your performance needs are so strict that you need to optimize == 15 to is 15, then you're probably doing something wrong: either run the code on PyPy or make a native extension.

[white nexus]

!e
hi = 999
print(hi is 999)

[fallen slate]

@white nexus :white_check_mark: Your eval job has completed with return code 0.

001 | <string>:2: SyntaxWarning: "is" with a literal. Did you mean "=="?
002 | True

[grave jolt]

That's because the 999s are compiled in the same code object

[white nexus]

🤔

[grave jolt]

!e

x = 998
y = x + 1
z = 999
print(y is z)

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

False

[grave jolt]

anyway, you should definitely not rely on CPython-specific details.

[white nexus]

ah, okay

[grave jolt]

!e

for i in range(5):
    print(i)
    try:
        1 / 0
    except ZeroDivisionError:
        raise RuntimeError("nah")
    finally:
        break

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

0

[grave jolt]

WTF!

[next lion]

!e raise RuntimeError("nah")

[fallen slate]

@next lion :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | RuntimeError: nah

[unkempt rock]

Why it didnt raise runtime error lol

[sour thistle]

!e ```py
for i in range(5):
print(i)
try:
1 / 0
except ZeroDivisionError:
raise RuntimeError("nah")
finally:
break

[fallen slate]

@sour thistle :white_check_mark: Your eval job has completed with return code 0.

0

[verbal escarp]

@lusty scroll made me scratch my head once with a finally clause that executed code after returning

[sour thistle]

interesting

but yeah, the break exits the loop before except: triggers

[verbal escarp]

finally can be very confusing

[sour thistle]

!e ```py
def foo():
try:
return 1
finally:
return 2
val = foo()
print(val)

[fallen slate]

@sour thistle :white_check_mark: Your eval job has completed with return code 0.

2

[sour thistle]

damn

[verbal escarp]

yup

please don't use those as interview questions 😉

i'd feel very bad for those interviewees

[radiant garden]

Yes, the subtleties of continuations in python are not very intuitive

[grave jolt]

huh, same in JS?

[native flame]

and java 🥴

jshell> for (int i = 0; i < 5; i++) {
   ...>     System.out.println(i);
   ...>     try {int x = 1/0;}
   ...>     catch (ArithmeticException e) {throw new Exception();}
   ...>     finally {break;}
   ...> }
0

[verbal escarp]

turns out its common behaviour.. only that nobody uses finally? 😄

[fallen slate]

@unkempt rock :white_check_mark: Your eval job has completed with return code 0.

001 | 0
002 | except part
003 | finally part

[radiant garden]

A subtle difference between an exception being raised and it exception taking effect