#internals-and-peps
1 messages ยท Page 155 of 1
probably since the other form (f(a, b for c in d)) is an easy footgun and allowing unparenthesized genexps in expression position only sometimes is a bit messy
This is breaking me a little...what happened here.
also might be confused for unpacking the genexp
Yeah, Python control flow has been raising weird questions for me for a long time. But I just accepted it. Except this one, it finally broke me. Not sure how I can continue now ๐คท
yeah, but it's laughing at me, not with me
!e
def foo():
try:
1/0
finally:
return
foo()```
@lusty scroll :warning: Your eval job has completed with return code 0.
[No output]
ok what
don't need a catch all block :p
somehow the pending exception must get cleared
!e
def foo():
try:
1/0
finally:
try:
1/0
finally:
raise SystemExit(2)
foo()```
@lusty scroll :warning: Your eval job has completed with return code 2.
[No output]
well that's funny, it looked like a successful exit in my terminal
what's next , continue in finally?
looks like continue swallows the exc too
it's funny because in the C API I think python will raise a fatal error if you do certain things with an exception pending
you are supposed to call PyErr_Clear() or PyErr_Occurred() (i think that's what they were called)
so seems that something is "erasing" the exception
what does this statement mean im paranoid that discord would leak my data
Related Companies: We may also share your information with our Related Companies for purposes consistent with this Privacy Policy.
its from discord tos
im trying to understand but can not
as I understand it, anything you give discord (e.g. profile info) could be given to third parties
Idk how I feel about pep 677 rejection.
I think it pretty much solidifies what the future lambda pep syntax would be
as far as I can tell, design decisions for something as big as Python had better be "better safe than sorry"... the rejection notice defends why it was neglected quite well
They should have just never introduced typing
They don't want it becoming its own dsl but it's also bound to be forever cumbersome when they reject stuff like that
link please? im having a hard time finding the rejection notice on the page
ah ty. so the site isnt updated yet
That works since 3.8 (IIRC). It used to not work, but that was a bug that got fixed.
wow, neat
I don't think it works in java but I'm too lazy to check
nevermind
class Test {
public static void main(String... argv) {
for (int i=0; i<10; ++i) {
System.out.println(i);
try {
throw new RuntimeException();
} finally {
continue;
}
}
}
}```
$ java Test
0
1
2
3
4
5
6
7
8
9```
thinking how cursed it could probably be made using generators ...
!e who needs try/finally
class T:
@staticmethod
def __enter__(): pass
@staticmethod
def __exit__(_, _1, _2): return True
with T():
1/0```
@lusty scroll :warning: Your eval job has completed with return code 0.
[No output]
Why is it showing a yellow triangle? ๐ค
There wasnt any output
ah, gotcha
I just learned you can do this
echo 'a = b' | python -m dis```
I think the main reason would be that it's dynamic
"properly" compiling it would be more complex, and probably with not that much gain
generating assembly is complicated, and requires doing several times over to do portably. In contrast, a bytecode VM is generally not slower unless you actually have an optimising compiler, which python doesnt really tend itself towards (these days, you could probably get one working, but 20 years ago, no), while writing a portable bytecode VM is as simple as writing it in C (though python isnt 100% ISO C, it is good enough for most platforms)
yeah and as mentioned languages with a lot of dynamicism and introspection like Python don't really lend themselves to good optimized ahead-of-time compilation
you can't get a lot of information at just compile time, e.g consider what machine code you would generate for this Python snippet
def sum(a, b):
return a + b
vs this C snippet
int sum(int a, int b) {
return a + b;
}
Not only are the types of arguments not known at compile time, even the definitions of functions aren't known at compile time, because they can change after the function was originally defined.
Is there any non-esoteric use case for that?
decorators?
In C, if you wrote, say,
double squared(double x) {
return x*x;
}
double area_of_circle(double radius) {
return 3.1415926535 * squared(radius);
}
``` the compiler is likely to inline one of those function definitions into the other. If you wrote equivalent Python code like ```py
def squared(x):
return x*x
def area_of_circle(radius):
return 3.1415926535 * squared(radius)
``` a compiler couldn't perform the same optimization, because a user of the module could legally do something like: ```py
import mod
mod.squared = print
mod.area_of_circle(1)
Tests. Monkeypatching mocks into code under test is the entire reason unit testing in Python is less painful than in most languages.
Hmm
I believe there are also some stdlib functions that replace their python implementation with an optimised C impl, but keep the python impl for readability
Yep, many of them.
But modification of a module from outside that module is rare except for tests
I mean, even that already breaks function inlining
compile/eval/exec also don't help things
Also, note that in Python, (almost) everything happens at import time.
are there even any dynamically typed languages that compile AOT?
The interpreter imports your main module, your main module imports some other stuff and then runs some stuff, and then your main module ends and Python is done importing it and it starts to tear down the interpreter and join outstanding threads and do a final GC pass.
That is, python myscript.py behaves almost exactly like python -c "import myscript"
Which is pretty weird ๐
Does this a pythonic way?
from tkinter import Tk, Menu, Frame, StringVar, Button, Toplevel, Label, filedialog
Or it's better?
import tkinter
import tkinter as tk would be ideal IMO, but IIRC it's one of the (VERY) few libraries that suggest using from tkinter import *
Also, this question is not really suitable for this channel's topic
ok
I don't like that only subclasses of BaseException can be raised. I feel like it should be based on having a __raise__ method, or something.
Why?
For contrast, C++ allows you to throw any type of object, and it's widely regarded as a huge misfeature
consistency with the rest of the language. the way that other user-defined types interface with the grammar is through dunder methods.
inb4 "all of C++ is a misfeature"
Metaclasses need to derive from type, don't they?
as if I've ever made a metaclass.
I pointed out that it's not inconsistent, though. There's other places where a particular object needs a certain base class to be used in a certain context
shrug it doesn't feel right, but i can't make a reasonable point why it should be different, too early in the morning ^^
As things stand today, you can except: to catch everything, or except Exception: to catch everything except things that should (almost) never be caught. If arbitrary objects were able to be raised, what would you replace except Exception: with to achieve the equivalent behavior?
hmm.. the issue is that we assume a hierarchy, which is only feasibly realizable with subclassing, but if we had something token-based, a set of objects, we could filter based on that
with a token-based filter, it would be possible to add or remove basically keywords
the objects in the set should be something of an enum
that way it could be discoverable
So... What would use code look like, in the case where we want to catch everything except stuff that derives from BaseException and not from Exception?
Taking your set metaphor and running with it, we could get ```py
except ~(BaseException - Exception):
so except Exception excepts exceptions except exceptional exceptions ๐
lmao
hah
I love this and will fist-fight Guido to get this behavior.
guido will win
I lift weights and am decades younger than him, so that's doubtful. In theory I'll be in the same room as him at PyCon, though I haven't decided what I have to do if he wins. And I already wanted to use my fist fight victory wish to get function composition.
I guess that given raise x and except y, there would have to be some dunder method of y that determines if it excepts x.
is it currently just based on an isinstance check?
Yes
For there to be a dunder method on the class that decides if it can accept a particular exception instance, the class would need to have a custom metaclass
Otherwise, you'd need to provide an instance, rather than a type, to except
From a practical perspective, instantiating / manipulating traceback objects manually is a pain (and is only implemented in 3.7+) so user-implemented __raise__ would be bug-prone and annoying to work with.
However, since it can be done manually, you could in theory make some cursed DSL that uses a custom __raise__ dunder to implement structural exception handling.
Please do
i have an ache that might need a pep
i've build https://github.com/amogorkon/fuzzylogic with explicit Rules and it was all nice and dandy for simple examples
then people started to actually use the thing and poked me that there needs to be a way to add more complex rules, like the ones you see in textbooks as tables
so i've tried to really make it work with code, but all i could come up with is this monster
def rule_from_table(table: str, references: dict):
"""Turn a (2D) string table into a Rule of fuzzy sets.
ATTENTION: This will eval() all strings in the table.
This can pose a potential security risk if the table originates from an untrusted source.
Using a table will considerably reduce the amount of required text to describe all rules,
but there are two critical drawbacks: Tables are limited to 2 input variables (2D) and they are strings,
with no IDE support. It is strongly recommended to check the Rule output for consistency.
For example, a trailing "." will result in a SyntaxError when eval()ed.
"""
import io
from itertools import product
import pandas as pd
df = pd.read_table(io.StringIO(table), delim_whitespace=True)
D = {}
for x, y in product(range(len(df.index)), range(len(df.columns))):
D[(eval(df.index[x].strip(), references), eval(df.columns[y].strip(), references))] = eval(
df.iloc[x, y], references
)
return Rule(D)
only to be able to parse things like
table = """
hum.dry hum.wet
temp.cold very(motor.slow) motor.slow
temp.hot motor.fast very(motor.fast)
"""
there has to be a better way!
pyparsing?, I can give it a shot to write a grammar if you want
if it has the same drawbacks as this approach.. :/
or is it possible to make it IDE-able (as in give autocompletion)?
i really can't think of a way to make it feel/work like a table but with valid python syntax so that it doesn't need eval()
i wouldn't mind if the table needs to be in a seperate file to work, as long as an IDE identifies it as valid python for completion
it is true that exceptions are one of the very few things in python that require a base class (metaclasses just use the callable protocol, they dont have to derive from type), the only other example I can think of is changing module types, where it must derive from a module type. But with exceptions, I do think it makes sense, since the exception caries some information about the error that occurred, and IMO it makes more sense to just have that rather than using a dunder and storing it elsewhere
That would also catch SystemExit. The question was how to avoid that while still catching things that should be caught.
@woven crescenthello, we do not allow this kind of request, and discord is a bad platform for such requests regardless. Consider https://www.reddit.com/r/datascience or https://www.reddit.com/r/python instead, reddit works better for finding people
can you execute arbitrary code when you instantiate a namedtuple because it uses eval?
code = f'lambda _cls, {arg_list}: _tuple_new(_cls, ({arg_list}))'
__new__ = eval(code, namespace)
i was trying to think how you'd get it to print something but I couldn't figure out a way
Lib/collections/__init__.py lines 380 to 382
if not name.isidentifier():
raise ValueError('Type names and field names must be valid '
f'identifiers: {name!r}')```
right I ran into that. but people always say eval is unsafe, but I couldn't find even a toy example of exploiting this
if you can add code to the namedtuple call, you can just run bad code, you don't need eval.
otherwise you'd have to find code that used data as the names for the namedtuple? I guess that might exist, but it seems unlikely.
eval is unsafe if it's running in a context that has higher privileges than the user and the user can give it unsafe code to execute
while the second point is probably not true with the check, the first one definitely is not
typically, the problem is eval'ing data
pls meme
We don't have any memechats.
||#the-pydis-mass-layoffs-lounge ๐||
oof.. PTSD inducing
this seems confusing, why doesn't it raise some error?
In [43]: def f():
...: ...
In [44]: copy.copy(f) is copy.copy(f)
Out[44]: True
Hmm, there is no types.FunctionType.__copy__. What does copy.copy do, in that case?
Lib/copy.py lines 107 to 108
def _copy_immutable(x):
return x```
returns the object unmodified
full set of relevant code:
Lib/copy.py lines 105 to 116
_copy_dispatch = d = {}
def _copy_immutable(x):
return x
for t in (type(None), int, float, bool, complex, str, tuple,
bytes, frozenset, type, range, slice, property,
types.BuiltinFunctionType, type(Ellipsis), type(NotImplemented),
types.FunctionType, weakref.ref):
d[t] = _copy_immutable
t = getattr(types, "CodeType", None)
if t is not None:
d[t] = _copy_immutable```
would you also expect an error on copy.copy("foo") or copy.copy(1)?
probably not expect, but I also wouldn't be surprised by it
what about copy.deepcopy([1, 2, 3]) ?
copy being an identity func seems weird
Ah, that's a good point, but I'd say that would be better solved with a flag
Lib/copy.py lines 182 to 183
def _deepcopy_atomic(x, memo):
return x```
`Lib/copy.py` line 197
```py
d[types.FunctionType] = _deepcopy_atomic```
but functions are mutable so I guess the code predates that change?
what would the flag be?
to keep backwards compatibility we'd need the current behavior by default, so you'd want to introduce a new flag called raise_if_immutable=False or something?
and if you set that flag, then a deepcopy of any object that (recursively) contains a string or int or float would fail?
seems... questionable. ๐
I don't know how it actually works but I meant the flag being set to True in deepcopy and false by default otherwise. This of course is not possible now but it feels like it would help prevent errors where the behaviour like the function copying not actually copying is not caught
hm. immutable objects are supposed to be interchangeable - the interpreter is allowed to, as an implementation detail, replace any pair of immutable but equal objects with a single object.
๐ that's uh. fun. i may need to change some code
it does that as an implementation detail for strings and integers and tuples in some places
Well in this case it looks like a mistake with the list not being updated, but I think there could also be cases where they would be passed by accident
it would be allowed to do it in more places if it wanted to. If you've assumed that two equal immutable objects will not be is identical, you've probably made an unsafe assumption.
though that would probably be caught at an another place
since functions are mutable, it probably should make a new function, in the case you originally brought up
I do agree with you there - that seems to be worth a bug report.
now if I could log into the bug tracker
if it hasn't already been ๐
haha.. ya
i wished there was a way to write such a table in python syntax to describe matrices
the numpy people probably would agree
pandas even
What does ... Mean?
Ellipsis. Created for use in slicing, occasionally also used to mean "replace this with actual code later".
and a constant?
can one subclass the ellipsis
!e no you can't.
class MyEllipsis(type(...)):
def __repr__(self):
return repr(super()) * 2
@lusty scroll :x: Your eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 1, in <module>
003 | TypeError: type 'ellipsis' is not an acceptable base type
if (!_PyType_HasFeature(base_i, (1UL << 10))) {
PyErr_Format(PyExc_TypeError,
"type '%.100s' is not an acceptable base type",
base_i->tp_name);
return NULL;
}```
so, basically, if you really wanted to subclass ellipsis, you could set
type(...) 's tp_flags &= 0x7ffffffffffffbff
oh , that flag isn't set on this python. hmm.
well even with tp_flags = 0 it still doesn't work ?

wow
that's incredibly poggers
does that mean that we may be able to do webdev with python instead of the language we don't talk about?
sadly no
I checked with Christian Heimes (I hope I didn't butcher that name) but it's still compiled normally, I don't believe that the Python is converted to wasm.
That is, CPython can run in the browser, but the Python doesn't have access to all the Web APIs.
It's the interpreter that becomes a wasm file.
https://repl.ethanhs.me/ It's how print() still works here afaik.
you can already use python instead of JS using brython
Unless I'm completely misunderstanding the whole process.
I've used brython a bunch in the past
Out of all "Python-programmed web" implementation, I like that one the most
It has a really interesting (not sure if I'd call it pythonic, but it's definitely convenient and unique) approach to DOM interactions and is overall fairly pleasant. IIRC it's not very hard to integrate with frameworks like Vue/React either, and probably even easier to integrate with svelte, but I've tried neither svelte nor calling it from brython
I wonder why this didn't work:
!e
from fishhook import hook
@hook(type)
def __len__(cls):
return 42
for x in list:
print(x)
@grave jolt :x: Your eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 7, in <module>
003 | TypeError: 'type' object is not iterable
Shouldn't this print list[0], list[1] etc.?
besides idle curiousity, is there a point to this? ๐
No ๐
I just thought it'd be a fun abuse of generics
hehe
i'm taking my first dive into brython atm.. somehow getting lost along the way
isn't that the only thing it needs?
it should have that for typehinting right?
The types use __class_getitem__ afaik
try list[0]
Ah, it might be that ot only uses __getitem__ 
I guess it makes sense, since it's a legacy protocol
!e
from fishhook import hook
@hook(type)
def __len__(cls):
return 3
@hook(type)
def __getitem__(self, item):
return self.__class_getitem__(item)
for x in list:
print(x)โ
@quick snow :x: Your eval job has completed with return code 143 (SIGTERM).
001 | list[0]
002 | list[1]
003 | list[2]
004 | list[3]
005 | list[4]
006 | list[5]
007 | list[6]
008 | list[7]
009 | list[8]
010 | list[9]
011 | list[10]
... (truncated - too many lines)
Full output: too long to upload
what is legacy?
__getitem__ ?
nah, legacy iteration
a collection is iterable if it provides only __len__ and __getitem__ - the interpreter will basically do iter(c[i] for i in range(len(c)))
ohh
!e
class OldCollection:
def __init__(self, *items): self._data = items
__len__ = lambda self: self._data.__len__
__getitem__ = lambda self,i: self._data.__getitem__(i)
oc = OldCollection(1, 2, 3, 4, 5)
for _ in oc:
print(_)```
@lusty scroll :white_check_mark: Your eval job has completed with return code 0.
001 | 1
002 | 2
003 | 3
004 | 4
005 | 5
how about that
yeah - "legacy" because it predates __iter__
it was the original way to implement custom iterables, IIUC
it gives a <class 'iterator'> instance
you only need __getitem__, it just takes the IndexError as the stop
aha
seriously?
!e ```py
class OldCollection:
def init(self, *items): self._data = items
getitem = lambda self,i: self._data.getitem(i)
oc = OldCollection(1, 2, 3, 4, 5)
for _ in oc:
print(_)
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
001 | 1
002 | 2
003 | 3
004 | 4
005 | 5
StopIteration is part of the modern iteration protocol, but not the legacy one
so they didn't originally use StopIteration
I don't think so
huh, interesting
There wouldn't have been a need for it before custom iterables
Though I'm just conjecturing. Could be wrong.
Implementing dunder len lets you iterate in reverse ,however, so reversed(OldCollection(1, 2, 3, 4, 5)) wouldn't fail
My mentor used code object in a logger? Where do you folks used it in your day to day life?
!e ```py
def f(x):
return x + 2**111023331924844412
f.code = f.code.replace(co_code=b'|\0d\1\x17\0S\0')
print(f(67))
@rose schooner :white_check_mark: Your eval job has completed with return code 0.
69
You can introspect a code object (directly or via inspect.signature()) to fetch the signature of a function, which is neat if you want to do "smart" behaviours. I'm personally doing it for a bunch of callback functions, to allow you to reorder or omit parameters, but have the calling code figure out the right calls.
Constructing them is useful if you're generating code, and want to do things that aren't syntactically possible or want to try and optimise further.
#bot-commands
yep some
the code type __init__ was not in possession of a signature itself, iirc
Did I break any law?
@lusty scroll my mentor extracted some useful info from code object of traceback to improve the logger.
ah, they'd be handy for that. what was it?
Printing line number, function name, module name, currently executing code and something like that the args values just before that and all.
oh thats so cool. I have tried to do that
there is a package called snoop that kinda does it as well
and birdseye which is kind of a visual version
Cool, good to know. Will check that.
I personally use __code__ objects for mostly "cursed" things
like editing bytecode instrs and what not
I don't think I've ever had to use it directly in real code thanks to inspect
The code in my eval-crashing blog post is for 2.7: https://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html For 3.10, the line in the middle has to be 0,0,0,0,0,0,b"KABOOM",(),(),(),"","",0,b"". For 3.11, it needs 4 more arguments, but when I make it 0,0,0,0,0,0,b"KABOOM",(),(),(),"","","",0,b"",b"",b"",b"",(),() (which I think is right), it doesn't crash....
eval() is tempting, but itโs truly dangerous. Hereโs how common safety attempts can be circumvented.
amusingly enough your bogus bytecode has now become valid, K is now opcode 75 aka RETURN_GENERATOR. In python built with assertions this causes a
>>> c = types.CodeType(0,0,0,0,0,0,b"KABOOM",(),(),(),"","","",0,b"",b"",b"",b"",(),())
>>> eval(c)
python: Objects/genobject.c:917: _Py_MakeCoro: Assertion `coro_flags' failed.
but in the release version that assertion is not there, so it happily flows past here https://github.com/python/cpython/blob/main/Python/ceval.c#L5120
interesting
(I just guessed at another opcode to use, I don't know if any of that was carefully chosen...)
I'm quite curious, what kind of code can crash ast.parse that isn't incredibly long
there shouldn't be any really
ast.literal_eval is marked to be safe to use except for resource constraints
I'm using ast.parse on untrusted user input, basically
so ast.parse (which ast.literal_eval uses) itself should not crash
but the input is hard limited to around 30k characters
you can still crash that with some tactical nesting
a lot of characters fit into 30k
@white nexus what user input are you parsing with ast.parse?
code that's been pasted to a pastebin
yeah that should be fine assuming you avoid cpu/ram consumption issues (cgroups is probably the best way on Linux), we do fuzz ast.literal_eval in CPython and note it to be safe for untrusted inputs
@white nexus can you tell me more? What are you doing with the code?
essentially parsing it for now to see if it is valid python code, and then might add some libcst walking to scan for different sorts of labels and similar. Not entirely sure yet, as I want to make sure that the ast parsing can't it down
but what will you do with it eventually? An obvious possibility would be to run it.
i doubt i'll do that, i'm just curious if i should plan for the possibilty of someone crashing it -- but it seems super safe
ok, what will you do with the code? Why is someone sending you code, just so you can see if it's valid Python? I don't understand the big picture here of what you are building.
same tbh, i just like integrating as many things as i can with other things right now
this phase will probably pass
@white nexus I can tell you from experience that it does not ๐
>>> re.escape(" ")
'\\ '
just why
https://bugs.python.org/issue46707 sort of relevant ๐
ooh fun! I can continue integrating different services with everything known to humans
then move on to AI
Assume that bug would apply to ast.literal_eval too since it's with the parser
hmm, fun
so that's basically blocking any ast.parse
or well
any ast parse on 3.10 in an async context needs to be run in an executor
If your regex is in VERBOSE mode, you have to escape the spaces to preserve them.
i looked thru the code already, thanks
yup, i learned something too.
does anyone know if there's a plan in the future to make assert a function as happened to print?
below pep is draft but implies maybe so
!pep 679
no it won't become a function
the pep describes py assert (expr, "message") becoming the same as py assert expr, "message" to support this ```py
assert (
some_very_long_expr_that_has_to_be +
broken,
"some very long message that has to"
"be broken",
)
yeah, requiring the parens would break loooots of tests. can see why they wouldn't do it
Part of me doesn't want this, just to inconvenience those who treat assert as a function. But other than that I'm for it.
!e
assert (2 == 4, "hi")
print("bye")
@boreal umbra :white_check_mark: Your eval job has completed with return code 0.
001 | <string>:1: SyntaxWarning: assertion is always true, perhaps remove parentheses?
002 | bye
Remove the syntax warning ๐
!e assert(2 == 4, "hi")
@white nexus :white_check_mark: Your eval job has completed with return code 0.
<string>:1: SyntaxWarning: assertion is always true, perhaps remove parentheses?
interesting, i didn't know the space was not required
if parentheses became valid, how would you assert a tuple
not that there's ever a reason to do that
but seems odd to have no syntax to express it at all 
assert a # a is tuple - old behavior
assert a,b - old behavior
assert (a,b) - new behavior (there are no builded tuples, it works same as old form without pars)
assert ((a, b))?
hmm
@dusk comet :x: Your eval job has completed with return code 1.
001 | <string>:2: SyntaxWarning: assertion is always true, perhaps remove parentheses?
002 | <string>:3: SyntaxWarning: assertion is always true, perhaps remove parentheses?
003 | Traceback (most recent call last):
004 | File "<string>", line 1, in <module>
005 | AssertionError
!d help
help([object])```
Invoke the built-in help system. (This function is intended for interactive use.) If no argument is given, the interactive help system starts on the interpreter console. If the argument is a string, then the string is looked up as the name of a module, function, class, method, keyword, or documentation topic, and a help page is printed on the console. If the argument is any other kind of object, a help page on the object is generated.
Note that if a slash(/) appears in the parameter list of a function when invoking [`help()`](https://docs.python.org/3/library/functions.html#help "help"), it means that the parameters prior to the slash are positional-only. For more info, see [the FAQ entry on positional-only parameters](https://docs.python.org/3/faq/programming.html#faq-positional-only-arguments).
This function is added to the built-in namespace by the [`site`](https://docs.python.org/3/library/site.html#module-site "site: Module responsible for site-specific configuration.") module.
is there a trick on how to make help() spit out specific docs when envoked on certain objects?
I'm pretty sure it just yoinks the docstring, you may have success by adding a "__doc__" attribute
!e ```py
from site import help
class Example: pass
instance = Example()
instance.doc = "Hey!"
help(instance)
ah, right, hold on
@spice pecan :x: Your eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 1, in <module>
003 | ImportError: cannot import name 'help' from 'site' (/usr/local/lib/python3.10/site.py)
I can never remember the correct way to import it, smh
it's
That's one way to do it, yeah
!e now let's see how wrong my assumption was
from pydoc import help
class Example: pass
instance = Example()
instance.__doc__ = "Hey!"
help(instance) โ
@spice pecan :white_check_mark: Your eval job has completed with return code 0.
001 | Help on Example in module __main__:
002 |
003 | <__main__.Example object>
004 | Hey!
What do the parameters _ArrayOrScalarCommon, Generic[_ShapeType, _DType_co] of np.ndarray mean in context to Type hints.
I found this unanswered stackoverflow post https://stackoverflow.com/questions/70229215/type-parms-of-np-ndarray
and the PEP 484 docs which are pretty intimidating.
If someone has a simpler explanation I would be grateful!
it's kind of documented here https://numpy.org/devdocs/reference/typing.html#numpy.typing.NDArray
otherwise i think i saw (maybe participated in?) a github issue about it where they said "it's WIP and pending pep 646"
!pep 646
Not sure if this is the place, but what is the first instinct (or the natural answer) of resulting thing when you face
>>> list(enumerate(['a', 'b', 'c'], [1, 2, 3]))
?
[(0, ['a', 'b', 'c']), (1, [1, 2, 3])](i.e. treat as alist)[(0, ('a', 1)), (1, ('b', 2)), (2, ('c', 3))](i.e. treat as azip)[((0, 0), ('a', 1)), ((0, 1), ('a', 2)), ((0, 2), ('a', 3)), ..., ((2, 1), ('c', 2)), ((2, 2), ('c', 3))](i.e. treat as a cartesian product)
Just a random idea. 3 seems most useful since you can just loop with both indexes... but I'm not sure which is the most natural one
Thanks!
That is much more forgiving! ๐
also btw we have #type-hinting now
Perfect!
I thought it was related to PEP 484 so asked.
Shall ask there from the next time!
Thanks again!
I want to level up my programing skills, can anyone give some reference work or some problem set?
is cpython's dict lookup implementation intentionally not vectorized or is it just that nobody's gotten around to it yet
afaik no one has looked into it (or at least not posted about it on the bug tracker or mailing lists), I'm sure if it's done portably and has a good speedup vs complexity benefit it would be accepted
Also see this: https://github.com/faster-cpython/ideas/discussions/219 for a bunch of experiments regarding the dict object's implementation
..again? ๐
Part of the problem might be that vectorising requires non-standard C code?
How so
Well compilers can optimise by using them, but there's no operator/standard C function for the necessary instructions. Could be done by writing individual implementations for different instruction sets with a C fallback, but would be a bit of a pain.
My understanding of vectorization is making sure to be able to write the loops such that the compiler can unroll then in some way to take advantage of the fact that say the target processor has 4 ALUs in parallel
i don't think compilers are very good at vectorizing array indexing, though
so you would need something like #include <immintrin.h>
How you write like this
what's the main difference between a package structure and a module structure?
and when should I choose which?
!code
Here's how to format Python code on Discord:
```py
print('Hello world!')
```
These are backticks, not quotes. Check this out if you can't find the backtick key.
Thx
what exactly does it do?
AClass.from_class()``` ?
@classmethod
def from_class(cls, config):
return cls(**{name: value if not name.startswith('__') for name, value in config.__dict__.items })
gib
would it be possiblle that an object with __slots__ gets passed in
that wouldn't have instance dict, that is
I don't know that yet 
ok
I always like inspect.getmembers
it returns tuples of key, value
so you dont even need items()
and it gets attributes from base classes and stuff
not sure if that makes sense
is config a dict or another regular object with attributes?
class Config:
owo = "smth"
Yep
is config an instance of that, or the class itself
Class itself
ok
nevermind what i saud abt instance dict
and you will always have __dict__ on type objects
so that's not a concern
the one question would be, do ou want attributes from base classes or not
Wdym?
will this classmethod be returning a newly geberated class
Yes
dayum
Yep
ok
Greyblue do be writing a thesis
!e This is my interpretation of it - I wanted a nice repr so I thought of using namedtuple
from collections import namedtuple
import builtins
class Config:
owo = "Smith"
other = 113
class NewClass:
pass
def from_class(cls: type, config: type):
o = cls()
orig = set(o.__dict__)
o.__dict__ |= vars(config)
diff = { k:vars(o)[k] for k in set(vars(o)) - orig if k[0:2] != k[-2:] != "__" }
ntc = namedtuple(cls.__name__, diff.keys())
return ntc(**diff)
print(from_class(NewClass, Config))```
@lusty scroll :white_check_mark: Your eval job has completed with return code 0.
NewClass(owo='Smith', other=113)
This is actually what I was thinking of originally:
def from_class(cls, config):
o = cls()
o.__dict__ |= vars(config)
return o```
what I think is strange about this is that config is a type, not an instance
I could see either an object (for type conversion/auto-mapping) or a dict (like from json)
or maybe types.SimpleNamespace
I wanted to make the namedtuple inherit the NewClass but couldn't see how easily and got tired of tweaking it
Seems like a lot of work, just use types.new_class
is this a new github feature?
somehow cpython made a comment in my name...? https://github.com/python/issues-test-demo-20220218/issues/46672
okay so i think they're testing some new features....
as the first issue isn't actually #1
apparently mannequins have been around for years
yeah
Not assigned to
CPython bugs is moving to GitHub
yeah, i was more talking about the tools on github that are allowing that
since that repo where they're testing it out started on a high issue number
They have probably skipped numbers so that it matches the ID on bugs.python
& that i commented on an issue without actually commenting
yeah, probably something either that github did for them or a tool somewhere for migrators
that's weird, it is written for 3.9 but still a draft
Does anyone know why contextlib.suppress does not error if you don't specify any Exceptions to suppress? Do you think it should? I'm trying to think of a valid reason why it does this bahavior?
cooper@home1:~$ python3.11
Python 3.11.0a5+ (main, Feb 21 2022, 08:52:10) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import contextlib
>>> with contextlib.suppress():
... print("Foo")
...
Foo
>>>
Just wondering and opened: https://bugs.python.org/issue46819
could be useful if you're collecting exception types from somewhere dynamically I guess, though that doesn't seem particularly common
Tbh. the current behaviour seems reasonable. Literally contextlib.suppress() might be pointless, but contextlib.suppress(*exceptions) with potentially empty exceptions isn't.
yeah, just like sum([]) is 0
I'd be more interested in does it suppress if nothing is provided
!e
import contextlib
with contextlib.suppress():
1 / 0
@grave jolt :x: Your eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 4, in <module>
003 | ZeroDivisionError: division by zero
as expected, it suppresses all the exceptions you pass in ๐
Just like with sum, all and any, this bring about some nice properties, like these being equivalent ```py
with contextlib.suppress(*a):
with contextlib.suppress(*b):
...
with contextlib.suppress(*a, *b):
...
well, not exactly equal
if you put code where all of these ... were they they would be very different ```py
with contextlib.suppress(*a):
...
with contextlib.suppress(*b):
...
...
Of course
That's why the code doesn't have anything in there ๐
In [1]: nums = list(range(10_000_000))
In [2]: arr = np.array(nums)
In [3]: %timeit sum(nums)
128 ms ยฑ 155 ยตs per loop (mean ยฑ std. dev. of 7 runs, 10 loops each)
In [4]: %timeit arr.sum()
2.39 ms ยฑ 3.11 ยตs per loop (mean ยฑ std. dev. of 7 runs, 100 loops each)
I assume one of the reasons numpy can do this 53 times faster is that it doesn't have to account for heterogeny, whereas I assume Python has to repeatedly do type checking to make sure x.__add__(y) or y.__radd__(x) works (and to access those methods in the first place). Could cpython optimize builtin functions/methods that involve lists if the c struct for the list had a is_homogenous flag with optimized behavior for homogenous lists of primitive-like types?
Also, some potentially interesting supplemental information:
In [7]: s = pd.Series(arr)
In [9]: %timeit s.sum()
4.65 ms ยฑ 5.29 ยตs per loop (mean ยฑ std. dev. of 7 runs, 100 loops each)
In [10]: s2 = s.astype(object)
In [11]: %timeit s2.sum()
312 ms ยฑ 1.12 ms per loop (mean ยฑ std. dev. of 7 runs, 1 loop each)
it's more than just homogeneity vs heterogeneity. On top of that, the numpy array is contiguous and the Python list isn't (or, really, it's a contiguous array of pointers to arbitrary non-contiguous objects), and the numpy sum() accumulates using a (fixed precision) C int, while the Python sum() accumulates using an (arbitrary precision) Python int.
I suspect, of those effects, accumulating into a C int is by far the most impactful in terms of the performance.
that is, you think accumulating into a Python int slows sum down even more than the list elements being non-contiguous in memory?
also, when you say a fixed precision int, do you mean size?
yes
well, yes - both size and precision of a C int are fixed, both size and precision of a Python int are variable
ints are discrete, yes? how can the concept of precision apply?
Python ints have variable range, C ints have a fixed range
but isn't that just a matter of how many bits the int is allowed to take up? I don't think that's necessarily the same as precision
not just how many bits it can take up, but also what those bits represent. Sure, maybe "precision" isn't the best word.
I've always assumed that
for line in file:
...
is mostly a shortcut for
while line := file.readline():
...
but that doesn't seem to be the case as I just got
self._last_file_pos = journal_file.tell()
OSError: telling position disabled by next() call
where the tell was under a loop over a file. What is it doing that this has to be diabled?
Here's where the flag is set: https://github.com/python/cpython/blob/cff4d5c5d29528299ec1ac5b3b3a6f7735577c01/Lib/_pyio.py#L2553
It looks like it's an optimisation - since chunks are being fetched from the binary file, there's going to be fragments being stored to be returned in the next read call. If you use readline() I think it then takes extra work to keep track of how much was used of the binary data so tell() can rewind appropriately, but if next() is called it assumes you're going through the file continuously so it can skip figuring that out.
Lib/_pyio.py line 2553
self._telling = False```
You can always use iter(file.readline, '') to force calls to readline instead.
pep680 doesn't sit right with me, currently mostly because of the unintuitive name and not being able to write. It feels like a pep sized band-aid.
Generally:
- It is needed to be in the stdlib
- it fulfills the bare minimum
- I take what I can get but it feels really defensive and the way it is now done does worry a bit for the future (only implementing the minimum really late, because it is already aimed at
nobody will want to maintain it)
with the name of the channel in mind, what are your general thoughts? I know it is done and accepted, but it seems like a nice starting point for a discussion.
!pep 680
i agree that it should be in the stdlib
i also think that they should have done it long ago??
Any existing third-party module named tomllib will break, as import tomllib will import the standard library module.
doesn't the import system check for third parties before stdlib?
It's pretty weird that we standardized on pyproject.toml as the main config file for developing, maintaining, and building Python libraries and applications before we had any way to parse TOML in the stdlib, so I agree it's overdue.
And since TOML files are generally written by hand but parsed by machine, I don't mind that it's read-only for now. Targeting a minimal viable surface also makes sense, since the stdlib is where modules go to die
setuptools is way behind though, but they are making strides
Targeting a minimal viable surface also makes sense, since the stdlib is where modules go to die
wdym
new releases get tied to python's release schedule
Only one feature release per year, a much higher barrier of entry for contribution than the average open source project, and extremely strict backwards compatibility requirements really stymie the evolution of modules once they become part of the standard library
That's part of the reason why numpy is not in stdlib, despite being an essential package
The fact that a backwards incompatible change to the standard library generally takes 3 years, for instance, means that even when something is recognized as a mistake, it often doesn't get fixed, because fixing it would often cost more (in terms of time, effort, and developer goodwill) than leaving the mistake around
ah, that makes sense
I still don't understand why they went with toml
I guess its bc y'all has security issues and json is tough to write by hand
Ini doesn't provide enough features
yaml also has a bit too much going on for something like pyproject imo
I really like toml for what it is as an advancement for ini, as ini is mostly useless with requiring some custom parsing for basic structures
Yeah but the syntax is the best imo
A list of dict looks atrocious to write in toml
I think there was some issue for making that better, but you can use inline tables in most cases where you would need a structure like that
What's the norm for caching? Is using a dict fine
it depends on what you are caching. For caching function calls, a dict or just functools.cache is fine. But for caching web requests, you may want something more elaborate, e.g. redis
yaml has a lot of complexity. If you look at the spec, it'is quite monolithic
Yeah yamk can be a pain, toml is ini with extra, but the resemblance to toml can make easier to understand.
Yaml can get quite weird and is relatively alien
yaml is nicer and easier to parse
unless toml got way more complex in recent revisions, yaml is definitely the more complex format here. 8 different variants of multiline strings, variable substitution, etc.
I am not saying that complexity isn't sometimes useful, but for describing python projects... toml is more than adequate
What do I need to know to be able to be an expert in python
At what point is someone actually an expert in anything? In either case, Fluent Python by Luciano Ramalho is a great book if you want to get an in-depth understanding of the language.
Generally, I'd say that understanding things like dunder methods and the method resolution order show an "advanced" understanding of the language.
Just want to get knowledge for in-depth python and possibly how is it built
the source code is online: https://github.com/python/cpython
the book "cpython internals" is good
building a language is a complex social, technical and economical process ๐
How does import_module()'s relative importing work?
I have a file, and I want to import a sibling next to it (from .friend import ...), but import_module() (and more specifically resolve_name()) seems to producing current_file.friend when I use ('.friend', __name__) for the arguments
I want to produce friend because in this case, the "main" point where absolute imports base around is the same folder.
shouldn't you do something like import_module('.friend', __package__) since you need to pass a package to import from
Lib/importlib/_bootstrap.py lines 902 to 908
def _resolve_name(name, package, level):
"""Resolve a relative module name to an absolute one."""
bits = package.rsplit('.', level - 1)
if len(bits) < level:
raise ImportError('attempted relative import beyond top-level package')
base = bits[0]
return '{}.{}'.format(base, name) if name else base```
^^^ can 200% confirm
I have the book and it is awesome
i think the best way to use it is by switching the cwd
import always checks the cwd first, then goes through sys.path etc
so if you know where the thing lives you want to import, do ```py
orig_cwd = Path.cwd()
os.chdir(installation_path)
try:
importlib.import_module(module_name)
except ImportError:
pass
finally:
os.chdir(orig_cwd)
that's the safest approach
That wouldn't be thread safe...
put it behind a lock if you must :p
it's the approach that the import system was about to take but then they introduced the sys.path hack
i've seen code comments in the source that lamented this
in justuse we do this consistently without manipulating sys.path, so i know it works
what underlying methods do all of the weird shortcut operators call?
for example:
a |= b
~c
d | f
I've not seen |= much before, but extremely curious how to abuse use it in practice
i.. means inplace
what is neg?
d | f is just or
isn't that a bitwise or?
is there a place where these are all documented, which symbol to what method?
the page i linked
or cpython source
none of those operations are bitwise per se, it depends on the implementation. the operations you can't change are not, and, or
ive used it for dict update
oh I thought you could change |
ah, yeah
True or False
but thankfully I already love or, and, not
they're already overpowered :)
and is so useful for protecting attribute access against a possibly nullable attribute
i tried to overwrite or, and, not when i tried to implement fuzzy logic by overloading booleans.. >.>
:-:
well, i had to settle for ~hot & ~cold, but that's not too bad either
no, it's not
but it's good to know there is a big difference between those symbols and the keywords, even though they are called the same colloquially
dunders are very powerful and should be taught fairly early imo
well, I know most dunders (I think) , just didn't realise that there was a <<= and other related in place edits
well, if you've never seen them, you also don't get any ideas on how to (ab)use them ^^
oh I already have the ideas :3
which is a shame, since they can provide a very nice intuitive API if applied sparingly
sparingly
and now it's time to go
essentially there's a bunch of bitmasks that I need to represent, and using some of these operators would make it a lot easier
couldn't you use numpy arrays for that?
ehh, can't introduce numpy as a dependency
too bad
right now the current system has a function definition for each flag (why??) and a decorator to collect them
idk how much python has progressed in the past 7 years but it was implemented on 3.6
so not sure
maybe it was implemented before then 
it uses a custom enum class because the built in enum class was slower lol
the decorators and such I was talking about earlier are in the flags file
need to test this ๐
i wonder if this was all worth the effort though
those timings are in the ns range
given how this library serializes possibly hundreds of items to an enum per second, it is, it is really worth it
i take your word for it
I haven't tested it recently against the stdlib but I think it's probably still faster
probably faster because it doesn't have to handle all the use cases of Enum
but what if we made the enum.Enum object C-handled ๐ค
although it would be so nice if IntFlag existed *in the custom impl
I thought it did exist for a while lol
doesn't it already?
wait
oh k
probably will make the custom thing slower
yeah :/
you can do that easily
class EInt(IntEnum):
a = 2 ** 0
b = 2 ** 1
c = 2 ** 2
d = 2 ** 3
bool(EInt.d & 7)
lol I didn't even realize they were the same
I was thinking about what was 2 ** 1
and then realised that's the same as 1 << 1
although 1 << 1 would likely be faster
faster? the only speed that matters is the lookup and &
you don't rebuild your enum every nanosecond (hopefully)
ahh
hehe
EInt.d and 7 is faster, nice
300
483.858
313.94142993274124
testEInt2
300
399.066
167.24355697028653
min, mean, stdev
def testEInt1():
return bool(EInt.d & 7)
def testEInt2():
return bool(EInt.d and 7)
using perf_counter
now let's see how your enum performs
oh true 
although given that a lot of these enums are used with try_enum which returns a proxy value if it doesn't exist... hm
your enum is faster, indeed
testD1
100
201.619
266.19238798822596
testD2
200
248.104
239.16056774010974
testD3
100
217.641
289.94540141612526
def testD1():
return D.a
def testD2():
return D["a"]
def testD3():
return D.a.value
about 3 times
smh don't give me credit for it ๐ญ
not too shabby
someone else wrote it who now has stepped away from maintaining the project
testDict
100
193.089
215.1669143874366
that's a pure dict lookup
still faster ^^
but the dot syntax is nicer
I think those would be compile-time optimized anyway
i didn't include the building of those in the timing tests
but i do wonder if numba could speed up these lookups..
hmm
hah a compiler optimization in python
dont see those too often
hmm, if numba speeds it up I could probably add it as an optional dependency
constant folding is one of the few that we have :^)
constant folding?
And also turning sets into frozensets in certain contexts (var in {1, 2, 3})
Folding 1 + 3 into 4 at bytecode compilation
oos. what happened there o.O
Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
yeah also converts lists to tuples
frozenset is an immutable variation of a set; if you have something like if var in {1, 2, 3}, instead of building the set every time, it caches a frozenset beforehand (I think? I haven't touched this one in a whiiile)
Numba really doesn't like things other than ints and floats tbh
what I'm curious about is the best way to make a custom flag thingyadoodle
It's in the name
but it does work with &
woah
holy crap, that's.. wtf
@njit
def testEInt1_jit():
return bool(EInt.d & 7)
testEInt1_jit
100
3606.791
1082232.7020728937
numba hard at work
Where I need to be able to manipulate a bitmask and have multiple attributes, there's gotta be a good way to do this
There's no "best" way IMO, depending on your needs you may want either just a bunch of int constants, or an enum.IntFlag, or a bunch of non-python ints from your target library
1 << 0..howmanyyouwant
non python ints from target library?
but the min is down to the hand-optimized version of @white nexus
smh stop giving me credit, I didn't write that
you posted it

Python's ints are not native, they're what other languages call bigints
i don't care who actually wrote it :p
and you have "pls mention" in your name ๐
Some libraries operate with native ints, and they sometimes provide ways to avoid converting from python to native and vice-versa in certain contexts
aha, median seems to give a better idea what's going o
median of the jited version is at 200
ah, this enum usecase is mostly converting bitmasks into flag instances using enums ><
it hits 100 30527 times, so it's not accidental
the minimum timing
that's about 1/3
curious
your optimized enum only hits the minimum 24386 times, which is lower
the median is the same at 200
I think one of the optimal ways to solve this problem might be to define an enum class to define the attributes and then have a flag class which collects the enum entries and deseralises it that way for each place that needs a bitmask
All in all, that's
- custom enum class
- custom flag class
- actual enum -- one for each bitmask
- actual flag class -- one for each bitmask
so i'd say give numba the prize
!pypi numba
what changes are made to use this?
@njit
def testEInt1_jit():
return bool(EInt.d & 7)
just @njit
i don't know how to incorporate it in your code effectively, but it's something to consider
and since decorators only run once... if numba doesn't exist then I could replace that decorator with a lambda to return the function unmodified with virtually no slowdown
you could
the only delay would be when first importing the module
and the first execution of the code, yes
ye
.bm speeeeeeed
the stdev is extremely high with numba, but only the first couple times
so as it is used more and more it'd be faster
?
yeah, make sure to use the median for timing tests, mean doesn't tell you the whole story in this case
i don't think timeit does account for these cases
yeah, I'll have to look more into it
if numbs does provide a speed benefit, I'll maybe add it to the optional installs and add internal support
how does this sound?
here is the code i used to test this https://pastebin.com/XVCqrUfr
Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
feel free to play with it
dunno, you need to consider ease of use and performance.. i really like Enum, but flags bit me in the butt before, so IntEnum is my go-to these days
i even used numpy for bitmasks before, but yeah..
the current implementation is here, prepare to weep
what docstring format is that?
it's used with an instance, and each of those methods is actually a property
numpy w/sphinx iirc
never seen .. describe before
sphinx directive
(at least not consciously)
although it may be a custom extension, not sure
ahh.. proper docstrings, what luxury ๐
documenting stuff is such a pain
I definitely didn't deprecate and replace several attributes on an enum only to be told after it was merged that I didn't update the docs :^)
of course not. who would do that..
just delete the docs ๐
@flag_value
def manage_events(self) -> int:
""":class:`bool`: Returns ``True`` if a user can manage guild events.
.. versionadded:: 2.0
"""
return 1 << 33
that's a little verbose imo
I'm under the idea that method shouldn't exist
also, versionadded? isn't that what a commit history is for? 
no, it's used in the docs
sphinx adds that when it creates the documentation automatically
ah..
well, it doesn't add that, but it handles it specially since it's a directive
it should be something like a dict of the name to the bit number that represents it
oh well. it seems like a good job for newbies to take care of ^^
hi blue, tldr trying to optimize enums, flags, and permissions :^)
Thank you, I forgot about __package__ ๐
have you implemented those on wumpy yet
did you implement them performance-consciously?
Yeah, because I am planning to change it to Cython as soon as I can use it for other reasons as well (there's a few deal-breaking things that I need fixed before I can use it really). I don't see how I could make it more efficient really ๐ค
https://github.com/wumpyproject/wumpy/blob/main/library/wumpy-models/wumpy/models/flags.py#L11-L83
It's the same thing as disnake's fields though
Albeit easier to read. Permissions are also more efficient though
.bm ๐
only recently i've did a little test to compare numba against cython
numba beat cython in my test performance-wise
I store two permissions for permission overwrite: https://github.com/wumpyproject/wumpy/blob/main/library/wumpy-models/wumpy/models/permissions.py#L354-L382
As opposed to a dictionary mapping that the user inputs. It's quite a different API though.
I can imagine that, Numba can do pretty awesome things. I am only planning to use Cython for the memory improvements. For example the bitfield above I managed to get down to the pure 32-bit integer + the PyObject headers.
Which is the minimum theoretically possible
i really wasn't expecting numba to be faster than cython, tbh, it's pretty crazy.. but it's also very limited and fiddly
in my time trial earlier it couldn't work with and but only with &
so if you tried to optimize your code using and instead of & before and then get the idea "oh, i could use numba to further speed it up" .. ๐
and timeit doesn't account for things speeding up over time.. it really should include some plotting
best matplotlib with xkcd style ^^
Haha true
๐ฆ
I think that makes some sense, since and involves calling __bool__ on both operands and returning one of them and & is an extremely basic bitwise operation
Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
it tries to overload and it seems
Ooh, interesting
Is there documentation somewhere on who maintains which standard library modules? Or at least a list of unmaintained ones? (Exceeding PEP 594)
I want to share this beautiful code I found in parser.c in block_rule
asdl_stmt_seq* a;
Token * dedent_var;
Token * indent_var;
Token * newline_var;
if ((newline_var = _PyPegen_expect_token(p, NEWLINE))
&& (indent_var = _PyPegen_expect_token(p, INDENT))
&& (a = statements_rule(p)) // statements
&& (dedent_var = _PyPegen_expect_token(p, DEDENT)))
{
_res = a;
goto done;
}```
that's 4 variable assignments in one if condition
in the spirit of the walrus operator
my attempt to pythonize it
with cleanup():
if (
newline_var := p.expect_token(Token.NEWLINE)
and indent_var := p.expect_token(Token.INDENT)
and a := statements_rule(p)
and dedent_var := p.expect_token(Token.DEDENT)
):
return a
You would need to put any user code that opens files or executes subprocesses behind the same lock. You can get away with that if you're writing an application, but certainly not if you're writing a library
isn't EInt.d & 7 entirely different than EInt.d and 7? Isn't the former checking whether the 3 least significant bits are set, andt he latter checking whether any bits are set?
EInt.d and 7 is really just bool(EInt.d) (or EInt.d != 0)
@verbal escarp ha someone redid the enums (this was in progress for a while) and they're even faster now
How did they get "faster"?
I know you are supposed to used literals for matching on dicts, but isnt it weird that you can't match against dict on structural pattern matching?
!e
class A:
def __init__(self, x): self.x = x
match A(x="foo"):
case A(x=m):
print(m)
match dict(x="bar"):
case dict(x=m):
print(m)
@next lion :white_check_mark: Your eval job has completed with return code 0.
foo
it looks weird, but dict(x=m) is a class pattern
!e
match dict(x="bar"):
case dict(keys=hmm):
print(hmm)
@grave jolt :white_check_mark: Your eval job has completed with return code 0.
<built-in method keys of dict object at 0x7f54acbd2740>
Oh right, it is bultin so it behaves like the other class patterns. That does indeed look weird
very common in the auto-generated parser
so i don't know but that example is small compared to something like a function definition
oh that code was autogenerated
yes python has stuff in Tools/peg_generator/ that does that
that's why there are spaces in stuff like this ```c
result = _PyAST_x( x , y , z , 5 );
I guess my credit goes to the writers of the PEG generator then ๐
the :r part?
regardless, check out https://docs.python.org/3/library/string.html#format-string-syntax
yes
I think you meant !r which would be a "conversion flag"
what about the :smth here?
also could spaces affect python space in any way?
def f():
smth()
__
smth()
the _ represents the spaces
spaces on a blank line are ignored.
something after a : in a format string, like :x, is a "format specifier".
Thank you;
Hol' up I didn't know that the object reference count was a signed integer?
The Immortal Objects PEP seems to imply that since worst-case apparently the reference count can become negative?
It's signed so you can decref to negative numbers for dead objects.
Probably to make it easier to check for dead objects, and so you can't overflow back to some absurdly high count.
Ah, also in debug mode it checks if it goes negative and logs that.
https://github.com/python/cpython/blob/main/Include/object.h#L505-L509
Include/object.h lines 505 to 509
#ifdef Py_REF_DEBUG
if (op->ob_refcnt < 0) {
_Py_NegativeRefcount(filename, lineno, op);
}
#endif```
i mean isn't length also a signed integer when it "isn't" supposed to be
Hey internals people!
I have an internal-to-python-adjacent question. Not on python per say, but I think this channel is the best place to ask it
I'm doing a report for school about bad syntax, and why the various excuses for bad syntax in various languages aren't good enough
I'm wondering if anyone can point me to any research regarding the effect that expressive, pleasant syntax has on workflow, and or, the well being of the people who use it
Or even about the study of language design from a human-centric perspective
what do you mean by "bad syntax"? also this might be better in #software-architecture
i think programming language ergonomics is a hotly debated topic and generally an "unsolved problem"
and it will vary a lot by person and by the problem at hand
some people think haskell is the most comfortable language, other people think python is, other people think kotlin is, other people think scheme is
syntax is also kind of intertwined with data models, type systems, and standard library availability
as well as language features like macros, compiled vs interpreted, statically vs dynamically typed, etc.
as well as considering that there are many "lineages" of programming and that sometimes things just come down to preference & what people are comfortable with
Un a word โ exclusionary. Does not communicate meaning implicitly and, therefore, limits the knowledge only to those with the training to understand it.
We use semantic programming to communicate implicitly and as a result, well written python is intelligible to pretty much anyone even if they aren't programmers let along aren't python programmers
consider that the Dvorak alternative layout to Qwerty was developed to be faster than Qwerty, but that most users are more efficient with Qwerty just because it's so much more familiar to them
what "communicates meaning implicitly" is very much relative
Relative, but measurable. Its the study of graphic design
f() is "clearly a procedure/function call" only if you already are familiar with C-style programming languages
Something we're quite capable of
i'm not sure where you're going with that
even in graphic design, a lot of things that people take for granted as absolutes are actually cultural and relative
e.g. associations between colors and various emotions
also the idea that programming should be "self-documenting" (perhaps what you mean by "semantic programming") is itself up for debate and probably has no conclusive resolution
Wow, I just can't seem to get a win
short of actual long-term user studies, i don't think there's much else that you can say on the topic of programming language ergonomics beyond expressing one's own preferences
sorry, i don't mean to be shooting you down
but i hope that you can see that this is a broad topic with lots of subtle angles
there is actually a lot to discuss here, but you might do better to focus on some small subset of it, and add focus to the research question
for example you can evaluate the ergonomic tradeoffs of certain python syntax features
I'm basically being told, by many people not just you, that the mere suggestion of applying better design principals to technical languages so they're easier to learn and use is short sighted and foolish of me
no, i am not saying that
i am saying that it's naive to expect that there are such guiding principles
basically it's an open area of study
it's not a solved problem
and generally we have a severe lack of actual user data
In absolute terms maybe not โ but there are some things we know
And there are objectively bad synatxes
sure, that's an interesting angle
Or at least, relatively bad
for example you can look at Malbolge as an extreme example
or silly esolangs like Brainfuck or Whitespace
but then look at APL or J or K, which are just as impenetrable to an untrained reader but are touted as serious programming languages, and some people find them to be tremendously high-productivity tools
My biggest issue โ what prompted me to do this presentation in the first place โ is boolean algebra notation
i suggest starting with something small then
start there and use it as a jumping off point to talk about some specific subset of syntax and language design
also consider that pure math, programming, and theoretical computer science all tend to use slightly incompatible notation at times
there might be something to be said there too. it's been discussed before, but i think constructively
In the early days of the web
Websites were just thrown together
But we got good. We developed the concept of UX design
Templates, standards
And now, web design is all about maximum communication in fractions of seconds
Even to the extent that "variable" and "function" mean different things in programming than in math
My point is that we can, and do, do the same for technical languages. And programming languages (for all their faults) are actually the tip of the spear on this front
you do realize that frontend web development is still a very very very active field in terms of language/framework/api design, right?
Math notations should take point
with many strong opinions and hotly debated tools
Oh of course, as it should be!
My point is that if such a field can be so active, with so many new and excellent ideas being forwarded all the time
That no notation should be considered safe from that
should? or is?
there are lots of sound arguments against so much churn
is all the churn really necessary for iterating towards a better dev experience? does it actually lead to better software? maybe/probably, but the tradeoff is tremendous complexity in some cases, and a large burden on devs to update their skills frequently
All of the churn is absolutely worth it, in my opinion
it would be a struggle to justify a breaking change just because "it makes the syntax prettier" outside of an early alpha, maybe a beta at latest
In thirty short years since the personal computing became affordable for everyone we've developed a system of writing and sharing code, learning code, and improving on each others code that rivals any other human system
not a system, many systems. with various degrees of mutual intelligibility
We already do but a lot of energy into "self-documenting" code, and alongside ubiquitiously available resources, a person can mainline knowledge directly into their front cortex faster than, I think, with any other technical skill
Its truly inspiring just how good at communication we have become โ given that computer code started originally as a way of talking to computers
Now we have computer code that is basically human language
Not talking about prettier. I'm talking about easier to learn and use. Big difference
Perhaps, but then you're losing the expressiveness you were praising above. You can't use = to mean assignment in a programming language, for instance, if you want symbols to mean what they mean in math.
An aside, a context-enabled parser might be able to handle it. The human brain could assuming the context is apparent enough. But we already have a symbol for equality anyway and its perfectly unoffensive
Because you're used to it and learned it.
Math has = and โ for what Python has as == and !=
Because == is a perfectly logical extension of = โ one which capitalizes on the well defined meaning that the = symbol already has
if you want some interesting syntaxes to look at,
COBOL - designed to be read by non-programmers, looks just like english, is nigh-incomprehensible to anyone but people who spent a lifetime writing cobol.
APL - symbol soup, yet an extremely expressive language
sequent calculus - great way to write down type axioms, fairly incomprehensible
lisp/prolog - different priorities than most syntaxes, but still have their own merits.
SML - designed to be easily definable by grammar.
Well, to be honest
I've already got more than enough arguments against my position. And I'm not really looking to do a survey of language syntaxes. I more wanted to make my case that there is no excuse for bad syntax
(In a modern technical notation)
What I'd really like is for someone to agree with me
So, resisting the urge to get salty and storm off like a pouting toddler
I'm just going to continue searching for some research that does support my position
Invariably, when you talk to a group of experts about making improvements to their craft so its easier and more accessible you are met with the arguments "its not so hard", "just put in the work and then you'll be fine", "its too hard to change", "I don't want to have to update my skills", and so on. I remain unconvinced
well, there are a number of aspects to a syntax, I don't think you can really say "bad syntax" "good syntax" as a binary choice in isolation. Ease of parsing, ease of specification, ease of reading, ease of writing, familiarity to probable users, expressivity, ... are often contradictory goals, and everyones preferences differ.
AFAIK, the closest thing anyone got to a universal syntax was XML, whose only real merit was "easy to extend"
I think most of the pushback here is some confusion/disagreement over the concept of "bad syntax". i.e. that in general it may not be a thing that is easy (or even possible) to define in a manner that's agnostic to the biasing powers of personal preference and the micro-cultures around certain languages/frameworks that software developers often become a part of
note that cobol and sql are often considered failures from the perspective of "non-programmers should be able to use them"
Sure, sure
But, "good" and "bad" doesn't mean "worst" and "perfect"
One example is that, as a community, we have pretty much adopted a set of standard for any modern language
my response is as it was in the beginning: what do you mean by "bad"? what do you mean by "no excuse"?
i think there absolutely are good excuses for making compromises on syntax, but at what point does something become "bad"? what do you hope to demonstrate as the thesis of this project? it sounds like you are stating an opinion, not developing an expository work
Just about any modern language is going to have functions, lambdas, classes, try-catch blocks, we've more or less agreed to do away with goto
not to mention the fundamental relativistic problems of what constitutes "bad", which again i have pointed out is broader than you are giving credit for
i'm not trying to argue against your thesis, i am trying to understand what you even mean by it
like, when you tout python as an example of "good syntax", because it is easier to understand by non-experts, I would actually argue that isn't true. Python is so easy to read b/c it hides a lot of detail from you through language features empowered by it's virtual machine. One cannot read python code and actually understand what the computer is doing without a Huge amount of background knowledge. Whereas with C or Rust code, it'll be much harder to read, but you'll have a much clearer picture of what the computer is actually doing when that code is executed. So in what sense is "Python syntax" better than "C syntax" better than "Rust syntax"? I'd argue they are simply different
These are things we take for granted today, but this hasn't always been the case. For a long time programming languages were the wild west, a badland of good ideas and bad competing against each other
this isn't true at all. common lisp has "goto" functionality which is in somewhat common use, and lots of people hate try/catch style exception handling
it also seems like you are conflating syntax with semantics
and there are lots of bad ideas that have stuck around, and lots of good ideas that have gone dormant or fallen out of fashion
some people fucking love pascal still!
Never. Fucking. Mind.
fyi many hot modern languages do not have try-catch blocks, and a lot of software engineers think they are garbage lol
The brains have spoken. Fuck me for thinking outside the box.
@static bluff i think you are really badly misinterpreting the feedback you got here. i suggest stepping away and re-reading all this later
@spice pecan Free Pascal is alive and well with the Lazarus IDE
i've even used apps written in Lazarus
yoooo sorry if I'm coming off as harsh. I think it's an interesting idea to research! I'm asking questions so you can help me define for myself your idea
Yeah I'm aware of that, and so is delphi (which is a dialect of object pascal)
No no! Programming languages are perfect the way they are. Anyone who thinks they can be easier obviously just hasn't done the work. And if you don't like one, then just pick another. Because there's no such thing as good design.
this is completely the opposite of the point i was trying to make
the point is that this stuff is entirely largely unsolved and there are no right answers, and might never be
so your question (which is stated in terms of "good" and "bad") might not be answerable
my favourite language syntactically in recent memory is probably raku, but writing a raku parser is a ton of work compared to sth like scheme where you can get it done in under a day. My point is that in the same way there are merits to still having a gnu-style pure HTML webpages, there are merits to syntaxes which are perhaps not "good design" by whatever arbitrary measure a given judge chooses.
@static bluff quite literally the thing I'm missing is - when discussing software languages - what does the term "good design" mean? I think that's honestly an open question
does raku even have a spec?
nope
exploring the merits of various syntaxes for various goals is an open and interesting problem
determining whether a syntax is good in general is reductive
XML vs JSON, haskell/SML/Rust function type syntax vs. C function type syntax, symbols vs words in PLs, ... are all interesting discussions to have. "Is rust syntax better than XSLT" is utter nonsense.
raku doesn't have a spec? mfw
Because == is a perfectly logical extension of = โ one which capitalizes on the well defined meaning that the = symbol already has
Which well defined meaning that the = symbol has? The well defined meaning that it has in math (is equal to) or the well defined meaning it has in C like languages (becomes set to)
also "the idea of xslt" versus "xslt as it actually exists" i think is a distinction that is lost in a lot of discussions
the = and == thing is not at all intuitive imo, you'd have to be told what it means if you've never seen programming before
also consider that lisp-derived languages do not have ==
prolog unifies the two concepts, and F# uses the same symbol for two different things thanks to ML-like syntax. I do like keeping = as equality and using sth like := or <= <- for assignment.
Just about any modern language is going to have functions, lambdas, classes, try-catch blocks
Rust doesn't have classes. Golang doesn't have try-catch blocks.
not to mention functional langs
it's not even unambiguous what a "class" is
it means different things to different people and in different languages
right, rust doesn't have "classes", but their structs have a lot of things python class does, except inheritance and a few other things
I would disagree, a python class has a lot more than a rust struct, even disregarding inheritance. The descriptor protocol, a metaclass, methods
fair enough, it does have methods though
Without polymorphic access and inheritance, it's not what people generally think of as a class. Though it accomplishes some of the same goals (encapsulation, information hiding, etc)
what's polymorphic access?
The ability to override methods in a subclass, I suppose
oh, well that's out since there's no subclassing. rust does have trait objects though, which let you dispatch on the concrete type of something that implements a trait
Fair enough, that's polymorphic as well (code can be written to accept various concrete types)
tbh I could totally live without inheritance (implementation inheritance)
Isn't composition usually preferred over inheritance
I've heard a lot of people assert that, but I think one should pick whichever reduces the amount of boilerplate.
Fair enough
polymorphic access
and here I was thinking I knew all the OOP esotericism. but the rabbit hole just keeps going.
Generally, yes. Inheritance tends to lead to code the is more coupled and more brittle than composition.
That's not to say both aren't valuable tools to have in your toolbox, but often what inheritance saves you in lines of code/boilerplate, it costs you in coupling.
I see
composition?
inheritance
class Car(ABC): ...
class CarGas(Car): ...
class CarEletic(Car): ...
composition ish
class Engine(ABC): ...
class GasEngine(Engine):
class EletricEngine(Engine):
class Car:
engine: Engine
You are just moving the responsibility up the chain, that way you can "compose" more complex objects without them being reliant on the specifics of implementation
The composition part is Car.engine, Engine and its subclasses are still inheritance (just a note)
A good way to think about composition vs inheritance is whether or not your entity is something or has something. An example I saw on SO used soccer teams to illustrate this - it has a list of players, among other things, but it isn't just a list of players
note of caution: if in the real world A is-a B, it doesn't mean that A and B should be related in a hierarchy somehow
Flashbacks of SO about LSP, on the idea of a Square inheriting from a Rectangle xD
Ah yes. Another example of inheritance that does nothing to demonstrate if there's any practical application.
The one in my undergrad was literally FlightlessBird
You can't have a fly method in the Bird class
how about rewrite some gui library with only composition
Would that be bad?
yes
It seems that one of the best use cases of inheritance is for... I don't know what to call it. When the base class isn't really a "thing" but you write a few methods in the subclass and it does the rest.
I'm struggling to articulate how this is separate from inheritance in general.
an abc?
That's part of it, but I'm talking about a larger design pattern. I think Django uses it?
I saw some submissions in an old Django code jam that barely involved any new logic. It was just a few modules with subclasses of Django types and boom, website. (Though not a very interesting one.)
i imagine trying to recreate behaviors with composition is way more work than it's worth
"base class" i think is a pretty good term for this
"abstract base class" is a specific kind thereof
although in practice a lot of "base classes" are not meant to be instantiated, so are conceptually abstract even if they aren't actual python ABCs