#internals-and-peps

1 messages Β· Page 151 of 1

raven ridge
#

In fact, switching to Python 3 had a decent chance of reintroducing encoding related bugs you had already squashed, as you navigated the bytes -> Unicode changes

torpid bridge
#

It's honestly an interesting perspective into rust's adoption. Writing less bugs is one of the driving powers of the language, but most people get on board because it's easier to write faster code instead.

raven ridge
#

re: Python 3, it turns out that even a lot of professional engineers have a pretty fuzzy understanding the difference between a UTF-8 encoded byte string and a Unicode text string. They're two different representations of the same information.

white nexus
#

what features of py2 made it able to be really easily testable? IIRC unittest was a backport.

additionally, testable functions have to be written in a way that is testable, that I don't understand how py2 made it easy, since it feels easy enough in py3.

#

I've seen for the most part, libraries that support 2.7 on pypi, restrict 3.0, 3.1, 3.2, 3.3, and sometimes 3.4, allowing only 3.5.3 and up to install.

I know the 3.5.3 is because asyncio was provisional, but what other features did 3.4 and 3.5 bring that were key?

raven ridge
# white nexus so, er, what is the difference

A Python 3 Unicode str string stores an array of Unicode code point numbers.
A UTF-8 byte string is an alternative representation for those same code points - namely, the UTF-8 representation.

white nexus
#

I just realised I replied to everything in the opposite order of how it was sent, lol

raven ridge
white nexus
white nexus
raven ridge
#

None that I can think of.

raven ridge
fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | len(as_str)=1 len(as_bytes)=4
002 | as_str[0]='πŸ˜…' as_bytes[0]=240
raven ridge
#

Python 3 Unicode strings operate on logical characters, more or less. Byte strings operate on the individual bytes that make up one specific representation of those characters or another.

white nexus
#

as str is a unicode emoji
encoded as utf-8
the len is different because of how the encoding works
this makes some sense I feel like this will bite me

white nexus
white nexus
raven ridge
#

They're two different representations of the exact same data, though. One representation operates on it at the binary level of one specific encoding of that data, and one operates on it at the abstract level of idealized characters

#

And if you don't have a good handle on what each of those representations is for, and when you need each one, it's really hard to understand why you need to keep converting between them in a Python 3 program.

white nexus
#

yeah idk when I would need a bytes, unless I'm using a method from a different library that needs bytes

raven ridge
#

When you want to serialize that data to a concrete representation. If you want to store it in a database or write it to a file or send it to a web server, you need to pick some sequence of bytes that represents that abstract textual string, and whoever reads it on the other side needs to use the same representation to make sense of what you've written

white nexus
#

oh that makes a lot of sense

#

but when manually messing with it, it's best to have it in string form

#

so that's why subproccess communicate and such are byte streams!!

raven ridge
#

Yep. And why things like open() take an encoding= argument: to allow you to write text strings which they'll automatically handle encoding into your chosen binary representation

white nexus
#

hmmm

#

but there's also the modes in open, "rb" and "wb"

#

so, if open takes ascii as encoding, and wb as the mode, what exactly would happen if we wrote a bytestream to the open file?

raven ridge
#

They're mutually exclusive. You can't provide an encoding if you open in binary mode

white nexus
#

it's binary mode and not bytes mode??

raven ridge
#

Same thing. The raw bits/bytes that the file is storing.

white nexus
#

I think after asking "are bytes different from binary mode" that might be my cue to go sleep

raven ridge
#

You either provide an encoding and open in text mode (so things you read/write are automatically de/encoded using the given encoding) or you give the "b" flag, and the file deals with raw bytes and you're responsible for doing any relevant en/decoding

white nexus
white nexus
#

anyways, I need to go sleep, I'll read your replies in the morning

raven ridge
verbal escarp
#

for me f-strings were the killer feature to switch to 3, but i also wanted to experiment with async, enums and pathlib.. dispatching was also on my wishlist (singledispatch let me down, though)

spice pecan
#

Bytemode also disables the automatic conversion of newlines to "\n"

verbal escarp
#

also something that changed quite a bit how i write my code in python 3 (.8) were positional-/keyword only arguments

#

that is one small underrated thing for me

grave jolt
#

I could 100% live without f-strings

tame grove
#

Is it a bug in Python 3.10??
a dot is missing between now() and strftime but it is suggesting that most probably a comma is missing.

unknown.png
grave jolt
#

ultimately it can't perfectly guess your intention

#

but you can always file a bug πŸ™‚

tame grove
tame grove
grave jolt
#

!e

if datetime.now(), strftime(...) != "19:50:00": pass
fallen slateBOT
#

@grave jolt :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     if datetime.now(), strftime(...) != "19:50:00": pass
003 |                      ^
004 | SyntaxError: invalid syntax
grave jolt
#

then it's probably a bug

tame grove
#

oh ok

tame grove
grave jolt
#

🀷

#

I'm not a CPython developer

tame grove
grave jolt
#

why would this be caused by a problem with windows?

tame grove
#

So I'm selecting Interpreter core
ok??

surreal sun
#

I think the keyword in that error is 'Maybe', it's more or less a suggestion because the program isn't sure what errored out, whether it was a missing comma, a missing ., etc.

raven ridge
#

True, but it might be able to make a better guess, even if it's a heuristic.

surreal sun
#

I think this error is likely raised in the lexer though right? What is the token that is assigned to it and if it's missing it raises the following error

#

I don't think i'm being clear oops

#

I meant like which "missing" group of tokens raises this error

naive saddle
surreal sun
#

ah

raven ridge
#

It should, hopefully, be able to avoid suggesting that you forgot a comma in cases where adding a comma would be a syntax error πŸ™‚

#

seems like Pablo would be glad for the test case, at least.

true ridge
#

It will only be printed if the expression in the context is within parentheses now

white nexus
#

..they were only added in 3.10, weren't they?

true ridge
#

no, they are still there but the context targetting is better now

white nexus
#

oh okay

true ridge
mild flax
# 
class ReusableGenerator():
    def __init__(self, iterable = tuple()):
        self._cache = []
        self._original = iterable
    def __iter__(self):
        yield from self._cache
        while True:
            try:
                new_item = next(self._original)
            except StopIteration:
                break
            self._cache.append(new_item)
            yield new_item
    # VS
    def __getitem__(self, idx):
        if idx < len(self._cache):
            return self._cache[idx]
        self._cache.extend(itertools.islice(self._original, idx - len(self._cache) + 1))
        return self._cache[idx]
```Are there any significant differences with implementing this with one of these methods vs the other? And is there another better way?
#

well I guess one big difference is that one of them is indexable, but other than that?

spice pecan
#

I personally prefer __iter__ for iteration and view __getitem__-based iterables as more of a backwards-compatible sort of thing. It's fine to provide both (and I'd probably do that), but if you just want iterability alone, I'd do __iter__

#

You could also try to use itertools.tee

prime estuary
#

The getiterm logic here's also a fair bit more complicated if you only need to iterate, it'll be constructing and destructing the islice iterator and the like each call.

spice pecan
#

Yeah, that wouldn't be very fun either if you're iterating one item at a time

#

Though it would be partially if not completely mitigated by the implementation of __iter__

grave jolt
#

a().b seems valid

grave jolt
mild flax
#

uhhh I think that was left over from something else I tried

#

now that I'm looking at it though i'd have to be careful of next

grave jolt
#

@mild flax btw, if you want to accept an iterable, do self._original = iter(iterable)

mild flax
#

yeah

#

btw, what would be the proper class in collections.abc to inherit from if I only implemented __getitem__?

grave jolt
mild flax
#

i dont think it likes that

grave jolt
#

why not?

#

ah, it needs an __iter__

radiant garden
#

I don't think __getitem__ without __len__ is a thing, at least not commonly

#

You could check the base classes though

grave jolt
# grave jolt ah, it needs an `__iter__`

then you can do this ```py
class ReusableGenerator:
def init(self, iterable = ()):
self._cache = []
self._original = iterable

def __iter__(self):
    for i in itertools.count():
        try:
            yield self[i]
        except IndexError:
            break

def __getitem__(self, idx):
    if idx < len(self._cache):
        return self._cache[idx]
    self._cache.extend(itertools.islice(self._original, idx - len(self._cache) + 1))
    return self._cache[idx]
#

hmmm actually no this is terrible

#

or maybe not

#

I guess it's not

mild flax
#

no worse than iterating with getitem i think

radiant garden
#

Yeah, collections.abcs with __getitem__ always have __len__

mild flax
#

which isn't a problem with __getitem__

prime estuary
#

__iter__ isn't required, if you have __getitem__ iter() automatically adapts. Though the ABCs wouldn't recognise that.

steel solstice
#

anybody happen to know the directive thats used for a code-block with sphinx?

#

i tried docutils.parsers.rst.directives.body.CodeBlock but the formatting is all funky

snow bronze
#

πŸ˜΅β€πŸ’«

# 
>>> type(obj)
<class 'dict'>
>>> len(obj)
170
>>> sys.getsizeof(obj)
9312
>>> obj2 = dict(obj)
>>> type(obj2)
<class 'dict'>
>>> len(obj2)
170
>>> sys.getsizeof(obj2)
4696
>>> obj == obj2
True
#

why does size cut half?

visual shadow
#

That must have been all the size it needed. So the more important question to ask is why obj wanted to have more size. The answer most likely lies in the overallocation python uses, so details about how obj was created is relevant

snow bronze
snow bronze
visual shadow
#

So, for things that grow dynamically, such as python lists or in this case dictionaries, python promises an average time complexity of append(list)/adding an item(dict) as O(1). to achieve this practically, it has a system of overallocating and growing, so every once in a while, it runs out of space, has to reallocate the whole thing, but it takes more memory than it needs, so that theres room to add newer items without taking an O(n) reallocation

#

While i dont exactly know the mechanisms of the .json method involved here, it's easy enough to speculate that .json must benefit from or require python dictionaries to be created incrementally, or possibly, even if they did not need to be done that way, they end up making the python dictionary item by item.

#

So that's what youre seeing. it's not obj2 becoming "more efficient" so much so as not needing the extra "overallocated" space because obj2 was given the entire input whose* length was known upfront (that being the len of obj dict)

#

so obj2 didnt need overallocation

prime estuary
#

Additionally for dicts, they want to avoid being too full - if that occurs, it's more likely there's hash collisions requiring checking multiple locations. So Python's dict currently doubles in size whenever it becomes more than β…” full.

grizzled spire
#

this shouldn't exist
SyntaxError: assignment expression cannot be used in a comprehension iterable expression

#

no real reason

spice pecan
#

the real reason is there's an implicit function scope

#

and the results of that can be very difficult to debug

#

If you aren't aware of it, that is

grizzled spire
#

for example has this propery but its allowed

spice pecan
#

if has no scope

grizzled spire
#

true

#

but like

spice pecan
#

Most things don't

#

In fact, I can't think of anything that does other than functions, classes and modules

grizzled spire
#

for itterables just make it add to the scope

#

if its defined

spice pecan
#

I guess something like an implicit nonlocal for assignment targets within a scope would work

grave jolt
#

walrus in a list comprehension is cursed

#

a list comprehension shouldn't have any side effects

spice pecan
#

It can sometimes be useful for the actual comprehension, but at that point it's usually way more readable to just have a loop

elder blade
# 
['a' if some_var else 'b' for item in my_list if (some_var := do_something_with_it(item)) is not None]
grave jolt
#

I guess this is roughly manageable

elder blade
#

Equivalent to: ```python
new = []
for item in my_list:
if not (some_var := do_something_with_it(item) is not None:
continue
new.append('a' if some_var else 'b')

#

I originally thought the behaviour would be that you got access to the last item, and agreed with you.. that said the more I think about it the more I like it

# 
[value if (value := something(item)) else None for item in my_list]
#

Another example

#

All of these can of course turn into ```python
[item if item else None for item in [something(item) for item in my_list]]

#

That said... What do you prefer πŸ˜‰

verbal escarp
#

object.__dir__(thing) works like a charm, but if i try to do object.__getattribute__(thing, name) some methods are included in the dir i can't access that way, why is that?

#

getattr(thing, name) catches those, interestingly

peak spoke
#

getattribute only does a direct lookup, skipping things like looking things up on the parent type

#

!e

class A:
    a = 0

class B(A):
    b = 1

getattr(B, "a")
object.__getattribute__(B, "a")
fallen slateBOT
#

@peak spoke :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 8, in <module>
003 | AttributeError: 'type' object has no attribute 'a'
verbal escarp
#

okay, makes sense

#

the asymmetry is a bit confusing

spice pecan
grave jolt
#

I would use a for loop or py bar = [item if item else None for item in (something(item) for item in my_list)] or maybe ```py
foo = (something(item) for item in my_list)
bar = [item if item else None for item in foo]

elder blade
grave jolt
#

you can put any expression in there

elder blade
#

Ooooohh, I missed the parenthesis

unkempt rock
#

Why are type hints are required in dataclasses? lemon_glass #help-chili

limpid forum
grave jolt
prime estuary
unkempt rock
#

interesting

#

!e py from typing import get_type_hints class C: x: bool y: int print(get_type_hints(C))

fallen slateBOT
#

@unkempt rock :white_check_mark: Your eval job has completed with return code 0.

{'x': <class 'bool'>, 'y': <class 'int'>}
quick snow
#

!e

class C:
    x: bool
    y: int
print(C.__annotations__)
fallen slateBOT
#

@quick snow :white_check_mark: Your eval job has completed with return code 0.

{'x': <class 'bool'>, 'y': <class 'int'>}
cloud crypt
#

getting back seldom-reoccurring discussions on HKTs, I feel that making TypeVars subscriptable as an opt-in seems like a good idea

as a short example of what's on my mind now, here goes

I = TypeVar("I", bound=Iterable, args=1)  # or something else
# having an exact amount of args could benefit simple runtime checks
# (the default would be 0, which makes perfect sense to me)```

let `Bound` be the type passed as `bound` for `I` type variable
then `I[T]` is *roughly* `Bound[T]`, with the main idea of preserving the actual type

I'm not exactly certain what would be required for type-checkers to implement this, but it seems like a fine idea to check against the `bound` when calling functions (and other cases)
(i.e. for `I[T]` we need to check if some type `A` is `Iterable[T]`)

when acting on these types, type-checkers would have to track the type to ensure it stays the same as the original `I`, but I don't think anything else is needed

if we look at it in action, here is a (slightly stripped) use-case
```python
W = TypeVar("W", bound="Wrap", args=1)

@dataclass()
class Wrap(Generic[T]):
     value: T

    def unwrap(self) -> T:
        return self.value

    def map(self: W[T], function: Function[T, U]) -> W[U]:
        return self.__class__(function(self.unwrap()))

class Derived(Wrap[T]):
    pass

wrap = Derived(42)  # Derived[int]
result = wrap.map(str)  # Derived[str]
print(result.unwrap())  # str```
I'd be happy to hear other inputs on this issue :)
#

I've come across the need to abstract over the type constructor quite often, so it is definitely one of the main features I'd love to see implemented
for example, I've been working on iterators that aim to bring rust-like operations to python

I = TypeVar("I", bound="Iter", args=1)

class Iter(Iterator[T]):
    iterator: Iterator[T]  # implementation detail, need not to be generic over type

    def unwrap(self) -> Iterator[T]:
        return self.iterator

    def map(self: I[T], function: Function[T, U]) -> I[U]:
        return self.__class__(map(function, self.unwrap()))

    def exhaust(self) -> None:
        exhaust(self.unwrap())  # this function is implemented elsewhere```
and it would be neat if anyone could derive from `Iter[T]` to add useful (and more specific) functionality:
```python
class ForEach(Iter[T]):
    def for_each(self, function: Function[T, None]) -> None:
        self.map(function).exhaust()```
(obviously a toy example, since `for_each` is a pretty general and useful function, but I hope I have given a use-case)
unkempt rock
#

what ide do you guys recommend rn i am using pycharm

verbal escarp
#

not really a good topic for this channel

surreal sun
#

I[T] would be quite weird

#

or would it only be used for when bound to something

cloud crypt
#

hm, I don't really think it makes sense to have non-bound type variables? not fully sure though

surreal sun
#

Don't people use non-bound type variables like a generic?
something like

from typing import TypeVar

T = TypeVar("T")

list[T]
#

i'm not fully sure though, because I don't use TypeVar all too much

#

I've mostly seen other people use it

cloud crypt
#

well, yeah, they do, but the point is, applying type arguments to a type variable would make sense only when you have something generic to apply them to

surreal sun
#

how would a type checker handle two non-bound typevars together though like
I[T]? where I and T are both non-bound typevars

#

would it be something like Any[Any]

#

genuine question btw, i'm quite inexperienced with typing

cloud crypt
surreal sun
#

so I'm gonna assume your proposal is only limited to bound typevars?

cloud crypt
#

essentially, yeah

surreal sun
#

Oh yeah, i'd agree then for sure

cloud crypt
#

args > 0 is going to be allowed iff bound is passed as well

#

since, again, it's not clear how to handle non-bound type variables, and, besides, I do not see a use-case for that

# 
from typing import Callable, Protocol

F = TypeVar("F", bound="Functor", args=1)
T = TypeVar("T")
U = TypeVar("U")

Function = Callable[[T], U]

class Functor(Protocol[T]):
    def map(self: F[T], function: Function[T, U]) -> F[U]:
        raise NotImplementedError()``` and that's what the possible implementation of a *functor* from functional programming can be
#

I quite like the way it works (or can work, rather), so what I'm especially looking for are edge-cases I could've missed

grave jolt
#

@cloud crypt now comes the hard part: convince a bunch of Python developers on why that is useful πŸ™‚

surreal sun
#

convincing python-ideas is like talking to a brick wall, except that brick wall sometimes agrees

cloud crypt
#

haha, indeed

cloud crypt
gusty marsh
#

Heres the corresponding section of the code, basically I am trying to reimpor the functions on file change event so they get updated but it seems its hanging on the from solution import line.

Logs:

/home/shivansh/Programming/python_projects/aoc-py/2021/01
importing...
/home/shivansh/Programming/python_projects/aoc-py/2021/01
importing...
imported...
<function part_one at 0x7f6d654f95a0> of 1...
❌ Testing of day 1 part 1 failed! Please try again. Here's the output:
        Expected: 7
        Recieved: 17
Modified
importing...
Screenshot_from_2021-12-06_07-11-24.png
white nexus
#

cough is this was you were dming me about

gusty marsh
fallen slateBOT
#

aoc/watcher.py line 52

class ModificationWatcher(FileSystemEventHandler):```
gusty marsh
white nexus
#

help channel may be more fitting

gusty marsh
deft ruin
#

Does anyone know if it has already been suggested (in a PEP or otherwise) that generator objects should be handled lazily in sequence unpacking? Specifically when used like:

a, b, *rest = some_generator

that rest will be the remaining generator, rather than a fully evaluated list of that generator?

elder blade
#

I am not sure whether I like that, it'll be a breaking change for sure

#

I take that back already, I can't decide

#

I feel like both behaviors have their benefits and can be expected

prime estuary
#

It would be very nice to have, but it'd be an odd special case.

#

Should probably be some additional syntax?

deft ruin
#

I agree that it would have been much better to introduce with 3.0 or something

native flame
#

a, b, rest = next(gen), next(gen), gen is fairly clean \πŸ€”

elder blade
#

Yeah

native flame
#

might be some way to use itertools.islice too

deft ruin
#

thats true, both ways of achieving the same thing. It was also the non-lazy behaviour which surprised me

prime estuary
#

You definitely could.

a, b = itertools.islice(gen, 2)
#

Though the downside is both of these are probably slow func calls, we'd have to see though if the 3.11 adaptions make it comparable.

elder blade
#

Oh wait...

#

!e ```python
def gen():
count = 0
while True:
count += 1
yield count

g = gen()
a, b = g
print(a, b, g)

fallen slateBOT
#

@elder blade :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 8, in <module>
003 | ValueError: too many values to unpack (expected 2)
elder blade
#

Makes sense

deft ruin
#

Lol, that was exactly the code I tried that made me consider this

#

because adding a *c to the unpacking, makes it hang forever

prime estuary
#

Thought, allow this to indicate it should not try to consume additional values:

a, b, ... = blah

A star-arg at the end would be prohibited since it's contradictory, use a walrus if you need to save.

spice pecan
#

some sugar for lazy evaluation would definitely be nice

deft ruin
#

Maybe we could borrow from JS and use ... (or just ..) as a lazy alternative to *

#

a, b, ..c = gen

spice pecan
#

I'm still rooting for _ to become an actual throwaway

deft ruin
#

Yeah would be nice

spice pecan
#

then you would be able to do a, b, *_ to lazily get the first two, and a, *_, b to get the first and the last without collecting everything in between and actually discarding it

deft ruin
#

how does that work for generators though...

#

maybe that is the reason it cant be. because when generators (or custom iterables) are involved, it must evaluate all intermediate values. But it wouldnt have to store them

#

you could at least save some memory

spice pecan
#

depending on the shape of lhs, either stop iteration on *_, or save the exact amount of args after it into a list and unpack that

deft ruin
#

but then it would only work with objects that define __len__

spice pecan
#

not really

#

you could achieve the same with a maxsized deque

#

keep pushing new items and discard irrelevant ones as you go

deft ruin
#

oh yeah fair point

spice pecan
#

a fair bit more work for the compiler, but I personally would love that

#

if you ignore the fact that _ is currently a valid identifier, it shouldn't be breaking either

deft ruin
#

PEP 677: Better sequence unpacking. _ as a memory free 'hole' and .. as a lazy alternative to *.
πŸ˜„

#

but surely the _ thing has been suggested before

spice pecan
#

Yeah, it's currently used for i18n

#

So I highly doubt this will ever get into the language

#

but if you give special treatment to unpacking into _, you can get both lazy and eager evaluation without a breaking change (except for _ and weird constructs that maybe rely on eager evaluation for side effects?) and without introducing new syntax

deft ruin
#

well, I think you can't have it both ways. Either it is a hole which does not store values (and can be placed anywhere) or it makes * lazy, and can only really be used at the end

#

because what if i want to use the generator again later

#

whats left of it anyway

#

Or maybe its smart enough to know, that if it is at the end, then it stores the remaining generator. Otherwise nothing

elder blade
spice pecan
#

well it currently isn't

deft ruin
#

impossible while keeping them as generators

#

the current implementation essentially converts to a list (or tuple?) and then goes ahead

spice pecan
#

but if you just use the equivalent of a sized deque to capture exactly N items, you get all the benefits

#

with a simple circular buffer you avoid storing intermediate values and only get the relevant ones, which are then popped into the assignment targets

#

and if you don't have any targets after the discard, there's no need to evaluate the rest

deft ruin
#

is it fine for random people like me to make PEPs? or is that frowned upon unless I really know what im doing

spice pecan
#

You should first get the general consensus in the mailing list

#

and if there's demand, you go further and draft a pep

deft ruin
#

Oh, I thought the mailing list was like invite only or something

#

interesting

#

Ah, non members can post to Python-dev

spice pecan
# elder blade That's impossible for generators though?

Explaining what I mean with equivalent python code:

a, _, b = sample  # Use current semantics
# Roughly 
t = list(sample) 
if len(t) < 3: raise ValueError("not enough values to unpack") 
if len(t) > 3: raise ValueError("too many values to unpack")
a = t[0]; b = t[2]; 

a, b, *_ = sample  # Lazily evaluate first two args 
# Roughly 
i = iter(sample) 
a = next(i) 
b = next(i) 

a, *_, b, c = sample  # Lazily evaluate, discarding intermediate values
# Roughly 
i = iter(sample) 
a = next(i) 
d = deque(i, maxsize=2)  # Two targets after *_, size = 2
if len(d) < 2: raise ValueError("not enough values to unpack") 
b = deque.pop_left()
c = deque.pop_left()

a, *b, c = sample  # Use current semantics 
# Roughly 
i = iter(sample) 
a = next(i) 
t = list(i) 
b = t
c = t.pop()
#

Sample is any Iterable

#

There's no way this is getting into the language, but a man can dream

deft ruin
#

A new syntax might be possible to add though...

spice pecan
#

Yeah, but I don't think the chances are good

#

new operators add a fair bit of complexity

deft ruin
#

and using the lazy unpacking, at least cases with .._ at the end, would work the same as yours

#

yeah

#

i also am not sold on the syntax

spice pecan
#

The main issue here is that all this can be implemented fairly simply with a function

#

and there's little ground to force it into the language

deft ruin
#

well, most things can be fairly simply implemented with a function, especially syntactic sugar things πŸ˜„

#

I agree that it certainly isn't necessary, but I do think it would be nice (if a better syntax were found. The more i look at .. the uglier i find it)

spice pecan
# 
def unpack_lazy(iterable, first=0, last=0, keep_intermediate=False):
    i = iter(iterable)
    yield from islice(i, first)
    length = None if keep_intermediate or not last else last
    yield from collections.deque(i, maxlen=length)

first, second, second_to_last, last = unpack_lazy(i, 2, 2)
*body, last = unpack_lazy(i, last=1, keep_intermediate=True)

ig keep_intermediate is not necessary, because it just turns it into regular unpacking, but yeah. Good enough for most purposes

lusty scroll
#

lazy unpacking is a thing?

spice pecan
#

no

#

but you can make it one

deft ruin
#

but it should be (imo)

spice pecan
#

sort of

#

lazy unpacking of n first items is just islice

#

and unpacking last n isn't really lazy

lusty scroll
#

so the idea is improved efficiency?

deft ruin
#

For generators the idea is to act more intuitively, and manageably

#

as said, it works also with islice but I think some syntactic sugar would be cool

lusty scroll
#

so it's like partially unpacking a sequence (or unpacking part of the sequence and leaving the rest in a separate argument)

spice pecan
#

yeah

#

take first N items and leave the rest intact

#

Or take first N and last N, actually discarding everything in between instead of accumulating into a list and then immediately throwing it away

#

(well, in most cases you won't even throw the list away, it'll just stay bound to whatever placeholder you give it)

white nexus
verbal escarp
#

oh man, i just went in functools to check update_wrapper and stumbled over this

# 
class Negator:
    @singledispatchmethod
    @classmethod
    def neg(cls, arg):
        raise NotImplementedError("Cannot negate a")

    @neg.register
    @classmethod
    def _(cls, arg: int):
        return -arg

    @neg.register
    @classmethod
    def _(cls, arg: bool):
        return not arg
#

that's soo ugly..

#

there has to be a better way!

feral cedar
#

slams desk

grave jolt
#

obviously

Negator = exec_haskell("""
    class Negatable a where
        neg :: a -> a

    instance Negatable Integer where
        neg n = -n

    instance Negatable Bool where
        neg = not
    
    -- export Negatable as Negator
""")
verbal escarp
spice pecan
#

I mean

#

What's wrong with a function? Is there a real need for a class here?

# 
def negate(arg):
    if isinstance(arg, bool):
        return not arg
    return -arg
white nexus
#

πŸ˜”

#

!e ```py
f = 5.4
print(-f)

fallen slateBOT
#

@white nexus :white_check_mark: Your eval job has completed with return code 0.

-5.4
spice pecan
#

Does it matter?

white nexus
#

and why does it not support floats.

white nexus
spice pecan
#

Depending on your domain, you may want int specifically

#

But then just typehint it as such and follow the consenting adults principle

#

If they pass a float and it breaks something, it's their problem, not yours - you did let them know that ints are needed

#

And if you intend to support any negatable type/any numeric type, you should either use a duck-typing protocol that defines __neg__, or hint it as numbers.Numeric (or something along those lines, my memory is hazy on those ABCs)

verbal escarp
#

well, for one, the stacked decorators simply look ugly, secondly, and more importantly, having several functions with the same name is not a good idea, greatly confuses IDEs and interpreters alike

#

it's a similar problem with properties, basically impossible to get nice looking code without messing up the tooling

wheat plover
#

singledispatchmethod is essentially a dict on first argument type (I think?) so if the decorator looks ugly, just use a dict (or match stmt)

verbal escarp
#

you can't overload an external function that way

grave jolt
#

the main question is: why would anyone need a Negator like this?

verbal escarp
spice pecan
#

It's a really poor example

verbal escarp
#

it is

grave jolt
#

or does singledispatchmethod just look at the concrete type?

verbal escarp
#

i think it only checks concrete type

grave jolt
#

that's pretty silly

verbal escarp
#

it's a workaround, nothing more

#

i've been disappointed by dispatching in python for a long while

spice pecan
#

I'd probably either generalize better (like above), or dispatch to specific overloads via match

wheat plover
spice pecan
#

But I do admit that it's... not as good as it could've been

verbal escarp
#

if you can't manipulate the patma, you're out of luck, mostly

wheat plover
#

I think problem with generalization is that multiple inheritance exists.
So MyWeirdClass(int, bool) would qualify for both

#

Something like that would make say, java throw errors on you

spice pecan
grave jolt
#

@verbal escarp I think you want typeclasses πŸ™‚

verbal escarp
#

tell me more

wheat plover
#

But python doesnt do overloading anyways so singledispatch only does so much

spice pecan
#

Ad-hoc polymorphism, sounds really fitting

grave jolt
#

unless I completely misunderstand what you want

#

@verbal escarp you want to extend an existing function someone else wrote to accept your own class, right?

verbal escarp
grave jolt
#

that's what typeclasses do

#

sort of

verbal escarp
#

how would you realize the Negator with typeclasses?

grave jolt
# 
class Negatable a where
    neg :: a -> a

instance Negatable Integer where
    neg n = -n

instance Negatable Bool where
    neg = not
``` πŸ™‚
verbal escarp
#

in python :p

grave jolt
verbal escarp
#

click

#

that looks fairly similar to singledispatch?

wheat plover
#

btw is this a good place to ask for library recs?

grave jolt
verbal escarp
grave jolt
#

can't you actually do what you want with singledispatch?

verbal escarp
#

as i said, i only stumbled over this ugly corner in the documentation while looking for update_wrapper.. i've used singledispatch before, but it always left a kind of stale aftertaste in my mouth

#

been looking for better alternatives to dispatching and property ever since

grave jolt
#

what's wrong with property?

verbal escarp
#

ever used it?

spice pecan
#

!pypi multipledispatch have you considered this?

fallen slateBOT
verbal escarp
grave jolt
verbal escarp
spice pecan
#

I guess it depends largely on your domain

#

But I've never had that issue

#

Can you give an example?

verbal escarp
#

i'll have to dig a little

#

one sec

grave jolt
#

I generally don't use property to be honest

#

in my mind it's just a tool for backwards compatibility, or when you have to conform to an interface, or something

#

so I usually just make a method 🀷

verbal escarp
#

no, properties are awesome as a concept

#

i do love the idea

#

but the code to actually get there is just.. meh

spice pecan
#

I find them much more useful in languages that obsess over access obstruction

#

But I don't see much of a difference between properties in, say, C# and Python

white nexus
fallen slateBOT
#
unittest.mock.PropertyMock 

class unittest.mock.PropertyMock(*args, **kwargs)```
A mock intended to be used as a property, or other descriptor, on a class. [`PropertyMock`](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.PropertyMock "unittest.mock.PropertyMock") provides `__get__()` and `__set__()` methods so you can specify a return value when it is fetched.

Fetching a [`PropertyMock`](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.PropertyMock "unittest.mock.PropertyMock") instance from an object calls the mock, with no args. Setting it calls the mock with the value being set...
white nexus
#

because if I'm mocking a property, being able to know when my methods don't use that property is somewhat nice to know imo

verbal escarp
#

sure, but properties are much more useful than keeping track of access

white nexus
#

oh, yeah, ofc

#

I mean, if I had to overwrite a property with an attribute, I haven't used a mock since I didn't know this existed

#

I've just used the value that it needs directly

verbal escarp
#

one thing i fiddled with a long time ago was a flyweight class that would abstract access to numpy arrays

white nexus
#

but haven't been able to apply side effects or anything

verbal escarp
#

a slotted class with an ID as the only real attribute

#

you could pass it around and do things like cell.material

#

then it would go to the numpy array, basically as an API

#

it was a huge class, only consisting of properties

#

so i experimented with different ways to define properties so that my IDE wouldn't complain about duplicated names on one hand or having 2, 3, 4 definitions with everything, polluting the namespace

#

sorry, i can't find the code anymore on the spot

spice pecan
#

I think the larger issue is not property (or any other similar python feature) itself, but moreso the fact that typechekers have a REALLY hard time keeping up with (fairly predictable, by human standards) levels of dynamic typing

verbal escarp
#

those issues aren't exclusive πŸ™‚ both ends are a mess

spice pecan
#

Python as a language gives you a lot of tools that allow for very powerful and, I can't really find a better word for it, quirky constructs, that are not necessary, but appealing as a concept. And when you try to apply typecheckers to this, they tend to go nuts

spice pecan
#

But, in my personal experience, one magnifies the other significantly, while the other barely ever pops up at all

#

I've seen and experienced first-hand a plethora of cases where things are pretty readable and function well, but then you start trying to shove typing in or fix that one thing the IDE is bugging you about

#

And embark on an adventure that results in a huge mess and something along the lines of "I should've really just kept it simple"

#

Typing is nice, and so is the dynamic nature of python. Do they mesh together well? Not at all

grave jolt
#

somehow TypeScript works pretty well

#

and JavaScript is dynamic as well

spice pecan
#

I wish python had a typing system as good as ts

grave jolt
#

I only had to //@ts-ignore (or similar hacks) some very arcane stuff

spice pecan
#

Javascript also manages to be fairly fast

#

And has a ton of money poured into developing it

empty kite
#

Is there a better way to type hint recursive functions than having <return type of final recursion> | Callable[<repeat all argument types>]?

grave jolt
empty kite
#

As a return type

grave jolt
#

a recursive function doesn't return a function, does it?

empty kite
#

Yeah you're right mb

spice pecan
#

A recursive function calls itself, and it returns the result of a recursive call immediately in a lot of cases, but it still returns the same type

#

unless you're into voodoo magic

grave jolt
#

yep ```py
def factorial(n: int) -> int:
if n <= 0:
return 1
return n * factorial(n - 1)

verbal escarp
#

but side-effects are hard to type anyway

#

wonder if any language has a way to type side-effects

#

maybe something like a doctest asserting proper manipulation of something?

spice pecan
#

Well, you'd most commonly see ways to type a lack of side-effects

#

Or a general presence of them

#

But I myself am not aware of a language that actively types the exact side-effects it has

#

(inb4 it turns out haskell does that, too)

verbal escarp
#

hmm

#

i was just thinking of parsers, but maybe a better abstraction would be an FSM

#

with behaviour ("return type") depending on the current state

#

that should actually be feasible in python

quick snow
verbal escarp
#

a closure whose return type is depending on previous input

verbal escarp
quick snow
#

(but it's been a looong time since I did any Haskell, so take it with a grain of salt)

#

But this kind of handling is typical for functional programming languages. It's the same in SML, IIRC

grave jolt
#

quick reminder that monad tutorials are prohibited ❌ in this channel

#

/hj

verbal escarp
#

lmao

#

if we empowered python's typing system to include input-dependent output and allowed literals as types, we'd be heading straight for a parser-language syntax for typing

#

just a thought

quick snow
#

may as well use a language that does types properly then, not the half assed type hints that are a necessary consequence of Python's dynamicness

verbal escarp
grave jolt
#

yeah, smoothly transition your application from Python to Agda πŸ˜‰

verbal escarp
#

xD

#

before that, we should have latex as valid identifiers πŸ˜‰

#

\pi

quick snow
#

Ο€ works

#

I'm sure there's a plugin for your editor of choice that translates LaTeX symbol names to Unicode

verbal escarp
#

i have to check that

quick snow
#

(Or just use the supreme way of entering text, the Compose Key)

verbal escarp
quick snow
#

not all of them, but all that I'd ever use in LaTeX or natural language

verbal escarp
quick snow
#

your coworkers (etc.) will love you

spice pecan
quick snow
#

378

spice pecan
#

.xkcd 378 I think

neon troutBOT
#
XKCD comic #378

Real programmers set the universal constants at the start such that the universe evolves to contain the disk with the data they want.

2008/2/1 - #378, 'Real Programmers'
quick snow
#

yep :D

verbal escarp
#

🀣

quick snow
#

(of course I actually have to remember two compose-y ways, because Vim has a similar, but incompatible feature)

verbal escarp
#

apropos half-assed type hints of python, any news on deferred annotations? everything still in limbo on that front or did they decide that pydantic has to die for the greater good by now?

quick snow
#

I thought I read a few weeks ago that the alternative proposal was accepted, no? So __future__.annotations will never come directly, but __annotations__ will become some kind of magical lazy dict.

verbal escarp
#

i must've missed that

#

okay, sounds hacky enough

quick snow
#

Not accepted, but "favored" by SC (PEP 649)

verbal escarp
#

i wonder why there's so much opposition for a DSL, it wouldn't even be the first one in python

#

ahhh.. oh no

#

"We definitely care about the overhead of type annotations. At one point pre-PEP-563 I think our codebase spent 1% of total CPU on executing annotations. That’s a lot, although I believe most of it was due to the old GenericMeta implementation that is also gone in Python 3.7 thanks to class_getitem.

I don’t think we know yet how the performance compares, or whether the difference is significant. I tried to do some performance comparisons a few months ago on a large well-annotated codebase, but I wasn’t able to get the reference implementation of PEP 649 to work at all so didn’t make those comparisons. I think that it would be valuable to have a working reference implementation and some performance data (and then perhaps some work on perf tuning: possibilities have been suggested in earlier threads about PEP 649) before making a final decision on PEP 649."

#

i guess that might count as an argument against a DSL

#

on the other hand, one could maybe pre-compile all annotations

quick snow
#

can't

#

How would you precompile foo: bar.bat, when bar is a user-defined object which can do whatever on __getattr__? Also, I think there was something about nested classes and get_type_hints not working with that.

#

basically the expression (might?) need access to the definition scope

verbal escarp
#

hmm.. it doesn't have to be "legal" code from the static compiler's pov, just turn the str into a code object that is executable in the runtime step

#

your dynamic example wouldn't be checkable by a static checker either, i think

quick snow
#

Right, isn't that almost what PEP 649 is doing? Haven't looked too deeply into specifics, got too frustrated by some peoples' attitude

quick snow
verbal escarp
#

yeah

quick snow
#

Right, but the whole argument is about usecases other than static typechecking; actually useful stuff like the things pydantic does

verbal escarp
quick snow
#

as far as I understood

verbal escarp
#

raises a couple of eyebrows

quick snow
#

That way you have no problems with forward references (the original usecase of the future import), but you still can do stuff pydantic etc. do

#

Other than pydantic, the only time I used type annotations was for registering commands in an fungoid esoteric programming language, so I'm not the most unbiased person in this debate, but I think PEP 649 is a great compromise

verbal escarp
#

okay, reading 649 more in detail, it does sound much smarter

#

although, i can foresee some potential issues

#

not sure how well it would handle access to the annotation during the limbo of definition time

#

he says that forward references shouldn't be a problem, but i'm skeptical

unkempt rock
#

any1 know why im getting 0

unknown.png
verbal escarp
unkempt rock
#

bruh

#

just help me

#

instead of being wierd

quick snow
#

In a help channel someone will help you, here it's unlikely

grave jolt
#

@unkempt rock Each topical channel has a particular topic, otherwise we'd just have 100 channels named #python-0 through #python-99. You should see #β“ο½œhow-to-get-help and claim a help channel.

unkempt rock
#

im currently learning about language developement and im reading a chapter about parser and Im on a bit called panic mode error recovery

# 
Before it can get back to parsing, it needs to get its state and the sequence of forthcoming tokens aligned such that the next token does match the rule being parsed. This process is called synchronization.```
#

I dont understand this

#

does anyone have a simpler way to explain this

#

I cant understand what synchronization means

pliant tusk
#

basically you dump tokens until you find one that makes sense

#

or you bail out

pearl river
unkempt rock
#

throw way temporarily?

pliant tusk
#

yea

unkempt rock
#

thank you

rich cradle
#

Not running, but so you can report other syntax errors after, since smacking one syntax error at a time like whack-a-mole is no fun

pliant tusk
#

^

#

i mean, you could technically make a language that auto recovers and keeps running but that would be unwieldy

unkempt rock
#

so if you find one error the parser cuts away the invalid tokens so you can find additional errors

elder blade
#

I don't think Python did this right?

unkempt rock
# pliant tusk i mean, you could technically make a language that auto recovers and keeps runni...

https://en.wikipedia.org/wiki/Esoteric_programming_language

Esoteric programming language

An esoteric programming language (sometimes shortened to esolang) is a programming language designed to test the boundaries of computer programming language design, as a proof of concept, as software art, as a hacking interface to another language (particularly functional programming or procedural programming languages), or as a joke. The use o...

pliant tusk
#

no python bails out

unkempt rock
#

this is what I am trying to do

#

im gonna make an extremely similar recreation of python with python and then make it entirely upside down

#

learning the theory right now

#

Thank you guys

#

❀️ πŸ‘

unkempt rock
#

syntax error wack-a-mole is the most relatable thing ever for programmers lol

#

you deal with 1 and 2 pop out

paper echo
elder blade
grave jolt
#

Do you think it's acceptible to have single-letter type variables?

flat gazelle
#

I find it a bit silly, but it is the universal convention by now

paper echo
gleaming rover
lusty scroll
native flame
#

I suppose its useful that having single-letter names makes it distinguishable from concrete classes

#

i don't think i've seen multi-letter names anywhere
(well, excluding suffixes like in T_co)

spice pecan
#

I like the C# convention of T for regular generics and TKey, TValue for mappings, for example

#

Makes type variables distinct, gives them sane names when needed

verbal escarp
#

all this typing theory.. weird it never really was a topic when i studied CS

#

times change..

#

datastructures, computability, parsing.. OOP of course with subclassing etc. but types per se as something important?

radiant garden
#

For theory of programming languages, yes

grave jolt
verbal escarp
#

who needs protocols and weird-ass annotations!

radiant garden
#

STOP DOING TYPES
This is real type theory done by real mathematicians
They have played us for absolute fools

gleaming rover
#

but honestly if you look back at e.g. Hungarian notation, there was demand for doing something like that

#

we just didn’t have the appropriate tools

grave jolt
#

hot take: putting Error or Exception at the end of your exception class name is Hungarian notation

#

not joking

quick snow
#

Except your custom exception classes aren't usually instances of Exception

grave jolt
#

?

quick snow
#

If I name my variable i_age (i for int), then that variable is an instance of int

grave jolt
#

ah

#

well, yes, it's a bit different

quick snow
#

But your custom exception class isn't an instance of Exception, it's a subclass

#

It would apply if you would name the instances of your exception *_exception

#

except MyException as custom_exception <- here I agree :D

lusty scroll
dusk comet
#

use Exception as metaclass

olive marsh
dusk comet
#

why is it working? SoSoHurt

# 
>>> MyAwesomeException
Exception('MyAwesomeException', (), {'__module__': '__main__', '__qualname__': 'MyAwesomeException'})
# 
>>> class X(metaclass=print): ...
...
X () {'__module__': '__main__', '__qualname__': 'X'}
>>> X is None
True
```now `print` is metaclass
unkempt rock
#

It doesn't implement the right protocol, so arguably it isn't a metaclass

#

Otherwise any function that takes three positional arguments is a metaclass

#

And I don't think that's really right, heh

quick snow
#

Now find a useful example of using metaclass= with something that isn't

#

!e Defining a string without quotes, maybe:

class Foo(metaclass=str.format): ...

print(repr(Foo))
fallen slateBOT
#

@quick snow :white_check_mark: Your eval job has completed with return code 0.

'Foo'
elder blade
#

Okay but wtf is this

raven ridge
#

!e print(str.format("Foo", (), {}))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

Foo
raven ridge
#

Pretty cursed.

magic nova
#

now make it so I don't have to pre-define Foo

spice pecan
#

That's an interesting solution

#

!e ```py
def stringer(func):
return func.name

@stringer
def unquoted_string(): pass

print(unquoted_string)

fallen slateBOT
#

@spice pecan :white_check_mark: Your eval job has completed with return code 0.

unquoted_string
grave jolt
fallen slateBOT
#

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

unquoted_string
spice pecan
#

Yeah I thought of that

#

Instantiating for each decorator call is such a performance hit /s

grave jolt
spice pecan
#

Each @attrgetter('__name__') would instantiate a new attrgetter, which is just an intolerable performance penalty

grave jolt
spice pecan
#

Yeah, but for each function you define, you'd have a separate one

grave jolt
#

ah

#

yeah

spice pecan
#

My joke was based on that

grave jolt
#

smh why have more than 1 function

lusty scroll
#

how many temporaries can we waste in a single expression?

surreal sun
#

temporaries just being short-lived objects?

lusty scroll
#

!e
yeah, is there some magic to reuse things like empty lists/tuples? like there is with small(ish) integers?

from functools import reduce
_, _, _, _, _, _ = [*reduce(type([]).__add__, map(list, (((1,),)+((2,),)+((3,),)+((4,),)+((5,),)+((6,),))))]
print(f"{_=}")
fallen slateBOT
#

@lusty scroll :white_check_mark: Your eval job has completed with return code 0.

_=6
lusty scroll
lusty scroll
vast saffron
lusty scroll
#

!e
in a similar vein:

import operator, types
@operator.itemgetter(0)
@operator.attrgetter("__args__")
@operator.itemgetter(0)
@operator.attrgetter("__orig_bases__")
class A(types.GenericAlias(list, "Hello World!")):
    pass
print(A)
fallen slateBOT
#

@lusty scroll :white_check_mark: Your eval job has completed with return code 0.

Hello World!
lusty scroll
vast saffron
# vast saffron I get the feeling <#470884583684964352> can't be contained! πŸ˜„ /s

This was a joke, I actually like to get to know those "cursed" tricks.

I call them python crimes and it actually quite enjoyable to do stuff that is unpythonic tricks.

Like an enum implementation that keeps a registry of subclasses, is a singleton and assignes itself automatically to other objects as category tag.

enum implementation:
Meta class that defines all necessary methods as class-methods (dunder), so they are never instantiated.

#

damn code to long to post the criminal meta-class

lusty scroll
grave jolt
#

speaking of

#

Is there a flake8 plugin that prohibits creating a metaclass?

#

I will write one if there isn't one

lusty scroll
#

arw you supposed to compile C extensions with -std=c99 ?

grave jolt
#

I am truly amazed how C programs even work, given that there are no namespaces

vast saffron
# lusty scroll that's just the built-in `Enum` class ...

Extended but yes. Is the way I learn best, seeing something and trying to recreate the effect without copying the code.

I also hate using black-box libraries, so this stuff also helped me understand their code.

it is still absolutely useless though (the stuff I made not stdlib Enum)

lusty scroll
#

these rules could be expanded to PEP 8 as well, imo:

Use 4-space indents and no tabs at all.
No line should be longer than 79 characters.

#

though I use two-space tabs in my personal code (with appropriate shame)

#

(I think I'm ok because it also says rules are there to be broken)

vast saffron
#

I understand the reasoning behind 79 characters width, but is there actually anyone that preferers this?

Even reading code at that width is kind of confusing for me.

lusty scroll
#

with 4-space indents it adds up to 79 pretty fast

rich cradle
#

That's one of (or maybe the only?) PEP 8 rule I absolutely reject, I am a fan of 100 or 120

vast saffron
lusty scroll
#

my phone screen can show only 78 columns, at least at what I consider a reasonable font size

flat gazelle
#

Afaik no. I think it was just chosen for maximum compatibility, not maximum utility to the average programmer

#

Actually enforcing it doesn't really make sense IMO

grave jolt
#

Long lines are just inconvenient to read in some contexts, like when looking at diffs side-by-side

#

but maximum line length isn't very important IMO

flat gazelle
#

Ye, there should probably be some limit

#

But 79 is too little

grave jolt
#

if your average line length is 70, then it's a mess

#

because there's clearly too much stuff happening in one line

peak spoke
#

it does make sense as a guideline for the code complexity, but cutting it off at it will imo lead to less readable code in most cases

lusty scroll
#

maybe it comes down to punch card size

#

where does it say 49?

#

PEP 7 and 8 say "No line should be longer than 79 characters" and "Limit all lines to a maximum of 79 characters" respectively, unless my eyesight is worse than I thought

grave jolt
lusty scroll
#

oh, haha

#

Well.. I expect discord to do the line-wrapping ping for me, so I wasn't being measured with line length πŸ˜„ But with email, can't assume even that much

#

(48 characters)

#

challenge accepted, @vast saffron

vast saffron
#

lets go back to the days of banishing consonants from variable names for size πŸ™‚

lusty scroll
unkempt rock
#

hi how can i run the codes that i copied in cmd?

signal tide
#

much more readable imo πŸ˜„

raven ridge
#

psh, lowercase is so 1970s.

# 
FR TM N LST:
    PRNT(TM)
surreal sun
#

are you talking about sorted?

#

I think that uses the timsort algo which is faster than the other two (I think)

#

correct me if i'm wrong

peak spoke
#

why wouldn't it use something a bit more complex if it performs better for the general use case

spark magnet
#

it uses a better algorithm

surreal sun
#

is it the timsort algo? or something else

peak spoke
#

python's already not great with speed so having it in the stdlib just makes sure users don't have to worry about picking a right one for the data (and python) themselves for most of the time

spark magnet
#

yes, list.sort uses Timsort

signal tide
feral cedar
#

it does use mergesort, but it switches to binary insertion sort for small granularities. this turns out to be much faster than just a normal mergesort

boreal umbra
#

!otn s bogosort

fallen slateBOT
#
Query results

β€’ bogosort-vs-timsort-the-final-showdown

boreal umbra
#

still waiting for that showdown.

white nexus
unkempt rock
#

Guys, I need idea for my semester end project idea..any suggestions ?

spice pecan
#

I vaguely remember something about Java using a modified version of timsort for object arrays, with the reasoning somehow being connected to pointers

#

Seems like Rust, Swift and V8 also have timsort, though I don't know if it's the default in any of those

grave jolt
spice pecan
#

Nice

prime estuary
#

The key part of Timsort is that it detects and takes advantage of already partially sorted sections in the list, which happen often in real situations and make it really fast.

verbal escarp
#

i'm annoyed by the pattern

def f(...,  excluded_names: set[str] = None, ...):
    if excluded_names is None: excluded_names = set()
#

is there some way to change the behaviour of mutable defaults? πŸ™‚

prime estuary
#

Well, right now on Python-dev there's a massive discussion over PEP 671, which adds a late binding default.

verbal escarp
#

if it was a class, the answer would be dataclass or attrs

verbal escarp
prime estuary
verbal escarp
#

i'm just looking at the pep itself, recalling it's been discussed here a while ago

prime estuary
#

Adding up the threads, it's 488 messages....

#

Did not realise it had gotten that much discussion.

verbal escarp
#

neither, although that might also indicate a "we can't come up with a viable solution to make everyone happy, let's just leave it as is"

lusty scroll
#

it appears I was able to get a much quicker subprocess run using os.posix_spawn than os.system or subprocess.run

verbal escarp
#

does it work on windows? πŸ˜‰

lusty scroll
#

no :p

#

I'm pretty sure at least

#

echo -E $'#include <stdio.h>\nint main() { printf("Hello World!\\n"); }' | gcc -xc -static-libgcc -static -O3 -o simple_print - ; python3 -m timeit 'import os; os.system("./simple_print")' | grep -vFe Hello ; python3 -m timeit 'import os; os.posix_spawn("./simple_print", ["./simple_print"], {})' | grep -vFe Hello ;

100 loops, best of 5: 3.62 msec per loop
2000 loops, best of 5: 173 usec per loop

verbal escarp
#

wait, what is that gcc doing there?

lusty scroll
#

building a simple, optimized, static "Hello World!" program to use as our test subject

verbal escarp
#

what's os.system?

lusty scroll
#

it just calls a shell with your command line as a single string

#

I'm not sure if it waits for it to complete

verbal escarp
#

The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function. See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes.

lusty scroll
#

!d os.system

fallen slateBOT
#
os.system 

os.system(command)```
Execute the command (a string) in a subshell. This is implemented by calling the Standard C function `system()`, and has the same limitations. Changes to [`sys.stdin`](https://docs.python.org/3/library/sys.html#sys.stdin "sys.stdin"), etc. are not reflected in the environment of the executed command. If *command* generates any output, it will be sent to the interpreter standard output stream. The C standard does not specify the meaning of the return value of the C function, so the return value of the Python function is system-dependent.

On Unix, the return value is the exit status of the process encoded in the format specified for [`wait()`](https://docs.python.org/3/library/os.html#os.wait "os.wait").
verbal escarp
#

you're not handling the output within the code itself, so that's probably the reason why it's faster

lusty scroll
#

the windows equivalent seems to be _winapi.CreateProcess

#

which is undocumented ?

lusty scroll
lusty scroll
verbal escarp
#

see telegram πŸ˜‰

lusty scroll
#

you got me on that, I didn't do anything "practical" like waiting for it to finish.. just thought the nuts and bolts were interesting between the two approaches

grave jolt
#

Why can't this be a decorator?

# 
@default(some_list = lambda: [])
def f(some_list):
    some_list.append(42)
    return some_list
#

like, in functools

#

or even py @default(some_list=list) def f(some_list): some_list.append(42) return some_list like in defaultdict

native flame
# grave jolt Why can't this be a decorator?

i was just writing up something similar, lol

In [92]: _SENTINEL = object()
    ...: def delay(*params):
    ...:     def deco(func):
    ...:         if not set(params) <= inspect.signature(func).parameters.keys():
    ...:             raise TypeError("some args missing in signature")
    ...:         sig = inspect.signature(func)
    ...:         dic = {param: sig.parameters[param] for param in params}
    ...:         new_params = [
    ...:             v
    ...:             if k not in params
    ...:             else inspect.Parameter(v.name, v.kind, default=_SENTINEL, annotation=v.annotation)
    ...:             for k, v in sig.parameters.items()
    ...:         ]
    ...:         sig = sig.replace(parameters=new_params)
    ...:         func.__signature__ = sig
    ...:         @wraps(func)
    ...:         def inner(*args, **kwargs):
    ...:             actual = sig.bind(*args, **kwargs)
    ...:             actual.apply_defaults()
    ...:             for k, v in actual.arguments.items():
    ...:                 if v is _SENTINEL:
    ...:                     actual.arguments[k] = eval(dic[k].default)
    ...:             return func(*actual.args, **actual.kwargs)
    ...:         return inner
    ...:     return deco
    ...:

In [93]: @delay('b')
    ...: def f(a: int, b: set = "set()"):
    ...:     print(f"{a=}, {b=}")
    ...: f(1)
a=1, b=set()

In [94]: @delay('l')
    ...: def f(l: list[int] = "[]"):
    ...:     l.append(5)
    ...:     print(l)
    ...:

In [95]: f()
[5]

In [96]: f()
[5]
grave jolt
native flame
#

achievement unlocked

#

i am half-tempted to publish this to pypi for the lolz

grave jolt
sharp plover
#

Has anyone tried out Pony ORM? As part of its implementation it contains a function to reconstruct the AST of a generator from its bytecode. I thought this was really cool. ```py

from pony.orm.decompiling import decompile
g = (x for x in range(10))
ast, _, _ = decompile(g)
GenExpr(GenExprInner(Name('x'), [GenExprFor(AssName('x', 'OP_ASSIGN'), Name('.0'), [])]))

native flame
#

referencing previous arguments is also possible now: https://paste.pythondiscord.com/awinunikuh.py

In [107]: @delay('x', 'l')
     ...: def f(x: list = "[1, 2, 3]", l: int = "len(x)"):
     ...:     print(x, l)
     ...: f()
[1, 2, 3] 3

In [108]: f([1, 2, 3, 4])
[1, 2, 3, 4] 4
sharp plover
native flame
#

yeah ast.unparse can do the ast -> source code part

native flame
#

oh

lusty scroll
verbal escarp
lusty scroll
quick snow
#

is a firm believer in the 80 line rule. Being able to have two files open side-by-side (even on a laptop) is great.

verbal escarp
#

other times it's great to have everything you want to say in one line, even if it's over 120 chars..

peak spoke
#

feels like that should be fine even with larger limits considering I usually have two files open and a big tab to the left of them just fine and my usual hard line limit is 120

quick snow
#

To be clear: I don't have a hard limit at 80, but my editor warns me if I exceed it.

peak spoke
#

I have visual guides on 79 and 99

verbal escarp
#

theoretically you could have a hard limit at 80 with black for everything living on github but run black on checkout with the limit of your desire locally

quick snow
#

Can you show me an example of where you need 120 columns?

#

Whenever I see this it's almost always either bad names, or overcomplex code

peak spoke
#

usually docstrings

verbal escarp
#

hmm.. i think i mostly get into fights over this with @lusty scroll with strings

peak spoke
#

but some code is also just simpler if it's on one line instead of being split up just for the sake of it

verbal escarp
#

not docstrings per se, but any strings

quick snow
#

Docstrings are text and can be hard-wrapped

#

Other strings can be an exception for better greppability

peak spoke
lusty scroll
#

I don't really like any of the options except shortening the string

peak spoke
#

There are also some cases when working with black where I get why it exploded something, but I really don't think it provides anything over a longer line (e.g. an exploded call that's around 5 similar calls but is a few characters longer)

quick snow
#

I feel like a style guide is not a convincing argument given that we're arguing over style

peak spoke
#

that or just having a summary/description part where the subject summary span multiple lines but sometimes it can't quite fit while getting the whole point into it

lusty scroll
#

textwrap.dedent is not a bad solution to multi-line strings thatdon't make the file look like it's indented wrong

verbal escarp
#

i prefer one-longer-line comments over multi-line comments since they're additional info and not necessary to follow the code flow, but breaking the code flow for comments is detrimental to readability

visual shadow
#

If there needs to be a long string literal, there needs to be a long string literal.

peak spoke
visual shadow
#

For comments break it into multiple lines. It's not an issue for readability, at best might take you a couple days

verbal escarp
peak spoke
#

compared to dedent, or compared to just not doing it?

visual shadow
#

At worst, perhaps a week to get used to it

verbal escarp
lusty scroll
verbal escarp
#

i always get itchy when i see those πŸ˜‰

lusty scroll
verbal escarp
#

yeah, that's probably the only thing we really disagree about code style

#

actually, black's line limit thing can also backfire in the opposite direction

unkempt rock
#

Hi

#

I need some help

verbal escarp
#

in justuse we use >> pipes in a few places, but it's nicer to have every pipe on its own line for clarity.. turns out that black disagrees and joins those lines to make a longer line, crippling readability for those

unkempt rock
#

Am I on the right place

verbal escarp
grave jolt
#

interesting phenomenon: if you break a complex statement into two statements, you end up having less lines, not more

foo = some_thing.complex_thing(
    bar=barify(settings.baz, consistency_threshold=420.69),
    fizz=Buzz(),
)
#->
bar = barify(settings.baz, consistency_threshold=420.69)
foo = some_thing.complex_thing(bar=bar, fizz=Buzz())
verbal escarp
#

except when it's not really a complex statement but some longer attribute lookup or proper parameter name

#

where you end up with ( and ) and [ and ] on seperate lines and instead of 1 line you have 5

native flame
#

re: late bound arguments
would you expect this to work:

def f(n: int => len(tup), tup = (1, 2, 3)): ...
#

considering that

def f(n: int => len(tup), tup => (1, 2, 3)): ...

should error (since late bound args would be evaluated left to right)

#

the pep itself is rather vague about this, it says # May fail, may succeed

verbal escarp
#

i was just fiddling around, but i figured i'd need some inspect voodoo:

# 
import inspect

def deferred(**defaults):
    def func_wrapper(func):
        def wrapper(*args, **kwargs):
            default_calls = {k: v() for k, v in defaults.items()}
            updated_kwargs = {k: v if v is not None else
                              default_calls[k] for k,v in kwargs.items()}
            return func(*args, **updated_kwargs)
        return wrapper
    return func_wrapper

@deferred(b=list)
def f(a, b=None):
    print(b)
    return b.append(a)
#

it also doesn't solve the issue with the proper documentation of the real default params

#

although, maybe it's possible to update the annotations via decorator

native flame
#

wait, no, that paste is outdated, that one doesn't preserve it

verbal escarp
#

i tried to improve on the str/eval approach

#

oh, yeah

#

that paste link looks good

#

although i wouldn't go the str/eval route

#

the signature updating looks well done

native flame
#

lambda: expr?

verbal escarp
#

nah, just require the value to be callable

#

and call it

native flame
#

oh lol

verbal escarp
#

benefit is you get immediate syntax errors and not sometime down the road

#

and IDEs can help with autocompletion

native flame
#

that is true

spice pecan
elder blade
#

As a user I would expect that to work though

quick snow
#

I really don't like the => syntax when combined with type hints. It looks too much like a callable type.

spice pecan
#

Referencing a param before declaring it (in source) is too confusing for my taste

native flame
#

i am on the side of not allowing it too

quick snow
# 
def double(fn : int => int):
    def wrapper(arg: int):
        return fn(arg) * 2
    return wrapper
spice pecan
#

Yeah, syntax could be better

pearl river
#

what PEP is this syntax for function types?

spice pecan
#

I don't know if it's in a pep yet, but it was mentioned a bunch of times during the discussion of the new peg parser

#

Either this or args -> return

native flame
#

oh wait nvm

#

i misread

quick snow
spice pecan
pearl river
#

Ah. To be honest, I feel like it'd be really annoying when having nested functions. ((int, int) => int, int) => (int => int)...

native flame
#

can't be worse than Callable πŸ€·β€β™‚οΈ

pearl river
#

Fn[int,int] => int, maybe?

#

totally not stolen from Rust

spice pecan
#

Callable is not that bad tbh, if we just had a bunch of common templates it would be fine

#

Like C# has Action<args...>, which takes args and returns void, Function<args..., ret>, which is basically Callable without the brackets, Comparison<T>, which takes two Ts and returns int, so on and so forth

grave jolt
spice pecan
#

That would look fairly interesting tbh

#

Assume None return for Fn[int, int] and override on rightshift

lusty scroll
spice pecan
#

That could work, but you could also just have type aliases

verbal escarp
#

i also wouldn't like => for the default syntax, thinking of better lambda syntax there

#

wait a sec.. what if we don't call it defaults and move a bit further

#

i just typed def fn(x: int ?= int) and thought about none-aware assignment, but what if that specified callable is always called on the argument that's passed in?

spice pecan
#

Using a single ? for None-aware assignment is still pretty iffy

verbal escarp
#

like for explicit type conversion

spice pecan
verbal escarp
#

if you pass in the function that is supposed to be called on the argument, it's explicit

spice pecan
#

Fair enough, but why not just call it on the argument yourself? :P

verbal escarp
#

hm

#

it's a tough nut

grave jolt
#

and then you can pass a lambda to ?=

#

and then use another ?= in the lambda

spice pecan
#

late-bound moment

#

I think := would be the most fitting option for late-bound syntax

verbal escarp
#

let's write everything as f-strings and eval() - because f-strings are cool :p

spice pecan
# 
f'{"write program as oneliner here"}'```
jovial flame
#

f that

grave jolt
#

oh, and you can call the function itself recursively in the lambda, right?

verbal escarp
#

let's imagine we have a nicer lambda syntax

#

then we could write def fn(foo: list[int] @ {=> [1,2,3]}) or somesuch

#

i think conceptually it's closer to what we mean with decorator syntax

spice pecan
#

It's matmul for the type

surreal sun
#

but ?= has a different meaning I think

#

I'm honestly looking forward to PEP 671 because
A. late default setting
and B. mutable arguments!

def fn(lst => list()):
  lst.append(1)

^ that won't be appending it to the entire default

verbal escarp
surreal sun
#

I'm rooting for ?= or :=

verbal escarp
#

but i'm still unsure if the deferred call has to be none-aware and couldn't just be called every time the parameter is passed in..

#

it could open up more possibilities

#

also could be more flexible with other sentinels

surreal sun
verbal escarp
#

it might require a slight change in how list() and other constructors work since ```python

list(None)
Traceback (most recent call last):
File "<pyshell>", line 1, in <module>
TypeError: 'NoneType' object is not iterable

#

if list(None) would return [], it'd be a trivial change

#

or make None iterable with no item?

spice pecan
#

I'd vouch for not tying it to None, because in some cases it is also a valid input

#

And not having to create sentinels for a fairly trivial param missing case would be very nice

peak spoke
#

Is the reason for typing.overload returning a function that raises NotImplemented avoiding potential accidents if the user forgets to define the implementation? I found it a bit strange that it doesn't just return the argument

surreal sun
peak spoke
#

and of course the type checker tries to be smart so I can't even monkeypatch it nicely

native flame
spice pecan
#

And raising NotImplementedError helps prevent that

empty kite
#

In Scala 3, you can define your program entry point with @main

@main def hello() = println("Hello, world")

which is a lot nicer than having to create an object and args parameter "just because":

object Hello {
    def main(args: Array[String]) = {
        println("Hello, world")
    }
}

Or even the dreaded

public static void main(String[] args) {}

Java requires. They both rely on beginners having to learn concepts that are of no immediate or apparent use for them to satisfy the compiler.

While not required, in Python, in many cases it makes sense to run your code inside a if __name__ == "__main__: guard. As far as beginners are concerned, this is equally, if not more so, arcane. Even the advanced Pythonista might think of it more as "useful introspection trivia" than a proper mechanism for program entry points.

So, what do you think of a main decorator you could slap on a function to execute it when it's in the "__main__" file? E.g.:

@main
def hello():
  print("hello world")

Is it possible? Useful? Needed?

gusty marsh
native flame
#

clever

gusty marsh
#

Looks something useful when you have short decorators and they are taking one whole line

lusty scroll
#

so we could skip adding module level code to run the "main" (or "run") function

#

not sure what the behavior would be--maybe it could all be handled by runpy

#

then there'd be no other changes needed except adding the decorator somewhere

#

looking at runpy.py, this won't work as I had hoped

#

it just calls exec , it doesn't create a module, so it wouldn't be just a matter of searching for a function "annotated" with @main

spice pecan
#

another language that does this is F# with its [<EntryPoint>]

#

And although it looks neat and shouldn't be difficult to implement, I don't see much value in having that

#

Python doesn't demand an entry point, top level code is the entry point

#

The need to differentiate between running a module as a script and importing it into another module doesn't arise often in general, and even less so in code written by beginners

#

And by the time they do need to differentiate, the explanation will not look arcane to them

#

I find it that if-main is shoved into code just because it is a concept familiar to people coming from other languages more often that it is genuinely needed

native flame
#

it's not too common to have modules that are meant to be both imported and run directly either

spice pecan
#

Yep, main code is usually separated into its own module

raven ridge
#

In Python, everything happens at import time. There is no time that code runs at other than at import time (and interpreter teardown time, I suppose). Essentially the entirety of a python program's run time is the import of the __main__ module.

People coming from other languages don't like that, and want to pretend Python is a language where you can define functions and globals in one phase and then use those in an entirely different phase, but that's fundamentally not the way the language works.

#

Or to put that a bit differently, Python already has an entry point, and it's __main__. Python just starts running whatever code is in the file or module you ask it to run, and that's all an entry point is. It's just that the entry point is not a function, but instead an entire module.

spice pecan
#

The solution to this "issue" is not an entry point decorator, but proper teaching practices that account for python's idioms instead of blindly copying over practices from other languages

#

And, frankly, a decorator may look less intimidating in code, but it is a significantly more complex concept than an if statement with string comparison (RE: forcing beginners to learn concepts that are of no immediate or apparent use to satisfy the compiler)

raven ridge
#

Probably 9 times out of 10, if someone wants a file that is able to be imported or run directly, and they want it to do different things in each case, the right thing to do is factoring it into a package with the importable stuff in __init__.py or a submodule, and the script in __main__.py - there's no good reason to keep both in the same file, once you've decided that what you're building is a library that supports being imported to provide some API

quick snow
#

FWIW:

def main(fn):
    if __name__ == "__main__":
        import atexit
        atexit.register(fn)

But I see this firmly in #esoteric-python territory

verbal escarp
#

it may be unusual, but far from esoteric

raven ridge
#

Running the "main" function as part of interpreter teardown is most definitely esoteric.

verbal escarp
raven ridge
#

Yes, it is. It takes the function that is decorated and registers it to be called after the main script ends, during the first step of finalizing the running interpreter

verbal escarp
#

it's a function that is registered

#

not the main function

#

fn could be a simple cleanup function

quick snow
#

Esoteric in the sense of "don't do this"

raven ridge
#

The stated purpose of this decorator was to explicitly specify an entry point for your script. Running your entry point while the interpreter is exiting is most definitely esoteric.

verbal escarp
spice pecan
#

It's very much implied in the conversation that it is a part of

raven ridge
verbal escarp
#

as i read it, that part of the conversation was done, but okay ^^

#

i see a much different problem with@quick snow 's snippet though

#

aside from the exit-entry thing, registering with atexit calls the registered functions in backwards-order as they were registered, so if you really want to use this pattern and have a proper atexit-cleanup function, you'd need to be careful in which order those functions are registered, otherwise you end up with exiting before entry

#

and if you decorate multiple functions, you're screwed either way

quick snow
#

I wonder what happens when you register an exit function in an exit function

spice pecan
#

That's a curious question

#

!e ```py
from atexit import register

register(lambda: register(lambda: print("Hello")))```

fallen slateBOT
#

@spice pecan :warning: Your eval job has completed with return code 0.

[No output]
spice pecan
#

They are skipped it seems

verbal escarp
#

i doubt that even called atexit

spice pecan
#

why wouldn't it?

verbal escarp
#

let's try

#

!e ```python
from atexit import register
register(lambda: print("foobar"))

fallen slateBOT
#

@verbal escarp :white_check_mark: Your eval job has completed with return code 0.

foobar
verbal escarp
#

okay, you're right

#

!e ```python
from atexit import register
register(lambda: print("foobar"))
register(lambda: register(lambda: print("Hello")))

fallen slateBOT
#

@verbal escarp :white_check_mark: Your eval job has completed with return code 0.

foobar
verbal escarp
#

yup

verbal escarp
native flame
#

hmm nevermind not sure what i was thinking

spice pecan
#

Accepting None as a parameter would weaken the typing as it could silently produce the wrong result instead of erroring

verbal escarp
#

i suggested to have list(None) -> [] to make it possible to call it as part of a default parameter

#

delayed default*

spice pecan
#

Yes, and I still think it would weaken the typing

verbal escarp
#

making None iterable with zero elements is arguably worse, i think

spice pecan
#

It definitely is, but that doesn't make the alternative better than it already is

#

IME, passing None in such cases is not intended and a result of a bug more often than not, and changing it to return an empty list makes it harder to spot

#

That might just be my experience, but I'm opposed to such changes

verbal escarp
#

so, what would you suggest as a delayed-default mechanism?

#

aside from the syntax discussion, it's also relevant how it should work

quick snow
#

I'm just opposed to the proposal

spice pecan
#

An abstracted away sentinel (to allow passing None) with the semantics described in the pep, so LTR evaluation in a block of code before the function body

spice pecan
#

It'd introduce a decent amount of new semantics that would be confusing for newcomers

#

And I wouldn't say it solves a huge issue, it's just convenient sugar

#

I think the fact that the community has wildly different opinions on syntactic options alone is a good sign that this should be reconsidered

verbal escarp
#

bikeshedding isn't a good way to make progress πŸ˜‰

native flame
spice pecan
quick snow
verbal escarp
spice pecan
#

and AFAIK syntax is the main point of disagreement, the rest of the semantics is mostly established already (among people who want it implemented, ofc)

spice pecan
verbal escarp
#

it's not just convenience. having None or another sentinel as default then calling set() etc. as actual default is hiding the actual default from the documentation

spice pecan
#

I'd rather have none-aware operators reduce boilerplate

spice pecan
#

The signature becomes less meaningful

#

Type hints partially solve that

#

If you see list[int] | None, you get a bit of information from the signature

#

Though you don't get to see the actual value of it

#

And neither can you get things like len(other_arg)

#

You could use Annotated I guess

# 
def func(items: Annotated[list[int], "Defaults to []"] = None): ... ```
#

I think something like HasDefault[T, default] would be nice, along with maybe typing.get_defaults that would return a dict of argument names to default values, taking into account whether or not they're typed as HasDefault

raven ridge
quick snow
#

Shouldn't the name items be enough of a hint?

elder blade
native flame
#

i forgot pablo galindo is your colleague πŸ‘€

verbal escarp
native flame
#

godlygeek

verbal escarp
#

ah

halcyon trail
#

did this channel get renamed from advanced?

swift imp
#

Speed ups for 3.11 aren't looking that impressive. Maybe I'm not understanding how much faster its supposed to be after a single year

elder blade
#

What kind of speedups would be impressive?

#

Say Python becomes as fast as X, what's the slowest X you can think of?

swift imp
#

I would like it to approach javascript speeds, which can be as fast as C at times.

magic nova
#

1.19x = 19% faster

halcyon trail
#

a 1.19x faster geometric mean isn't 19x faster

swift imp
#

Sorry he's saying 19 percent

halcyon trail
#

yeah, small difference πŸ™‚

#

19% is a nice win but it's not earth shattering when python is 20x as slow as many other garbage collected languages

#

and yes, I mean 20x, not 20% πŸ™‚

swift imp
#

Shaving 10s of ms off of basic ops is nice but when you're normally in 100s of ms to begin with I'm not really feeling that "wow'd" about it.

magic nova
#

this is just the first of the improvements in the pipeline

swift imp
#

I'd like to see ms -> us type improvements

halcyon trail
#

I mean that's 1000x

swift imp
#

yeah...

halcyon trail
#

that can't happen because python is only roughly 100x slower than C or C++

swift imp
#

I mean I know jack-butt about doing any of this but like javascript is damn near, if not more, dynamic as python and can be as fast as C at times

peak spoke
#

The improvements aren't magical, you can't really do anything extreme without restructuring how the interpreter works as a whole

halcyon trail
#

"can be as fast as C at times" in reality means, as fast as C, almost never

#

"typically", JS is probably going to be at least 10x slower than C

peak spoke
#

js also had a lot of money poured into it as the whole web runs on it

halcyon trail
#

which is is still an amazing accomplishment

swift imp
halcyon trail
#

eh it's ok, I've seen lots of benchmark threads come and go, in the end it depends which benchmarks you choose, how you weigh them, how you run them, etc

#

the ballparks I'm giving are based on many many benchmarks I've seen over time

#

So if you designate C/C++ (and probably Rust) as 1x, you have python/ruby ~ 100x, Java 2x-3x (with caveats; like latency vs throughput), most other statically typed GC languages falling in about a 5x-10x range, etc

swift imp
#

Well these are just straight up qsorts, and they even use numpy and python is still really slow but javascript is keeping somewhat relative to C

#

Which I think speaks volumes to how much overhead python itself just adds

halcyon trail
#

I'd have to look at the benchmark and understand what is happening, my guess is that if JS is keeping up with C it's because something is implemented in C lol

swift imp
#

So this was C without any compiler optimizations

halcyon trail
#

LOL

#

what is the point of such a comparison?

#

or did the author just not have enough experience with C?

swift imp
#

They did C with optimizations, js, C without optimizations, and Python

halcyon trail
#

so you're saying JS is similar to C without optimizations?

swift imp
#

js is was faster than C without optimizations by like 200 ms which including the setup time for java. The optimized C was well over 2x as fast

#

Yeah

#

Which sure I guess fine but it was still a magnitude faster than python with numpy

halcyon trail
#

okay, I mean nobody really cares how fast C is without optimizations.
Yeah, python is definitely slow. JS isn't really as a language designed better for speed afaik but like Numerlor said, a ton of money and time got poured into optimizing it

#

I mean the number of man hours that went into optimizing V8, probably dozens of excellent C++ programmers at Google worked on this for years

#

compared to python where you have 1-2 people working on it in their spare time

swift imp
#

lets just hope microsofts funding for Guido and Shannon make it really fast

#

I think they have 3 full time developers funded

halcyon trail
#

it's just that python is in a rough position vis a vis performance, it's dynamically typed and it doesn't use an LLVM approach a la Julia, so it's really just going to be piles of work, to make it say, only 20x slower than C

#

meanwhile if you start any project that vaguely cares about perf (that isn't data science-y) you have to justify why you aren't using pretty nice, easy, GC languages that are way faster than python πŸ€·β€β™‚οΈ

swift imp
#

I think people still favor python for stuff that is IO bound, if its not data-science-y task

halcyon trail
#

well, depends which people? Some people favor python, some people favor java, some people favor Go, etc