#internals-and-peps

1 messages ยท Page 95 of 1

unkempt rock
#

i need a particulars py requirement.txt

grave jolt
#

@unkempt rock pip doesn't remember the history of your installations -- it only knows which packages are installed. So the best solution would be to just look at the imports and reconstruct the required packages from that.

#

If you have installed packages A, B, C, where A depends on B and B depends on C, there are 4 possibilities:

  1. You only use A directly -- if it stops depending on B, you don't need B.
  2. You use A and B direcrtly, but you don't use C. So if B stops depending on C, C doesn't need to be installed.
  3. You use A and C directly, but you don't use B. So if A stops depending on B, B can be dropped, but C should be kept in either case.
  4. You use A, B, C directly -- they all need to be installed

So here pip can't decide whether requirements.txt should contain A, A, B, A, C or A, B, C

unkempt rock
#

k

#

but there are some tools

#

like pipreqs

#

and pip freeze

sacred tinsel
#

looks like pipreqs does try to reconstruct the reqs.txt from imports, but I've never used it

deep crow
#

I m trying to run a powershell script which should over a file select natively. If I run PS>powershell.exe ./myfile.ps1 , I get a window but if I do it with subprocesses I don't

#

it's rather printing those strings. I have asked on #kivy or IRCNode but people are avoiding windows discussions

pliant tusk
#

It may be easier to just open the file selector natively thru python

pliant tusk
#

nevermind win32 api is garbage

spiral willow
#

Subprocess may not be started in userland

grand shore
#

Hey. I am thinking about extending dataclass with some optional param in field which determines that class field is returned by asdict function or not (similar to init=False). Should I create PR in GitHub or how can I do it properly?

median yarrow
#

pantry

spark magnet
grand shore
spark magnet
#

Some people might say, "sounds like you want a schema and serialization tool, use one of those," for example.

grand shore
spark magnet
#

yeah, maybe it won't be.

#

Python-Ideas can be unpredictable ๐Ÿ™‚

grand shore
strange heath
boreal umbra
#

Am I getting my Python history right? I believe I read that map, filter, reduce, and lambdas were all implemented by a Lisp user who wanted them, and Guido or whatever powers that were at the time just accepted it since the person had already implemented it. If that's all true, did the key keyword argument already exist for comparison functions? were people using the callables in operator at that time?

strange heath
#

dude i didn't even know who guido was until today

#

......

#

no comment

spark magnet
#

@strange heath you don't have to reply, "i have no idea, i'm dumb" to the discussion that happens here ๐Ÿ™‚

strange heath
#

ok

boreal umbra
#

The reason I ask is that reduce is the only of those three functions that I don't have any reservations about liking, and when you consider that there are alternatives to creating sorting keys, I can start to see why some people wish lambdas weren't in the language.

spark magnet
#

i didn't know that some people wished lambdas weren't in the language.

boreal umbra
#

maybe I'm reading too much in to those who say they're "not pythonic"

astral gazelle
#

You mean the current form of lambdas or lambda functions in general

raven ridge
astral gazelle
#

Cause i dont particularly like the current form of lambdas

boreal umbra
#

I wasn't aware that lambdas had undergone any non-implementation changes

raven ridge
#

list.sort(key=...) was added in 2.4

boreal umbra
#

another thing: why are map and filter built in but not reduce? I would put all three in functools

#

if they ever do python 4, these are the kinds of cleanup I want to see

raven ridge
boreal umbra
#

(I also want to see upper camel cased class names used consistently in the builtins/stdlib and the removal of lower camel case where it exists)

raven ridge
#

there's too much Python 2 code out there to even be thinking about Python 4 ๐Ÿ˜…

boreal umbra
feral cedar
#

python 2? who's she

raven ridge
#

I'm still writing Python 2 + 3 compatible code, because I write libraries, and there are groups in my company that are still trying to get off Python 2. We're hoping for the end of the year...

spark magnet
swift imp
sacred yew
#

map is one of my most used builtins, it would be annoying if it was moved to functools

swift imp
#

Really?

#

A list comprehension with zip?

sacred yew
#

imo map(int, lst) looks better than (int(x) for x in lst)

swift imp
#

Reduce is really hard to emulate with a a comprehension, filter and map arnt

feral cedar
#

is it even possible?

#

how would that even work, without just creating a list comp with side effects

swift imp
#

Probably not, which is why I think it should be builtin but filter and map shouldn't

boreal umbra
# spark magnet should `int` be `Int`?

it wouldn't ruin my day if int became Int, though I suppose one could argue that classes that represent primitive-like types can be exempt. I'm not sure where one would draw the line (str?).

sacred yew
#

if you're going to move it, filter should be moved to itertools instead of functools

raven ridge
#

bytes? bytearray?

boreal umbra
#

I'd be fine with map and filter going to itertools.

sacred yew
#

map(some_func, lst)

#

often some_func is str/int

boreal umbra
#

I'd still just use a comprehension of some kind

gleaming rover
#

I would love a natively curried map etc.

#

but itโ€™s not going to happen

boreal umbra
#

Curried?

gleaming rover
# boreal umbra Curried?

the transformation of a function taking multiple arguments to one that successively takes single arguments and returns a partially applied function

sacred yew
#

so you can do seq_to_int = map(int) and then do seq_to_int(["2","3","4"])

raven ridge
#

you can do that using functools.partial

boreal umbra
#

One could have that backwards compatibly by making the iterable argument optional, yes?

raven ridge
#

sure, but - why special case that just for map? We already have partial for currying.

boreal umbra
#

I guess. I've never used partial

gleaming rover
#

but it's inherently less FP-friendly

#

not just map

#

there are a few reasons for this, but one of them is nicely pipeable functions (ReactiveX uses this pattern a lot)

raven ridge
fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

[2, 3, 4]
gleaming rover
# boreal umbra Curried?

example (not Py):

this.gameService.load().pipe(
    filter(game => game.gameTime > 60),
    map(game => game.homeScore),
    reduce((acc, next) => acc + next, 0)
)
#

so you can read top down

#

as opposed to right to left (because functions would be applied inside out)

raven ridge
#

you can do that with Python, arguably more readably - though it forces you to name the intermediate steps.

games = self.game_service.load()
long_games = (g for g in games if g.time > 60)
home_scores = (g.home_score for g in long_games)
return sum(home_scores)
spark magnet
#

"encourages" you to name the intermediate steps.

#

whenever someone asks, "here's a long line, what's the right way to format it," i always answer, "use more, shorter, statements"

gleaming rover
#

but, again, less FP-friendly

#

also

#

it's not really clear from that snippet, but the result can be passed elsewhere and further processed

#

so it's not really a generator thing

#

and use of anonymous functions like that is quite a common FP idiom

#

I would say that naming every step can add noise

#

as an aside: similar Scala code, but with method chaining:

# 
games
    .filter(_.gameTime > 60)
    .map(_.homeScore)
    .reduce(_ + _)
spark magnet
# 
return sum(g.home_score for g in self.game_service.load() if g.time > 60)
#

I certainly hope no one reaches for reduce in order to sum things.

gleaming rover
#

I hope not, either

#

but it was the simplest illustration I could find

#

that was still meaningful ๐Ÿฅด

raven ridge
#

the Python version with named generators is so much easier for me to read than that. I'm sure part of that is familiarity, but, still.

limpid forum
#

when I had group from groupby I did sum(1 for _ in group) (because groupby group doesn't have len)...

#

thankfully i could later change that to actual elem.get("qty", 1) because we introduced a field... wait, no... it was a string (because a custom field in a tool was always a string) and could be empty... ugh, it was int(elem.get("qty") or 1) or 1 or sth like that ("0" string would be True but int would return 0`)

gleaming rover
#

although Scala has especially terse lambdas, to be fair

#

the thing is that with RxJS (at least) you can't name your steps

#

the abstraction doesn't work that way

#

well, you could name the individual functions, but that would be both misleading and increase nonlocality of reference (not sure if the right term)

#

but anyway, one really nice thing about writing code like that is that it's quite easy to scan what operation is being performed at each step

#

because the operators are aligned

raven ridge
#

I think the Python version using named generators is pretty readable, albeit more verbose.

gleaming rover
#

I would say the benefit of having names is that the intended meaning of each step is clear

#

so it depends on whether you want to emphasise the intended result, or what processing is actually being done

#

but anyway, different paradigms

red solar
raven ridge
#

what do you mean when you say "dependency on"?

#

Looks like it's used in ctypes, and in attempting to detect whether a stack overflow is about to occur on Windows.

red solar
#

hmm

#

ig would it be possible to build cpython with a compiler that doesn't have alloca?

raven ridge
#

hm. I don't think so - it looks like ctypes unconditionally uses it.

#

though, now that I think of it, the interpreter itself may build, and just not provide the ctypes module.

red solar
#

if that's an option for modules, that's pretty cool

raven ridge
#

modules are separate shared libraries, and if one module fails to build it doesn't stop others from being built.

blissful comet
sleek marlin
# 
    if str(message.author.id) in bonk:
        pass
        #print('found!')
    end = time()
    print(end - start)
    
    start = time()
    if str(message.author.id) not in bonk:
        pass
        #print(message.author.id)
       # print(type(message.author.id))
    end = time()
    
    print(end - start)

Output
5.817413330078125e-05
2.9802322387695312e-05
#

How are if not statements faster

raven ridge
limpid marten
#

Consider using the timeit module for better testing.

raven ridge
#

!e ```py
import timeit
print(timeit.timeit("1000 in lst", "lst = list(range(1000))", number=10000))
print(timeit.timeit("1000 not in lst", "lst = list(range(1000))", number=10000))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | 0.3987090536393225
002 | 0.3824900588952005
raven ridge
#

!e ```py
import timeit
print(timeit.timeit("1000 in lst", "lst = list(range(1000))", number=10000))
print(timeit.timeit("1000 not in lst", "lst = list(range(1000))", number=10000))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | 0.3420707928016782
002 | 0.26386456098407507
raven ridge
#

!e ```py
import timeit
print(timeit.timeit("1000 in lst", "lst = list(range(1000))", number=10000))
print(timeit.timeit("1000 not in lst", "lst = list(range(1000))", number=10000))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | 0.31232740404084325
002 | 0.2853736230172217
raven ridge
#

!e ```py
import timeit
print(timeit.timeit("1000 in lst", "lst = list(range(1000))", number=10000))
print(timeit.timeit("1000 not in lst", "lst = list(range(1000))", number=10000))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | 0.37548045301809907
002 | 0.44427160965278745
raven ridge
#

Clearly just random variation even trying lots of times, but you can see that they're very close. They're doing exactly the same amount of work.

limpid marten
#

If you were to create your own list using a bloom filter, you could probably get good performance for the not case.

raven ridge
#

If you're gonna use hashing, you may as well use a set, and then both cases are O(1)

sleek marlin
#

lemon_sweat I'm not there yet 0(1)

thorn thorn
#

how can i make my script to run an url, save the image from there and post it to twitter every day?

severe lichen
#

What I really hate about list comprehensions, there's no way to do that pattern

for i in iter:
  if func(i):
    return i
raise ValueError()

Any is similar, but something like first(iter, key=func) or first(i in iter if func(i)) would be neat.

native flame
#

next(i for i in iter if func(i))?

severe lichen
#

Hm, that's good, i don't use next() and iter() often. StopIteration instead of ValueError, but you can always catch and re-raise.

#

Another, rarer, pattern is

lst = []
for i in iter:
  if not func(i):
    break
  lst.append(i)
return lst

Something like len([val for val in [3, 2, 1, 0, -1, 0, 1, 2, 3] while val > 0]), for example.
EDIT - disregard that, forgot about itertools.takewhile.

torpid solar
#

I am reading a hindi language text from an excel sheet using opepyxl module and the output is not in hindi characters

#

Somebody please tell me what to do

boreal umbra
torpid solar
#

Okay sorry

sleek marlin
#

I personally think this can go here because it talks about under the hood stuff

smth = get smth from api

if smth exist:
assign = assign
Do_smth_(assign)
else:
assign = assign
Do_smth_(assign)

assign = assign
if smth exist:
assign = assign
Do_smth_(assign)

if smth exist:
assign = assign
else:
assign = assign
Do_smth_(assign)```

#

What do you guys think is better

echo lake
#

third ig

sleek marlin
#

@nova iris
@feral cedar
NOW FIGHT which is better

feral cedar
#

what

#

no

sleek marlin
nova iris
sleek marlin
#

Most people say third here too

nova iris
#

lol yayyy

#

i mean, psvm argues that second one is better since there's less duplicate code

#

but i think third is good

#

bc third is more concise, cleaner, and more understandable @sleek marlin

astral gazelle
#

no ternary option

feral cedar
native flame
#

yeah ternary would be pretty compact do_smth(assign if smth_exist else assign)

sleek marlin
#

I do smth else too brainmon

nova iris
#

wait the teacher is telling us to do work ๐Ÿ˜ฆ

#

cy'all later

nova iris
#

this chat died so hard

undone hare
#

This channel is about the quality of the messages, not the quantity

undone hare
abstract finch
#

ok

unkempt rock
#

keys = open("wanttowrite.txt", "w")

white nexus
sharp panther
#

we shall put (Code Runner bot) to run the code in the server

grave jolt
fallen slateBOT
#

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

hi
grave jolt
#

Also, this channel is for discussions about the Python language itself. If you want to discuss this community, check out #community-meta

unkempt rock
#

Ok so does anyone know how I would make an OCR that can monitor my health in a video game like warzone?

unkempt rock
#

is this: def func(thing, thing2: int) faster than this?

def func(thing, thing2):
  thing2 = int(thing2)
pliant tusk
#

they do 2 different things

#

def func(thing: int) is a type hint that thing should be int

# 
def func(thing):
  thing = int(thing)``` converts `thing` to an `int`
unkempt rock
pliant tusk
#

no the first one does not convert anything

peak spoke
#

The annotation does absolutely nothing except being saved to __annotation__

feral cedar
#

discord.py does some kind of magic with them that converts them based on the annotations

radiant fulcrum
#

you can do alot with them

#

but python doesnt do anything with them by default

unkempt rock
#

OH

boreal umbra
#

@unkempt rock these are annotations. Originally, they were designed for storing arbitrary data, like def cookie_func(x: 'number of cookies', y: 'cookies eaten per minute') -> 'number of cookies needed': You could use them for whatever you want. But now it's recommended that you only use them for storing typing data. And one could create a Python runtime where that type information gives you performance optimizations, but no one has made that.

unkempt rock
#

ok

polar robin
#

Where is a good place i can start to learn python?

boreal umbra
sleek marlin
# 
S = 10 > 5 and "hi" or "bye"
print(S)``` how is this hi
gleaming rover
boreal umbra
#

Here's an idea I haven't thought about very thoroughly:

def some_func(arg): ...  # might raise ValueError or KeyError, otherwise returns Tuple[int, int]

match some_func(23):
    case ValueError: ...
    case KeyError: ...
    case (x, 0): ...

etc.

#

How bad is it?

spark magnet
#

@boreal umbra have you seen the pattern matching pep that was accepted?

boreal umbra
spark magnet
#

I'm fuzzy on many of the details myself.

raven ridge
# boreal umbra How bad is it?

That's already syntactically valid, and does something different than what you're proposing. So, your time to get that proposal implemented is limited to the next 2 months, until the 3.10 feature freeze.

boreal umbra
raven ridge
#

As things stand with the PEPs that have been accepted so far, py match some_func(23): case ValueError: ... would match unconditionally, and would result in the object returned by some_func(23) being assigned to ValueError.

#

considering how busy the pattern matching mini-language already is, I really dislike the idea of making it able to match on both the return value and the exceptions that might be thrown.

feral cedar
#

well, obviously, the solution is to implement a Result type

raven ridge
#

you could already do that. If you made the function return an exception instead of raise an exception, you could match on that with the syntax that's already accepted.

#

you could also just make your own Result type and then use pattern matching to unpack it ๐Ÿ˜„

gleaming rover
#

this reminds me of Try

unkempt rock
#

!e

fallen slateBOT
#

You are not allowed to use that command here. Please use the #bot-commands channel instead.

woven robin
#

Why does Python use len(list) instead of list.len or list.len()? I would expect lists to have a len attribute. Perhaps having that attribute would mean that mutating lists would become more expensive because you would also have to change this attribute?

native flame
#

the dunder list.__len__() works, which is what len(list) also calls under the hood I assume

woven robin
#

That doesn't help me understand why it is the way it is.

gleaming rover
#

or perhaps more authoritatively

feral cedar
gleaming rover
feral cedar
#

icic

toxic ermine
#

does anyone have worked on reading CSV file using python and pandas, and insert it into snowflake DB?

wide shuttle
#

The choice made here is that Python comes with built-in tools and mechanisms that you can provide support for in the types you implement.

#

Want support for len (and, if combined with some other dunders, support for other things like iteration), you implement a __len__ for your type.

#

In a way, this guarantees uniformity across all types: Is it .len .length .size?

#

No, it's always len(object), which is supported under the hood by the dunder method __len__, whether it's a str, list, or any object that's Sized, whether its type is built-in or user-defined.

unkempt rock
#

Is seek starting seeking from start in text file or there where bookmark is

raven ridge
# wide shuttle In a way, this guarantees uniformity across all types: Is it `.len` `.length` `....

I think this is the big reason. The fact that len exists makes it so that there's only one obvious way to get the length of a collection, and everything that's collection-like has to make itself work with len, because users expect it. If instead it was a method of the class, there would be no len builtin to work with, and no common base class across all collections, so nothing to enforce that the same method name would be chosen by every collection-like class.

raven ridge
wide shuttle
#

It also fits in nicely with all the other hooks you get to hook into Python's data model. There are a lot of "double underscore" (dunder) methods that you'll probably rarely, if ever, call yourself.

raven ridge
#

That's true, but it's not obvious why "length" ought to be part of the data model.

#

I can't think of any operations where the interpreter itself will call len on your object implicitly, like it does for most other dunders.

#

Even the legacy iteration protocol uses __getitem__ and IndexError, not __len__, IIRC

#

(yep, I double checked that)

wide shuttle
#

Maybe not, but does that make it a special enough to be a special case that breaks the pattern?

#

We do have the relationship between str -> __str__, repr --> __repr__, and so on

raven ridge
#

But the interpreter calls those automatically, that's my point. f"{x}" calls x.__str__, f"{x!r}" calls x.__repr__, {x} calls x.__hash__, etc. The thing that makes those methods part of the data model, instead of just regular methods, is that the language itself uses them.

#

I don't think that's the case for __len__. At least, I can't think of any time the interpreter itself ever calls __len__ on a user defined type.

unkempt rock
#

Fwiw, it. calls len in the legacy iteration protocol

In [1]: class Foo:
   ...:     __getitem__ = None
   ...: 

In [4]: reversed(Foo())
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-22e4a800c3a9> in <module>
----> 1 reversed(Foo())

TypeError: object of type 'Foo' has no len()
raven ridge
#

Ah, nice. Only for reversed, not regular forward iteration? That would make sense.

#

!e ```py
class C:
def getitem(self, i):
return i

it = iter(C())
print(next(it))
print(next(it))
print(next(it))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | 0
002 | 1
003 | 2
raven ridge
#

Yeah, forward iteration doesn't need it. Makes sense that reversed would, though.

wide shuttle
#

I'd say that general, uniform pattern is more important than strictly going by whether or not it's called internally somewhere (ignoring reversed)

unkempt rock
#

If I do seek(113) to start writing from 113th line in my text file, text is in 114th line, why this

radiant fulcrum
#

well without any context as to what seek is im assuming indexing is starting at 0 which is why its line 114 starting counting at 1

unkempt rock
#

how to fix it

true hollow
#

@unkempt rock line 1 for you is line 0 internally

#

to access line 113 use 112 or whatever

feral cedar
#

i'm guessing the context is file like things? in that case seek works with bytes, not line numbers

#

file.seek(5) will go the the 6th byte in the file

raven ridge
# wide shuttle > But the interpreter calls those automatically, that's my point. I understand ...

That's not the pattern at all. The pattern is that there's a dunder for every method that's part of the data model, and the data model is defined by the things the interpreter needs to call. Not all built-in functions correspond to a dunder method, but every method the virtual machine ever decides to call automatically does. And the only non-dunder method I'm aware of that the interpreter ever calls on its own is keys.

#

That's why we don't have __all__, __any__, __ascii__, __hex__, __oct__, ...

wide shuttle
#

I don't agree with that statement, I think

#

I think there is a pattern that for objects implementing a certain interface, you have a universal way of accessing them

#

len(sized_object) fits that

#

abs(object) --> __abs__

#

ascii is a special function that adds some functionality on top of repr (which calls __repr__)

#

any does not need such an implementation because it relies on another interface an object could have (the object is iterable)

#

So does all

#

The reason I've always read is the uniform access principle to give you a single way of doing something that you can implement support for by using a special method: len originally worked with just a few builtin types, but __len__ was introduced to give you the tools to write Pythonic types of your own that could emulate that builtin behavior that worked for builtin types. From that perspective, it has less to do with a pattern of "what the interpreter needs to call" and more with the design question of how to allow the user to define types that behave similarly to built-in objects with a similar, uniform interface.

raven ridge
#

I think your definition is the special case. Clearly __init__ doesn't exist to fulfill some uniform interface, and clearly it does exist because it's part of the data model, because type calls it automatically.

#

(or maybe it's object that does, ๐Ÿคท)

#

len and abs are the weird ones here. Most dunders are for things the interpreter VM needs to call.

wide shuttle
#

I think it's not necessarily a special case

#

Dunder methods allow you to hook into the data model and basic operations within Python to write objects that act just like builtin objects do

#

Allowing you to support len(object) with a dunder method still falls within that categorization

raven ridge
#

__abs__, __dir__, __divmod__, __format__, __len__ - I'm able to find some dunders that exist only to support a built-in function rather than to support the interpreter. But there aren't many of them.

#

The grand majority of dunders are called by the interpreter itself

wide shuttle
#

They allow you to hook into the data model and the workings of Python to influence how your objects behave

raven ridge
#

Yes, we agree about that. We're disagreeing about what "the data model" is.

wide shuttle
#

Whether it's with builtins or with protocols like the descriptor protocol, it allows you to write objects that behave just like builtin objects would

#

I'm not sure about that; I wouldn't necessarily call all of that the "data model"

#

Not all dunder methods are described in the data model chapter of the documentation

spark magnet
#

I might have missed it in the long discussion here about len and special methods, but one factor in the design is that strings weren't objects in Python 1, so they couldn't have a .len() method. len(xyz) allowed lists and strings to be treated more uniformly.

undone hare
#

That's really interesting

grave jolt
#

seems like bool being a subclass of int

#

all those little things keep backward compatibility, but IMO build up into a little bit of a mess

#

like you can do type(...) and .__class__

#

and all the aforementioned dunders

pliant tusk
#

is there anywhere to read about how strings in python 1 worked?

spark magnet
peak heron
#

this just came up on my company's slack, can someone explain what the hell is going on here?

>>> None < None
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '<' not supported between instances of 'NoneType' and 'NoneType'
>>> (None,) < (None,)
False
>>> (None,) < (True,)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '<' not supported between instances of 'NoneType' and 'bool'
>>> [None] < [None]
False
>>> [None,0] < [None,1]
True

Why does it seem like None is only an unorderable type in certain cases?

astral gazelle
#

Isnt the length of the tuple checked before the elements are compared?

#

Huh i guess not

peak spoke
#

It'll only start comparing after skipping equal items

spark magnet
#

It might be using "is" first

fringe badge
#

hi

peak spoke
peak heron
#

hm, if so that would certainly explain it

peak heron
spark magnet
#

i was just getting that URL!

peak heron
#

hahah

grave jolt
spark magnet
#

from tiny acorns, mighty oaks grow

grave jolt
#

I actually like LaTeX more than reStructuredText

#

but maybe it has issues with global references and such

#

also can't easily be rendered to HTML

unkempt rock
#

whare a f-strings much faster then % and .format?

spark magnet
unkempt rock
peak spoke
#

I really doubt string formatting speed is going to be the limiting factor

astral gazelle
#

Sounds like a simple benchmark you can write yourself tbh

spark magnet
unkempt rock
spark magnet
#

it won't work ๐Ÿ™‚

grave jolt
#

f-strings are faster, but in a trivial way, i.e. it probably doesn't have a large impact on your performance

unkempt rock
#

but why?

#

i know it doesn't affect much, but why?

spark magnet
unkempt rock
flat gazelle
#

if you find yourself in a situation where string formatting is a real issue, it may be time to use ropes instead

grave jolt
# unkempt rock ok thanks for the info

.format() also needs to parse the string that you pass in (find all the {} and inspect the contents inside of them), whereas f-strings are parsed at compile time.

#

I guess this is the biggest hit

unkempt rock
#

okie

oblique crystal
# spark magnet For Python's 30th birthday last week, Skip Montanaro revived Python 0.9.1: http... 
WARNING: some scripts are executable and have a first line saying

    #! /ufs/guido/bin/sgi/python

This is unlikely to give good results anywhere else except in my
office.  Edit the first line before installing such scripts; to try
them out, you can just say "python file.py".  (The .py suffix is not
necessary in this case, but makes it possible to debug the modules
interactively by importing them.)```
haha this is funny
undone hare
#

haha

#

smh Guido

normal monolith
#

I need help with some simple code

spark magnet
normal monolith
#

Hi there

#

Itโ€™s quite simple really I just need to

#

Do this I donโ€™t know what Iโ€™m doing wrong Iโ€™ll show you

astral gazelle
normal monolith
#

Oh

pseudo cradle
#

How do we feel about functions that return a variable amount of values?

#

Good practice/not good practice? Classes or dicta better?

flat gazelle
#

Python doesn't have a good way to handle that result, but if it is the best way, sure

spark magnet
#

@pseudo cradle definitely don't return differently shaped tuples at different times

sacred yew
#

@pseudo cradle whats the usecase?

pseudo cradle
#

@spark magnet Iโ€™ve seen a few functions that do so based on certain kwargs. You donโ€™t think itโ€™s a good idea?

spark magnet
#

no, it sounds bad. can you show an example?

pseudo cradle
#

Sure one sec

#

I canโ€™t find it immediately on hand, but thereโ€™s a scientific function that generates a list of values and they have a kwarg โ€œreturn_timesโ€ that additionally returns a list of the times those values were recorded at

#

So the return is either list or list, list

spark magnet
#

i would rather always return the same kind of result.

#

if it's expensive to keep the times, then let a kwarg mean "real tiimes or zeros"

peak spoke
#

Or just replace it with None or something like that, much more discoverable when it's called improperly

pseudo cradle
#

Yeah thereโ€™s a financial and computational cost to performing the full function

#

Or Iโ€™d just always calculate it

pseudo cradle
wide shuttle
#

Does your function handle multiple responsibilities for it to return different kinds of values at different times?

sacred tinsel
#

the train_test_split utility from sklearn takes *arrays and returns a list of length 2 * len(arrays) which is one of the rare cases where it make sense, in my opinion:

>>> from sklearn.model_selection import train_test_split as tts
>>> 
>>> data = [1, 2, 3]
>>> tts(data)
[[1, 3], [2]]
>>> 
>>> data2 = [4, 5, 6]
>>> tts(data, data2)
[[1, 2], [3], [4, 5], [6]]
>>> 
>>> data3 = [7, 8, 9]
>>> tts(data, data2, data3)
[[3, 1], [2], [6, 4], [5], [9, 7], [8]]

splitting variable amounts of vectors here cannot be done with separate function calls because they need to be split the same way (the vectors must be same length, and the function decides the indices that go into the lhs and rhs vector and splits all arrays the same way)

#

although that's actually not a good example at all since it always returns a single list lol, just the length changes

limpid forum
#

But it's still always returning a list of something. It's not like once it will return a value, then the list, and some other time it will be a tuple - the return type stays the same and is clearly defined

sacred tinsel
#

yes I realise now that it's not particularly relevant sorry

flat gazelle
#

I would say it is fine in this case, since that kwarg will generally be static

#

for example, df.boxplot has a return_type kwarg, which makes it return either pyplot axes, a dict, or both.

limpid forum
#

For the topic... I've once had to modify my complex code to move some more info outside of some functions (client wanted the info I dropped early to be present in notification later). So I changed it so that it that I would return two values (aka a tuple, if return wouldn't be unpacked). But it was only sometimes that the other value would be present. I added None to the other return case in the function so that I'll always have this pair tuple (and I unpacked it instantly), instead of having to check for the return type etc.

pseudo cradle
#

So generally I work with fairly complex computationally expensive algorithms. Due to the cost, I only want to compute things that are absolutely required. So 9 times out of 10, the function only needs to do X, but 1 time out of 10, it might need to do X + Y. One problem is that the function may be called in a lot of different contexts, maybe in a data research pipeline, maybe in an app, etc.

#

But dealing with how the returns are handled has been a design problem I've been struggling with the last few days

#

One suggested by my boss has been to simply have a dict, which I think is fine for internal workings within an application, but kind of annoying to deal with as a module somebody is using (which is another usecase)

flat gazelle
#

why not make 2 functions, one which computes both and one which computes just one thing

pseudo cradle
#

I could, but I'm oversimplifying, it could be X + y, or X + Z or X + Y + Z....

#

It see the logic, but it might get out of hand pretty fast

flat gazelle
#

I do think various alternative functions are a solid option, but you are right it could get complex fast

#

if you think that passing the function flags as the what to include and exclude would be nicer, well, do that

pseudo cradle
#

Yeah, it would be nicer in this context. The only problem is how to handle the returns. My initial attempt (foolishly) just returned tuples of varying sizes, but that is error prone as one might imagine

flat gazelle
#

is it? You will get an error if you unpack into a wrongly sized tuple

pseudo cradle
#

Well, I got yelled at for doing it by people who probably know better on this server, haha

flat gazelle
#

well, you definitely shouldn't return them based on varying input values, but I don't think there is much trouble when you do that based on static flags.

pseudo cradle
#

Yeah, I agree with not returning on varying inputs, but hmm. Maybe I won't discoutn it based off of static flags yet.

#

It's a tricky problem, there's so many things to consider.

#

At least, that's my perpsective when discussing it with other people

visual shadow
#

no idea about the context, but would classes make sense?

#

if you want to calculate certain pieces of the equation/calculations and make them available on an as-needed basis, you could always do that. heck, could even design a "lazy computation" for the different pieces as properties on your class for example. just thinking out loud

pseudo cradle
#

I think it would be fantastic. My only concern/caveat would be if it was possible to make an instance of this class json serializable

#

Also classes are scary ๐Ÿ˜›

#

jk, I need to use them more

limpid forum
pseudo cradle
#

I've used namedtuples before and I like them. Dataclasses, too.

limpid forum
#

I've only used named tuples for the cases like i described above - when I need to return several things but started to get confused about the order. XD

#

I wish i had discovered them earlier, tbh

#

Because then my code would be clearer

pseudo cradle
#

Have you messed around with dataclasses? A colleague made a strong sell to me about dataclasses over namedtuples and I was convinced

#

They're similar enough

#

But dataclasses aren't iterable

limpid forum
#

I'm not doing a big enough stuff to really get into it. Mainly integration scripts between systems. Or generating reports from one system. Mostly standalone stuff, basically

pseudo cradle
#

It's always so interesting to hear about what people do. The programming field is more diverse than I gave it credit for

limpid forum
#

I could live (and I did) without the named tuples. I mainly deal with stuff with list/dict comprehensions etc

raven ridge
pseudo cradle
#

Yeah, that's the approach my boss originally suggested

raven ridge
#

Nicer than a variable length tuple because callers don't need to know what order the fields will be returned in.

pseudo cradle
#

nod

#

I just haven't really seen any public open source modules use that approach, so was wondering if there was some inherent bad reason to use it

raven ridge
#

I think classes are generally the better option because of the ability to lazily compute things, but it's a bit annoying to make arbitrary classes json serializable, so that might be enough to sway me to the dict approach

pseudo cradle
#

Yeah, that's an annoying concern I have. Most of my returns will also have to be json serializable in one way

#

Do jsons just use the str dunder?

#

does json serialization, rather?

raven ridge
#

Nope.

#

By default, it will just give an error if you try to serialize a user defined class. dump and dumps take a default= kwarg that you can set to a function that takes an object and turns it into something JSON serializable.

#

object_pairs_hook can reverse that transformation on the decoding side, though that's annoying as well

pseudo cradle
#

oh ugh

#

Gross

sand goblet
#

!e ```py
import sys

s = ""

for _ in range(5):
print(sys.getsizeof(s))
s += "ร "```

fallen slateBOT
#

You are not allowed to use that command here. Please use the #bot-commands channel instead.

sand goblet
#

Does anyone here know why this makes the memory instantly go up by 24 bytes once the character is added?

#

Iโ€™m wondering what itโ€™s doing which makes it need to do that

#

It prints: 001 | 49 002 | 74 003 | 75 004 | 76 005 | 77

#

And it only does it when a non-ascii character is added

#

Otherwise, if youโ€™re just adding โ€œaโ€ for example, it only goes up by 1 each time, so 49, 50, 51, 52, 53

pliant tusk
#

its due to how non-ascii characters are handled internally

sand goblet
#

Whatโ€™s it using the extra memory for?

#

Yeah

#

Itโ€™s confusing because itโ€™s a one character string

raven ridge
#

The extra space is to store the UTF-8 representation of the string, mostly. If a string contains only ASCII characters, the string as an array of 8 bit Unicode code point numbers and the string as an array of UTF-8 bytes are the same, so it can use one array for both purposes

#

If a string contains non-ASCII characters that optimization no longer applies

sand goblet
#

So it needs another array for a 2nd representation?

raven ridge
#

Yep.

#

With ASCII, the two different representations are byte for byte identical, so they can be shared.

#

Outside of ASCII that's not true and they must be separate.

pseudo cradle
#

What gg said

#

You can store any ascii character in a single byte

#

In fact, in C it was often a handy way of defining a single byte integer of value 0-255, lol

#

That's one of the things I had to unlearn when going to Python

sand goblet
#

So both arrays are using the same space in memory?

raven ridge
#

For ASCII, yep.

sand goblet
#

Because i see that it only ever increases by one byte per character

#

For that example

raven ridge
#

!e One chunk of memory can be interpreted as either an array of 8 bit integers representing Unicode codepoint numbers, or as an array of 8 bit integers representing UTF-8 strings, because for values from 0 through 127, it holds that:

for x in range(128):
    assert chr(x) == bytearray([x]).decode("utf-8")
fallen slateBOT
#

@raven ridge :warning: Your eval job has completed with return code 0.

[No output]
raven ridge
#

!e whereas for 128:

x = 128
assert chr(x) == bytearray([x]).decode("utf-8")
fallen slateBOT
#

@raven ridge :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 2, in <module>
003 | UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
raven ridge
#

The only integers with this property are the numbers from 0 through 127, inclusive - which is the ASCII compatible subset of UTF-8

#

That is, for a Unicode codepoint below 128, and only for those 128 codepoints, the UTF-8 representation of that codepoint is a single byte containing the codepoint number.

spark magnet
#

@raven ridge this doesn't explain why after the string has jumped in size (supposedly to hold both a utf8 and a fixed-width form of the string), adding one character only adds one byte to the size.

raven ridge
#

The link I pasted at first does. The extra 24 bytes store a pointer to the UTF-8 representation, the size of the UTF-8 representation, and the number of codepoints in the string

#

Each of those 3 things is 8 bytes.

#

After that original jump, adding one character just adds a single integer to an array of 8 bit integers. Because the character is in the Latin 1 subset of UTF-8.

#

If you add a character with a codepoint of 256 or higher, you'd see another jump.

#

At that point it needs to switch from an array of 8 bit integers to an array of 16 bit integers, and adding ร  after that will add 2 bytes each time instead of 1.

#

It does look like sys.getsizeof() doesn't include the size of the cached UTF-8 representation, though...

sand goblet
#

If you add ๐Ÿคฏ then it increases by 3 extra

# 
import sys

s = ""
print(sys.getsizeof(s)) # 49
s += "ร "
print(sys.getsizeof(s)) # 74
s += "๐Ÿคฏ"
print(sys.getsizeof(s)) # 84
s += "a"
print(sys.getsizeof(s)) # 88```
#

Even though it seems like it would be 81

raven ridge
#

!e That's because:

print(ord("๐Ÿคฏ"))
print(2**16)
fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 | 129327
002 | 65536
raven ridge
#

The codepoint for ๐Ÿคฏ is higher than the biggest unsigned 16 bit integer, so Python needs to switch to using 4 bytes per character instead of 1 or 2 when you add it.

#

When you add ร  it jumps from 1 byte per codepoint to 2, and when you add ๐Ÿคฏ it jumps from 2 bytes per codepoint to 4.

sand goblet
#

But what are the 3 extra bytes

#

Since it goes from 74 to 84 even though 74 - 1 + 4 + 4 = 81

raven ridge
#

Ah. Because "๐Ÿคฏ" is stored not as a single codepoint but as a surrogate pair, I think, making it 72 - 2 + 4 + 4 + 4

#

though it's weird for it to be using both surrogate pairs and 4-byte integers for the codepoints, so I could be wrong about that. I'd expect it to use one or the other, not both...

spark magnet
raven ridge
#

yes - like I said, I'm pretty sure the size of the UTF-8 representation isn't being counted at all, just the size of the pointer to it

#

codepoints in 128-255 take 2 bytes to represent in UTF-8, but can be represented as a 1 byte uint8_t holding the codepoint number, instead.

sand goblet
#

So both pointers do use two different spaces in memory, itโ€™s just not paying attention to the space for the other array?

#

Actually can you have two arrays which are using the same memory space in c?

#

Or is that not possible?

spark magnet
#

@sand goblet for the utf8 string, it's not the same memory

raven ridge
raven ridge
sand goblet
#

Yeah I guess doing that wouldnโ€™t make sense

#

Since you donโ€™t need the array tell the program how to interpret itโ€™s data

#

It just needs to know the size of each piece of data

raven ridge
#

in the case where CPython is doing this optimization - for all-ASCII strings - it works because "an array of characters as a UTF-8 encoded string of ASCII compatible characters" and "an array of single-byte integers representing the Unicode codepoints of a string of ASCII compatible characters" are represented exactly the same in memory, because a character is a one-byte integer type in C.

#

!e ```py
abc = "abc"
array_of_1byte_ints = b''.join(ord(c).to_bytes(1, "little") for c in abc)
array_of_bytes_in_utf8 = abc.encode("utf-8")
print(array_of_1byte_ints == array_of_bytes_in_utf8)

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

True
raven ridge
#

the representation of "abc" as 3 single-byte integers back to back in memory is b"\x61\x62\x63". The representation of "abc" in UTF-8 is b"\x61\x62\x63". These are the same, so CPython can cheat and use one array, knowing that if it made two arrays they'd just hold exactly the same data.

#

they're two conceptually different things, but for strings that consist of only codepoints between 0 and 127, they will always be equal.

sand goblet
#

Why canโ€™t it use the same 1 byte array for ร  if its ord value is under 2^8?

spark magnet
#

because utf8 for codepoints 128-255 take two bytes each

raven ridge
#

!e ```py
print(len("ร ".encode("utf-8")))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

2
sand goblet
#

Why canโ€™t there be more than 128 8-bit characters?

#

Why is it 128 instead of 256

spark magnet
#

that's how utf8 works. If it allowed 256 8-bit chars, there'd be no bytes available for encoding codepoints beyond 256

raven ridge
#

the most significant bit of a byte is set if the byte is a part of a multi-byte representation of a codepoint, and unset if it is a single byte codepoint. That leaves 256 / 2 == 128 single-byte codepoints (the ASCII codepoints), and all others must be multibyte.

sand goblet
#

So the last bit tells it if it needs to keep reading?

raven ridge
#

first bit, but yes. Not just if it needs to keep reading, but how far to read.

sand goblet
#

It tells it either if itโ€™s 8 bits or 16 bits

spark magnet
#

@sand goblet some codepoints need up to 4 bytes.

#

(the utf8 encoding scheme can go up to 6 bytes, but Unicode only needs 4)

raven ridge
#

the first byte tells you whether the codepoint is represented by a single byte (if it starts with "0" as the most significant bit), or two bytes (if it starts with "110") or three (if it starts with "1110") or four (starts with "11110")

sand goblet
#

Oh

raven ridge
#

and all of the bits marked with "x" in the table I linked are the actual bits that make up the codepoint number.

sand goblet
#

So thatโ€™s how it can use 7 bits, because if itโ€™s a character under 127 then it only needs to read the first bit to determine that?

raven ridge
#

right. and that leaves 7 bits to store a value, and 2**7 is 128, so it can store values between 0 and 127.

#

and those 128 characters happen to be exactly the ASCII characters.

#

not by coincidence; UTF-8 was designed to be backwards compatible with ASCII.

sand goblet
#

But I forget why if you have the string "ร ", sys.getsizeof says its size only goes up by 1 per character added, instead of two

#

Itโ€™s because the array itโ€™s measuring just uses 8 bits?

raven ridge
#

I think sys.getsizeof() is ignoring the (lazily created) UTF-8 representation, and is only counting the array-of-codepoints that the string owns. And that array is an array of 1-byte, or 2-byte, or 4-byte unsigned integers.

sand goblet
#

and doesnโ€™t do the leading bit thing?

raven ridge
#

so because all of the codepoints in "ร ร ร " fit inside a 1-byte unsigned integer, it can use just 3 bytes to represent that, and just stores b"\xe0\xe0\xe0"

#

each time you append another "ร " to the string, it adds an extra \xE0 byte to its array of single-byte integers storing codepoint values

sand goblet
#

And then if the ord value for a character is past 2**8, it moves everything in a 2 byte array

raven ridge
#

yep.

sand goblet
#

And then into a 4 byte array if itโ€™s over 2**16

raven ridge
#

!e And that \xE0 I mentioned is because:

print(ord("ร ").to_bytes(1, "little"))
fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

b'\xe0'
sand goblet
#

Alright, I think all of it makes sense now. Thanks for explaining it, guys

marsh sorrel
#

Hi I would like to know how to detect hand motion, hand movement gesture and convert it into some commands, I found leapmotion but need to buy the hardware, can Opencv detect hand motion? Like swipe up to scroll up the pages of my monitor. Anywhere I can get a mentor?

deep bramble
#

This discussion is about python implementation itself, this would be probably more fitting for a help channel

sacred yew
hallow tulip
#

GodlyGeek, are you the same godlygeek with the github vim/nvim repos?

sand goblet
raven ridge
raven ridge
#

And for the ASCII case it can just use one length field for both things because it knows there cannot be any surrogate pairs

sand goblet
#

Alright, that makes sense. Thanks

sacred yew
hasty portal
wheat beacon
#

can someone help me with OCR?

#

I have a sample but I cant get it to get the right characters even though the text is literally a screenshot of a text file

unkempt rock
#

But it seems like someone should look into this because it is causing headaches for anyone that needs to run custom web servers against proxy clients

#

and if anyone needs more information then has been provided in the original issue, I am always here

deep crow
#

can someone help with pyaudio?

spark magnet
#

@unkempt rock by "separate transmission" you mean in two separate send() calls?

unkempt rock
#

yea

#

they sometimes

#

and only sometimes cause the connect to be split into two network frames

#

meaning you have to read twice on the other side to get the full message

#

normal web servers already take this into account because they also serve normal traffic that needs it

#

but for CONNECT it's not really needed and you can just do one write

spark magnet
#

i didn't think any correctly written network service would care how many frames were used?

#

but i'm no expert

unkempt rock
#

no, normally one doesn't care

#

but when building custom application that do really high throughput

#

it matters a lot

#

and I found this particular code to cause A LOT of performance downgrading because of this multiple write implementation

#

if you change it so that the request is formed and then sent as a whole, you could save a lot of implementation complexity on the other side

#

and the annoying part about this is that it only starts to break once you reach high network traffic

#

so it was super hard to replicate

#

and find for that matter

spark magnet
#

sounds like you should open a bug at bugs.python.org, or make a pull request.

unkempt rock
#

that's the thing, I don't use python :S

#

I just joined in here because this issue was dead for almost a year now on that other repo

#

and I figured I should maybe try and come into this discord and poke some python pros

spark magnet
#

what involved you in this at all if you don't use python?

unkempt rock
#

I was building a global proxy network with a ton of users and once our traffic spiked this started breaking it

#

so here I am

#

:S

#

could you prehapse assist me if I run into trouble while making a ticket ? because I have 0 knowledge of python or how things work around these parts :S

spark magnet
#

i'm not sure what help you;'ll need, it's a bug tracker ๐Ÿ™‚

#

though tbh, if the bug is, "things go slower", they might not consider that a bug.

unkempt rock
#

well, an enhancement then ?

#

i'm not really concerned with what it's called as much as how it'll improve things (hopefully)

unkempt rock
#

is this alright ?

spark magnet
#

I'd change the title to mention the module involved, not the line of code.

unkempt rock
#

better ?

spark magnet
#

great

unkempt rock
#

okey, i'll leave it as is .. thank you for the help โค๏ธ

swift imp
# 
reduce(lambda v,f: f(v), list_of_callables, init_val)

Is one of the best ways of applying a list of callables iteratively.

Change my mind.

spark magnet
#

i would put that into a function if i were going to use it, and then once it's in a function, i'd write it as a for-loop.

grave jolt
#

It's just a very cryptic of writing function composition, right?

# 
compose(foo, bar, baz)(fizz_buzz)
swift imp
#

its equivalent to this

for idx, fnc in enumerate(list_of_callables):
  if idx == 0:
    intermediate = fnc(init_val)
  else:
    intermediate = fnc(intermediate)
grave jolt
#

right, so function composition

spark magnet
#

it's equivalent to:

def apply_all(fns, val):
    for fn in fns:
        val = fn(val)
    return val
#

(not sure how enumerate got mixed into it)

swift imp
#

bc im a dumbo sometimes

wheat beacon
#

can someone help me with OCR

ionic crag
grave jolt
#

It's alive

unknown.png
spark magnet
swift imp
#

God i want it

spark magnet
#

get it

severe lichen
#

PATTERN MATCHING COMING BABY

grave jolt
grave jolt
spark magnet
#

i hope it's no big deal

severe lichen
#

But I swear, it would be a mess to learn.

DOG = 'dog'
CAT = 'cat'
def speak(animal):
    match animal:
        case DOG:
            print("woof")
        case CAT:
            print("meow")
speak("cat")
print(DOG)

I wonder what happens.

grave jolt
#

yup, I'm not very fond of all that business...

peak spoke
#

Well, no going back so we'll see how it looks in practice

grave jolt
#

Haskell/Purescript get away from that because all variables start with a lowercase letter, and all type constructors with an uppercase one

severe lichen
#

Aggggh, I hate python capitalisation

#

dict, yet Counter

spark magnet
#

would case +DOG: work?

severe lichen
#

probably
import myscript
myscript.dog

grave jolt
#

sees unary +
javascript flashbacks

severe lichen
#

maaaybe .DOG

spark magnet
#

@severe lichen i'm kind of glad that the capitalization is ad-hoc: it lets us get away from knowing what type things are.

#

class? function? why does it matter?

severe lichen
#

It makes for very annoying name clashes
like lists = [[], []]
for list in lists:
print("Now your list constructor is reassigned")

peak spoke
#

I'm fairly sure backwards comp plays a role in builtins being single word all lower but only constants and exception classes using cap words helps distinguish a bit

spark magnet
#

though, "list" is a bad variable name anyway

grave jolt
severe lichen
#

Also, module name clashes.
like json = response.json()
json = {"reassigned_module": "Yes"}

spark magnet
#

json is also a bad variable name ๐Ÿ™‚

#

but it;s true: python is inconsistent

severe lichen
#

Well, it's either four-letter word for a local variable or local_variable_used_for_very_specific_purpose_by_LordDeathXXX_check_my_patreon

peak spoke
grave jolt
#

also strftime

#

and friends

#

that use the global-vowel-shortage-case

severe lichen
#

Pretty sure straight from C. Just like returning -1 when substring is not found in string

grave jolt
#

yeah

#

that's horrible

silver canyon
#

does anyone know how to fix a discord bot?

#

thats not complete

grave jolt
peak spoke
#

Overall naming could've used a much bigger overhaul in the 2->3 transition along with the other big breaking changes, don't think we'll get anything anytime soon because of that

silver canyon
#

there are so many channels

grave jolt
#

well, life is complex

#

moderators see even more channels

severe lichen
#

You can introduce 3 new keywords in Py4 and backward compatibility be damned. What are they? Also, you get to make a new operator.

spark magnet
#

what are you talking about?

grave jolt
#

the devil is making a deal with the PSF through Oouja

severe lichen
#

I'm talking about what would you do if you had creative freedom and no concern for legacy.

grave jolt
#

well, if you were allowed to introduce new semantics...

severe lichen
#

No braces, though. It's a personal request from Guido.

grave jolt
#

I see, you both have a deal going already...

severe lichen
#

Yeah, that was for not making Perl the main language for ML

grave jolt
#

Well, I would like to have a 'canonical' type checker, sort of like TypeScript. With no weird bugs everywhere, no weirdness (like how mypy treats callable attributes' annotations), and support for slightly more advanced things like generic (polymorphic) values (not just functions), good type narrowing, maybe manipulation of records.

#

Another issue (which is also the case with TypeScript, I suppose, but it's not a frequent use-case) is that some typing constructs like dataclasses or ORM models aren't expressable in the type system itself -- you just hope that the typechecker has a plugin system, and that someone has implemented a plugin for your favourite metaclass-infested library

severe lichen
#

Btw, about type narrowing. I remember C# pattern matching having cool stuff like

void speak(Animal animal) {
    match (animal):
        animal as Cat:
            animal.meow()
        animal as Dog:
            animal.woof()
}

I don't remember the exact syntax, but you get the idea - typecasting and matching in one line

#

Okay, looked up, I forgot everything from C# pattern matching. -_-
Back to Python, then!

#

Also, with typing.NamedTuple (what a weird namespace to place it), i feel record manipulation is in pretty good place.

#

The only gripe I have that that NamedTuple and json dicts don't have a common interface. book["author"] would fail for a NamedTuple.

charred wagon
#

It's not that weird - the original is collections.namedtuple

grave jolt
#

then why not collections.NamedTuple?

peak spoke
#

It serves to add typehints to a namedtuple, makes sense to place it in typing. Same thing for TypedDict

severe lichen
#

or collections.typedtuple

grave jolt
#

and NamedTuple doesn't just add types, it also allows adding methods

#

without an extra inheritance step

charred wagon
#

It wasn't like that originally.

#

3.6 added more features to it, but moving the namespace at that point would be a breaking change.

#

It's certainly an outlier, especially without knowledge of historical context. Even so, it still isn't that weird.

#

With the types originally being just typed versions of collections, it made sense to put them in a separate namespace. Otherwise it would cause confusion. Maybe it could have been collections.typed.NamedTuple but I'm not going to argue that.

knotty plank
#

i collect all important data from my high school days
now its time to arrange them
i hv some yt videos which is no longer available on youtube I need a program which will take the title from my local file and put it on yt and verify if its deleted or still available

i hv some audiobook which is now not available on audible (all arranged)
i hv some ebook
i hv 4TB+ Local Documentary Collection
i hv enormous amount of data
but manually checking all 700000+ Files is too much

what i want is
a program that will take names from my local files and put it into google and check what file it is
is it a series or Documentary
then move that files in specific location as well as custom location
like
government & military (local folder)
i want any documentary which is about politics or military to be put inside same folder

plz help me
otherwise manually it will take 6+ month
i hv already wasted 7 month
in return i will give access to all the files i hv
its 17TB + in total
one 6TB hdd hv 3.7TB documentary
Directory:-
https://www.mediafire.com/file/xxxldw461bt6r0j/6TB_HDD_Video.html/file

strange heath
#

@knotty plank wrong channnel

knotty plank
#

@strange heath where to ask?

#

sry

strange heath
#

@knotty plank in a help channel

feral cedar
wheat beacon
#

can someone help me with pytesseract

tiny whale
#

hey all, let me know if I should ask this somewhere else. I'm trying to do this:

# 
from typing import TypedDict, Final

account_schema: Final = {"email": str}
AccountBase = TypedDict("AccountBase", account_schema)

class Account(AccountBase):
    account_id: str

AccountPatch = TypedDict("AccountPatch", account_schema, total=False)
#

Idea is that I have two types used in communication in my database. Account has all account data fields + ID itself present (returned by my SELECT *s). AccountPatch has all account data fields optional, with no account_id present (because ID can't be updated)

#

since account_schema is Final, why can't it be treated as a Literal dict? I get this error when running my type checker (mypy):

# 
account.py:4: error: TypedDict() expects a dictionary literal as the second argument
account.py:9: error: TypedDict() expects a dictionary literal as the second argument
#

actually Account and account_id are irrelevant to the example, should have focused on why AccountBase and AccountPatch specifically don't accept Final dicts

limpid marten
#

A literal and the specifier final are different.

tiny whale
#

The Final qualifier serves as a shorthand for declaring that a variable is effectively Literal.

#

is there something here that makes that not apply?

limpid marten
#

Ah, that is neat.

#

Could be an issue with mypy.

tiny whale
#

do you think that the dict being mutable could have anything to do with it? cause theoretically I could do account_spec["b"] = "c" and Final doesn't stop type checker from allowing that

#

whereas the pep really only gave an example for int, which is immutable

limpid marten
#

Hm, that could be a good intuition, yeah that'd make sense.

#

Final just means you can't reassign it, doesn't mean you can't change it's members.

tiny whale
#

Yea I'll just create a mypy issue for it, may be bug but might also be intended

ornate vine
#

Error coming can anyone help?

limpid forum
#

What error?

#

Ah, also, this isn't the channel for that. This is a channel for discussing python syntax and other stuff in Python itself

wispy glen
#

any admin/mod here

round fox
#

Ping them? There seem to be a few on

astral gazelle
#

Dont ping mods unless you need something moderated

swift imp
spark magnet
raven ridge
#

Nice! If you use pair = (0, 12) does it correctly flag line 8 as missing?

spark magnet
#

8 and 9, yes

#

the branch coverage will require some work

raven ridge
#

That's pretty good for out of the box tracing support

raven ridge
swift imp
#

Yes but I'm not sure how its dispatching based on the type in the method argument annotation

raven ridge
#

well, it's not - it's relying on singledispatch to do that

#

it's just making it so that the singledispatch.register call is being performed on a bound method, rather than on a function, if I'm understanding correctly

swift imp
#

I think its that closure in __get__ where it happens

#

I'm trying to make my own version of multiple dispatch for work.

raven ridge
#

right, the closure in __get__ is looking at the first of the *args that were passed to a bound method, which already excludes self, if I understand correctly.

swift imp
#

yeah, i just debugged it locally. I just want to fully understand this. I wish I could get pattern matching at work bc it would literally solve what I want to do. I'm trying to dispatch on literal value of key and the type of the value for that key. So the args to my methods are the key values and their type annotation is the type of the value.

deft pagoda
craggy basin
#

Not sure if this has been posted before, but there are exploits in the C used in cpython, see this:

desert peak
#

"floating points as untrusted input"

#

that sounds exceedingly rare

#

"sprintf is used unsafely" LOL

raven ridge
#

yeah, sounds exceedingly rare. But it has already been patched, too.

hushed ether
#

Is it possible to make anti ddos with Python?

sacred yew
#

this is not a help channel @hushed ether

swift imp
#

Does this seem like a bad idea?

class IndexableMeta(type):
  def __getitem__(cls, key):
    return cls.registry[key]
peak spoke
#

Depends on what the registry does I guess. You could also use __class_getitem__

swift imp
#

Doesn't the data model say not to do that with __class_getitem__?

peak spoke
#

It's discouraged, but I can't really find a reason for that and using it instead of specifying a metaclass looks much clearer to me

swift imp
#

Yeah, I mean I guess the metaclass is a bit much. At the end of the day I am doing a subclass registry and I just think being able to index on the parent class (i.e. the class holding the registry) is much clearer. However I can see how others dont think that.

spark magnet
#

@swift imp it's a global variable, so it's bad

undone hare
#

What is a global variable?

swift imp
grave jolt
undone hare
#

.>

spark magnet
#

in this case, cls.registry is global: there's only one of them for your process.

undone hare
#

I mean, what variable in this snippet is global?

flat gazelle
#

there is one cls.registry per class that uses that metaclass

swift imp
#

Which is the point

undone hare
#

Right

flat gazelle
#

I would just use self for that argument, rather than cls

spark magnet
#

depending on how it's used, you have all the downsides of a global variable

grave jolt
undone hare
#

Right, I see

grave jolt
#

I guess you could make some sort of 'cascading constants', like CSS variables. But I don't know anywhere it's useful besides styling stuff

flat gazelle
#

if the registry is immutable, its fine

#

or at least statically known

swift imp
#

yeah so I was going to do __setitem__ too

grave jolt
#

yeah, then it's just a global constant @flat gazelle

flat gazelle
#

what is the actual goal here?

grave jolt
swift imp
# flat gazelle what is the actual goal here?

I have a json schema set up with the jsonschema module and I want to define one registry class per top level object in the json, whose subclasses provide processor method that will be dispatched to depending on the attributes of the json object.

#

Basically my the registry keys are the subschemas, stringified, and the values are the subclasses providing the appropriate processor.

flat gazelle
#

I would just add a classmethod on the parent class

swift imp
#

classmethods instead of subclassing?

spark magnet
#

@swift imp if these are truly constants, then it could be ok. consider if you want to test with different sets of values in that registry.

flat gazelle
#

overall, this isn't that bad of a solution though

swift imp
#

Thanks everyone, this is why I love this channel because I'm considered the expert at work and I definitely not lolol

grave jolt
#

I should probably ask more often here about the stuff I make in my personal projects

#

because sometimes it comes out as total garbage that doesn't make sense

spark magnet
#

are you and i working on the same projects!? ๐Ÿ™‚

grave jolt
#

Wormhole bug in git?

#

Type hint question: if I want to annotate attributes of a class, is it better to do that in __init__ (partially by using type inference), or to do that explicitly in the class declaration?

#

Or maybe I should specify the public attributes in the class declarator, and private in __init__ if needed?..

swift imp
#

are they class or instance attributes? If they're defined in __init__ they're instance and you can annotate them in that, like the body of __init__.

grave jolt
#

instance attributes

flat gazelle
#

I think the convention is in declarator, but if the linter understands it, I would say its fine either way

swift imp
#

Pretty sure the linter will understand bc pylint yells if you define an instance attribute outside of __init__.

grave jolt
#

typecheckers understand both

class Point:
    x: int
    y: int

    def __init__(self, x: int, y: int):
        self.x = x
        self.y = y

and

class Point:
    def __init__(self, x: int, y: int):
        self.x = x
        self.y = y

but sometimes I need to explicitly annotate some attribute anyway:

class Point:
    def __init__(self, x: int, y: int):
        self.x = x
        self.y = y
        self.quack: Dict[str, int] = {}
swift imp
#

that third example should be fine...

grave jolt
#

the issue with the first one is that I have to write out everything twice, which makes the annotations even more bloated

swift imp
#

I wouldn't write the first one. Declaring it as a class attribute seems pointless no?

#

It actually lives on the instance.

grave jolt
#

It's not declaring it as a class attribute -- it's still an instance attribute annotation.
To annotate a class attribute, you'd use ClassVar

class Point:
    quack: ClassVar[Dict[str, int]] = {}
swift imp
#

Oh see, I thought that was only for dataclass. That's nice to know.

#

So I guess you having to re-annotate in the __init__ signature is necessary then because they're required arguments.

grave jolt
#

yep

#

because type inference isn't that good yet

#

especially in mypy

swift imp
#

Yeah its rough, seems awesome but then vscode is colored in red from pyright and its just a nausiance.

supple geyser
#

is logical expression < faster than == ? In my head == should be faster?

swift imp
#

I think that's a loaded question and depends on the type and what dunders the type has implemented and how they're implemented.

grave jolt
grave jolt
#

And why do you care whether < is faster than ==? The difference should be insignificant anyway

#

besides, they're not equivalent, so replacing one with another can introduce a bug

supple geyser
#

if I do sort left (lesser) first, then I wonder if == to equal, or sort right (greater) is faster, since the third is just the else case

#

It's a theoretical argumentation more than practical

#

sorting a list of 10 digits, even the algorithm is insignificant.

silver burrow
#

Guys is there anyone who is familiar with linear algebra

supple geyser
#

to some extent

#

@silver burrow

silver burrow
#

basically there is a simple question that I wanted to ask

#

it's simple as I guess but I don't get how to solve it

#

@supple geyser

supple geyser
#

dm

grave jolt
fallen slateBOT
#

:incoming_envelope: :ok_hand: applied warning to @digital venture.

raven ridge
#

I wonder if py def foo(a, b=1, c): ... should be allowed, and interpreted the same as ```py
def foo(a, b=1, *, c):
...

#

currently you get "SyntaxError: non-default argument follows default argument" - but it's been that way since before we had keyword-only arguments. Now that we do, it seems like it might be reasonable to implicitly treat the first non-default argument after the first default argument as the start of keyword-only arguments.

#

Though, honestly, I'm not sure if that would be more or less surprising for people.

flat gazelle
#

I feel like an error is fine here.

cold forum
#

Is it a bad pattern to have an ABC class with concrete subclasses that only have classmethods (i.e. no __init__ and no instances)?
Say I have this base class PackageBase, with abstract class methods like setup() check_updates() get_version(), etc.. Its subclasses implement those class methods for specific packages (fetch stuff, unzip, w/e). This way they basically exist as a singletons, because it doesn't make sense (for my project) have multiple instances of those packages.

flat gazelle
#

well, there cases where singletons makes sense, but why not have duck typed modules instead? Or even decorated functions.

raven ridge
#

I'd use modules for that as well.

swift imp
cold forum
#

@flat gazelle @raven ridge but with classes I get mixins, methods inheritance (even though they're just class methods), and stuff like that. Like I can have PackageBase <- VersionedPackage <- SomeActualPackage.

swift imp
#

!run

from inspect import signature
def foo(a, b=1, *, c):
    ...
for sig in signature(foo).parameters.values(): print(sig.kind)
unkempt rock
#

Does anyone know how to make an account checker for like combo lists on python?

worldly hedge
raven ridge
worldly hedge
#

Its just an example of how it works because I havent found anything like it on the internet, I was looking for examples on how to implement intercommunication between microservices and I thought screw it imma do it myself

flat gazelle
#

hmm, classes do seem better suited for that

#

though I feel like packages could/should be instances

unkempt rock
#

does anyone know how to make an account checker in python

flat gazelle
#

but then you get trouble with overriding methods. You can have decorator setters like property, but it isn't all that great

#

but IG you need a decorator on every method anyway since it is a classmethod in your case

cold forum
#

I don't plan on having setters though. Properties would be nice, and of course never instantiating the class I can't use them that way (they're just descriptors in a class ofc).
Basically these concrete classes just act as definitions, with some pieces of code to specifically fetch and install the package, get the remote version etc.

#

(By package I mean generic resources/files, nothing to do with python packages)

#

Say then I have a class like SomeTool. And say that tool depends on the package (resources) being installed. In my mind it makes more sense doing a SomePackage.is_setup(), rather than SomePackage().is_setup()

#

And, I might be wrong, having a duck typed module sound like boilerplate / duplicating code, or explicitly referencing other modules (ie no inheritance, no mro)

strange heath
#

wrong channel i'm sorry

flat gazelle
#

well, my idea was

package = VersionedPackage("mypy", "1.0.0", other args)
@package.install
def install(env):
    ...
``` instead of singletons
cold forum
flat gazelle
#

it is a similar pattern to how flask handles its apps

cold forum
#

Looking at it I like the decorator pattern

#

That's true

#

I like it, I think I might end up doing that. Thank you. py_sun

#

The only doubt I have now is how to reference them as dependencies in the SomeTool class. Should I keep a mapping as a registry of all the known packages, with names as strings?
I kinda liked the idea of just importing the package class(es) and saying: ```python
class SomeTool(ToolBase):
dependencies: Container[PackageBase] = (PackageA, PackageB, ...)

flat gazelle
#

import the package instances

cold forum
grave jolt
#

Maybe the real fix is to not use so many global variables ๐Ÿ™‚

flat gazelle
#

or you could have dependencies be the whole modules and metaprogram the package out of there

#

but I do think just importing the packages is best

cold forum
#

I think I might do something like that.

#

Thank you

#

Time to whip out that typing.Protocol I guess

eternal pike
#

:/

boreal umbra
#

Consider this:

>>> sorted([1, 9, 2, 8, 3, 6], key=lambda x: x % 2 == 0)
[1, 9, 3, 2, 8, 6]

Is it specified behavior that if you sort something according to a key that renders non-equivalent values equivalent in terms of their sort order, their original order is preserved within each "equivalence class"?

spark magnet
#

yes

#

sorting is stable

boreal umbra
#

So that's what is meant by "stable sorting"?

spark magnet
#

yes

boreal umbra
#

I'm learning lemon_hyperpleased

spark magnet
#

it means you can sort by a secondary key, and then a primary key, and it comes out the way you want.

boreal umbra
#

and the way that you do that is having a key function that returns a tuple? or is there a way to specify levels of keys that's more "built in" than that?

#

oh I see what you mean

flat gazelle
#

Tuples are the convention. Proof is that attrgetter and itemgetter take multiple args

boreal umbra
#

orly

#

I'm learning more than I ever thought I could lemon_hyperpleased

spark magnet
#

@boreal umbra if you can do sorting on two values at once with a single key function, great. Sometimes it's not possible.

boreal umbra
spark magnet
#

for example, if you have two string columns, and want to be ascending on one and descending on the other.

#

if they were numbers, the key function could just negate the descending number, but you can't negate a string.

feral cedar
#

for that example specifically, you could create a function that turns strings into a tuple of their ordinal value, but that's a bit :/

spark magnet
#

a bit? ๐Ÿ™‚

boreal umbra
#

a bit <whatever adjective :/ represents>

spark magnet
#

in this case, :/ represents, "a crazy idea"

feral cedar
#

yeah, it would work though

sand python
#

the above question came up in #help-carrot, you don't need to "invert" the string itself, just the comparison, right? I came up with this: ```py
from functools import total_ordering
@total_ordering
class Inverter:
def init(self, obj):
self.obj = obj

def __lt__(self, other):
    return self.obj > other.obj

def __eq__(self, other):
    return self.obj == other.obj

usage example, ascends on i[0], descends on i[1]

items = []
for x in range(3):
for y in range(3):
items.append((str(x), str(y)))
print(sorted(items, key=lambda i: (i[0], Inverter(i[1]))))

raven ridge
#

As nedbat said, there's a much simpler solution

sand python
#

of course you can sort twice

raven ridge
#

!e Yeah.

items = []
for x in range(3):
    for y in range(3):
        items.append((str(x), str(y)))
items.sort(key=lambda x: x[1], reverse=True)
print(sorted(items, key=lambda x: x[0]))
fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

[('0', '2'), ('0', '1'), ('0', '0'), ('1', '2'), ('1', '1'), ('1', '0'), ('2', '2'), ('2', '1'), ('2', '0')]
sand python
#

but they did say sometimes it's not possible, with the ascending/descending strings as an example of that

#

which I didn't think was correct

raven ridge
#

yeah. It's possible - your Inverter does it - it's just much trickier.

#

The fact that sorting is stable means that sorting twice does the trick pretty easily.

#

FWIW, your Inverter class works pretty similarly to how functools.cmp_to_key works.

#

holding onto an object, and using < on that object as part of the key comparison.

boreal umbra
#

Are class decorators exactly the same as function decorators, in the sense that they're just nice syntax for thing = decorator(thing) as a statement right after its definition?

sand python
#

yep

sand python
#

i guess I should mention, the Inverter thing was just an exercise of possibility for me. sorting multiple times should be much faster

spark magnet
#

@sand python thanks, that's a nice solution.

#

and btw, i'm not sure which is faster

grave jolt
#

Can someone explain what this change/optimization means?

unknown.png
#

is it something for subinterpreter stuff?..

flat gazelle
#

is that for 3.10?

grave jolt
#

yes

#

_PyEval_EvalCode takes 16, yes sixteen parameters!
ok, that's pretty worrying

#

so it's a refactoring of some god function?

#

BDFL:
Of course, the thing I'd really want is a way to state that all references to builtins are meant to have the exact semantics of those builtins, so the compiler can translate e.g. len(x) into a new opcode that just calls PyObject_Size(). (I can dream, can't I?)
so it's sort of a move to something akin to my esoteric nail decorator that turns global lookups into constant lookups, but much better?

raven ridge
# grave jolt Can someone explain what this change/optimization means?

Sounds like so far it's just turning two dictionary lookups into one. Instead of looking up __builtins__ in globals() and then looking up print in __builtins__, it's now able to look up __builtins__ as a slot on the function object itself, saving the need for a dictionary lookup against globals()

#

Basically, caching a reference to the __builtins__ dict so it doesn't need to be looked up in as dynamic a fashion

grave jolt
#

ah

#

got it, thanks

golden fractal
#

ah

unreal hollow
#

What do you guys think is the best language for code design or code philosophy? I've been using OOP forever now but I might try out FP

limpid marten
#

I'd say it depends on the language specifically. Doing FP in Python for example is miserable, and in most languages where it isn't optimized say like Haskell is for it, it's going to be quite slow.

#

Use what makes sense for the problem you're trying to solve.

barren moth
#

Hello, I have some hard time understanding aync/await (asyncio). If I await for some wss message that comes every 1 second and function proccess of that message last 2 seconds, does that mean that I will get every second mesaage?
If yes, how should I fix that if message flow wouldn't be constant (not every second), but I would just like to prevent my code to miss messages? Using threads and FIFO? Or is there a cleaner option?

I also code in nodejs but async/await should work the same way as asyncio...

charred wagon
#

No, you won't miss messages. It'll process the next message as soon as the event loop isn't blocked. When the event loop is blocked, it doesn't discard tasks. It keeps waiting instead.

sacred tinsel
#

trying out a functional language is always a good idea in my opinion

barren moth
charred wagon
#

Can't say without seeing the functions code. I can't help with this anymore tonight. You should move this up #async-and-concurrency

barren moth
glass pelican
#

hello :)

civic flare
#

how do you get an MIT license?

#

xd

feral cedar
#

do you mean use one?

civic flare
#

yeah

#

sorry

feral cedar
#

you just use it

civic flare
#

I don't have to do anything?

feral cedar
#

github? gitlab?

civic flare
#

oh

#

I thought you had to literally go to MIT, graduate, and then get it

#

lol

feral cedar
#

a bit more simple than that lol

civic flare
#

oh lol

#

anyway thanks a lot!

feral cedar
#

you just have to have a document in your repo with the license in it

civic flare
#

oh, cool

feral cedar
#

github has a nice thing explaining it, let me find it

civic flare
#

okay, thanks

feral cedar
civic flare
#

alright, thanks!

raven ridge
#

It's called "the MIT license" because it's the license under which MIT released it's open source code back in the day.

#

Anyone else could choose to release their own code under the same license.

tame laurel
acoustic crater
#

anyone else excited for match case being added in 3.10?

#

I'd been following it but hadn't seen it in the "what's new" doc until today

#

it's nice that it accepts "dotted" constants as literals, I think the capturing behavior will confuse a lot of people who think it's switch

wide shuttle
#

It was only added now to the changelog because it was accepted a few weeks ago and was first included in the alpha that was released less than 24 hours ago

#

I'm excited. I'm preparing a talk about it for work.

acoustic crater
#

I've got a project I wrote a few months ago that I wanted to refactor to refine my OOP skills and it'll be perfect for that (it's a blackjack engine)

#

right now the logic I wrote to test various possible rulesets against various possible hands is messy and could be cleaned up a lot with pattern matching I think

clever hill
#

Hi Everyone, I made this menu system for my application and it has a lot repetitive code in it. I feel like I could structure this better. Does anyone have time to take a look at it and give me some feedback?

topaz charm
#

hey guyhs

#

what does init do

#

because ive seen a lot of videos

#

which put it in def function classes

#

but i just dont know what it means

#

stuff like

#

stuff____stuff

#

stuff

#

stuff

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied mute to @topaz charm until 2021-03-02 19:22 (9 minutes and 59 seconds) (reason: duplicates rule: sent 4 duplicated messages in 10s).

feral cedar
#

um

#

well the init function is used to assign instance variables when you initialize a class

white nexus
white nexus
fallen slateBOT
#

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

oblique crystal
sacred tinsel
#

I just think it's great to get a taste of something different, it can grow your perspective and allow you to come up with better solutions even in languages that aren't purely functional

#

or just reading about how a certain problem would be solved in something like haskell

#

in terms of what we can gain out of it, I think all of the philosophical pillars - less (or no) state, pure functions, immutability - lead to code that is both easier to reason about and easier to test, even in an OOP-first language like Python

limpid forum
#

I also think that learning at least a bit of purely/mainly functional language is a good idea. It makes you... switch your thinking a bit. When you learn it, you have to use functional stuff, you have to understand it. There are no loops, you have to make use of that other stuff.
And later you can apply it to other languages with functional elements

#

Like... you know how many stuff you can do with zip? I learned it by working with a person who loves functional programming. We coded in Scala (or rather: he coded, I joined because he didn't have time to do it all by himself and mainly worked with writing similarly to what was already written), big data stuff - then it makes a difference if you have a million loops or chain generators and such.

raven ridge
#

It also teaches lessons that apply to other domains. Immutable state plays a crucial part in distributed systems design, for instance.

gleaming rover
#

recursion with a trampoline though...

#

๐Ÿฅด

silk pawn
# 
        # WOOO PATTERN MATCHING
        if compat.PY310 and type(obj) is dict:
            match obj:
                case {tags.TUPLE: _}:
                    restore = self._restore_tuple
                case {tags.SET: _}:
                    restore = self._restore_set
                case {tags.OBJECT: _}:
                    restore = self._restore_object
                case {tags.FUNCTION: _}:
                    restore = self._restore_function
                case _:
                    def restore(x):
                        return x

ok so this works in 3.10.0a6, butmatch throws a syntax error in 3.8.6 (and probably 3.9.2 also but i haven't tested that), are there plans for a syntax backport in future releases of 3.6+ so that libraries can start integrating this matching code without breaking anyone not at 3.10

#

or does it go in a different file actually

#

i doubt putting it in a different file will help actually

peak spoke
#

No, backporting things like that is not really a thing afaik. Not sure if the old parser can even make sense of it

silk pawn
#

huh ok

#

so people needing to maintain libraries that don't require cutting edge python versions will have to wait a few years to include this?

#

or is there a workaround for older versions

grave jolt
#

as with any feature

peak spoke
#

If they want to use new features they have to use new python, the versioning wouldn't make much sense if every feature like that got backported officially

silk pawn
#

oh hm, so this is the error in 3.8.6:

  File "test.py", line 2, in <module>
    import mylibrary
  File "/root/.pyenv/versions/3.8.6/lib/python3.8/site-packages/mylibrary/__init__.py", line 60, in <module>
    from .unpickler import decode
  File "/root/.pyenv/versions/3.8.6/lib/python3.8/site-packages/mylibrary/unpickler.py", line 211
    match obj:
          ^
SyntaxError: invalid syntax

could we put from .unpickler import decode in a try/except syntax error block and import a different version of the file if the user isn't using 3.10

#

lemme test that ig

peak spoke
#

You can check the version before importing

silk pawn
#

yup yup

peak spoke
#

But if you're maintaining one path then that sounds easier than pushing for pattern matching and having to maintain both

silk pawn
#

then that sounds easier
which approach are you referring to by "that", try/except syntax error or checking the version first

grave jolt
#

@silk pawn Why even make the pattern-matching implementation when you have a non-patma one?

silk pawn
#

depending on how you use it of course, but still

peak spoke
#

Just leaving it as is, because adding the conditional imports becaue of pattern matching means you now have 2 different code paths to maintain. Either continuing to not use pattern matching until the majority of your user base adopts it (or versions below 3.10 reach EOL) or just forcing everyone to adopt the new python version sounds much easier

#

I didn't look at it that much, but I don't think you'll get speed increases that significant

grave jolt
#

Just a thought: maybe people would adopt new Python versions faster if libraries dropped support for eariler Python versions faster?

pine apex
#

That's a good thought

peak spoke
#

I am always a bit dumbfounded when maintainers insist on supporting EOL versions like 3.5 and cutting features because of it

grave jolt
#

I guess there's the downside that people will just sit on their old version of Python and not receive important security updates...

silk pawn
silk pawn
grave jolt
unkempt rock
grave jolt
#

and your own database adapter

unkempt rock
#

nice one

#

i rely on libraries a lot

grave jolt
#

but people have deadlines and budgets

unkempt rock
#

yea

peak spoke
#

You still have python libs that may not be available in earlier version, but any app with an appreciable size will use 3rd party packages as that's one of the points of using python in the first place

silk pawn
#

it seems like a contradictory statement to make though because the older versions of python don't even have the peg parser iirc, so how would they use it to be backwards-comptaible?

grave jolt
silk pawn
#

ohhhhhhh

grave jolt
#

so if you have a variable named match, it will still work

#

because it's never ambiguous whether it's a match statement or a variable name anyway

silk pawn
#

got it, i thought backwards-compatible meant that if you introduce a feature, it won't cause errors in versions before the feature was introduced

grave jolt
#

That's not backwards-compatibility. If the language ignores something it doesn't understand, that's just horrible language design ๐Ÿ™‚

#

nobody likes debugging HTML and CSS

silk pawn
#

fair point

raven ridge
#

That's forwards compatibility.

#

Like, adding a new optional argument to an existing function, for instance

grave jolt
#

any old program will still work on the new version of the library

#

an example of forwards compatibility would be ignoring all extra arguments.

raven ridge
#

er, you're right - that was a bad example

raven ridge
boreal umbra
#

I love keyword only arguments for that reason lemon_hyperpleased

grave jolt
#

although

#

maybe it's not that bad

boreal umbra
#

Doesn't blindly passing **kwargs to a function that doesn't take **kwargs just drop any that it doesn't take?

#

Or does it cause an error?

raven ridge
raven ridge
boreal umbra
#

Boo

raven ridge
#

unless the **kwargs dict is empty, anyway.

boreal umbra
#

But you can swallow extra kwargs by having star star kwargs as a parameter and ignoring it, yes? (On mobile)

raven ridge
#

yep.

boreal umbra
#

I'm gonna do that to all my functions

boreal umbra
grave jolt
raven ridge
#

classes designed to be dropped into an inheritance hierarchy to do something extra - usually one specific feature.

boreal umbra
#

I like those

#

Not that they're necessarily good

#

I just like that they exist if you want them.

acoustic crater
boreal umbra