#internals-and-peps

1 messages ยท Page 139 of 1

flat gazelle
#

after the tailcall, where would to code under except etc go? You no longer know that you were originally in that function

verbal escarp
#

yeah, it's tricky

lusty scroll
#

gone :0

verbal escarp
#

i wonder if you could throw away the stack and just return an "empty" exception at this point

lusty scroll
#

with/finally also gets in the way of that i guess

flat gazelle
#

and honestly, I will take consistency over some minor performance improvements any day

pliant tusk
#

my tco cant optimize py try: return tail_optimised_call() except: ... because python inserts a POP_BLOCK op before the RETURN_VALUE op

verbal escarp
#

lovely

lusty scroll
#

can the block pop be "anticipated"?

verbal escarp
#

i was thinking of implicit function arguments, but that would be even worse :p

pliant tusk
lusty scroll
#

oh

verbal escarp
#

all very tricky ๐Ÿ˜

lusty scroll
#

There's no more low hanging fruit left? like removing metaclaesses? ๐Ÿ˜›

verbal escarp
#

low hanging fruit.. pff :p all eaten

lusty scroll
#

too bad .๐Ÿคง

#

add a JIT, it worked for java right.. ๐Ÿ˜‰

verbal escarp
#

simplifying stuff, removing unnecessary optimizations for the greater good might be "low hanging"

#

in combination with a JIT, exactly

lusty scroll
#

like... adaptive interpreter

verbal escarp
#

yeah

#

thinking of numba

#

i was really baffled to see how a simple for-loop optimized by numba was faster than the same thing in numpy

lusty scroll
#

we just need numba on the entire stdlib

flat gazelle
lusty scroll
#

numpy uses flat C arrays right?

flat gazelle
#

afaik yes

lusty scroll
#

what about the stdlib array module.. replace some of the most highly accessed lists with static arrays

verbal escarp
#

oh god

#

array.. that's just crap if you ask me

#

not even sure why it's in the stdlib

flat gazelle
#

for C interop

peak spoke
#

it has some uses, but replacing lists is not one of them

lusty scroll
#

then the implementation just needs to be replaced maybe, it should be faster?

peak spoke
#

it'll be slower than lists for most operations you'd use them for as you'll be converting back and forth from native types to objects

lusty scroll
#

or is the problem still that everything's allocated in the heap

flat gazelle
#

you can't use stack allocated things in python since they would go out of scope when you give python access to them

#

maybe you could make it work for internal functions, but I would assume that is already done where possible just by implementing the function in C

lusty scroll
#

but still a big part of why it's slow? or not?

#

the dynamic allocations

flat gazelle
#

not really, it's afaik mostly indirect calls and pointer chasing

lusty scroll
#

ah

flat gazelle
#

dynamic allocations are quite well optimised, as you will quickly find out when you compile python with malloc instead of the custom allocator

lusty scroll
#

why default to it then, pymalloc is default right?

surreal sun
#

What is malloc? All Iโ€™ve heard of is tracemalloc which is for the GC

flat gazelle
#

pymalloc is the default, and it is the well optimised one

#

I do wonder how python would do with jermalloc

lusty scroll
#

what's the custom one

flat gazelle
#

pymalloc is the custom one

#

my point was that compared to conventional malloc, the python allocator is faster for python

lusty scroll
#

oh, i just thought you were saying the opposite. makes sense

surreal sun
#

Ohh is pymalloc the allocator that puts the memory into the pools, arenas, etc.?

flat gazelle
#

afaik yes

lusty scroll
#

glibc malloc does use arenas too.. dunno about pools

surreal sun
lusty scroll
#

jemalloc is supposed to also be well optimized, iirc

#

i suppose one might be able to LD_PRELOAD it to test? if it's not too deeply linkrd in

spare furnace
#

What do you guys think is going to be next language people shift to after python? Julia, Rust, Python4?

verbal escarp
#

that question doesn't make much sense

flat gazelle
#

I don't think there is going to be an after python

#

Julia will probably end up quite major in data science

verbal escarp
#

and Python4 isn't a thing

flat gazelle
#

but a major part of data science is to this day in scala

#

or well, apache spark

verbal escarp
#

Rust is playing in a totally different niche

spare furnace
#

There isn't going to be a python 4?

verbal escarp
#

no plans whatsoever

lusty scroll
#

they're very close

100 loops, best of 5: 2.88 msec per loop

python3.11 -s -S -m timeit -c  "list(x for x in range(0, 5000))"
100 loops, best of 5: 2.86 msec per loop
spare furnace
#

Oh yeah I haven't heard of any plans but why wouldn't they do it eventually?

lusty scroll
#

probably a bad benchmark, but oh well :p

verbal escarp
#

"eventually" can be in 50 years

spare furnace
#

50 years is far too long in this field lol. I think in 10 years you will need to consider it.

lusty scroll
spare furnace
#

We also have new types of computers like quantum

verbal escarp
#

they all work nicely with python 3

flat gazelle
#

if quantum computing ends up useful enough to actually replace current computers, then there will probably be entirely new languages

spare furnace
#

I'm not sure if you will replace computers but it's like GPU right

verbal escarp
elder blade
flat gazelle
#

array is for some byte twiddling and C interop

spare furnace
#

You might want some ability to tap into quantum but maybe that's better off as a cloud API

elder blade
verbal escarp
elder blade
#

"Optimizing Python " doesn't mean that everything magically runs faster

spare furnace
lusty scroll
verbal escarp
#

we don't need a new language for that

lusty scroll
#

i was comparing them in the least scientific, easiest way possible

spare furnace
flat gazelle
#

I would argue that quantum computing is too different idiomatically from python to be considered the same language, even if the syntax looks would look kinda similar

lusty scroll
elder blade
flat gazelle
elder blade
verbal escarp
flat gazelle
#

I guess a quantum programming DSL would technically qualify as python, but it is still very different code

verbal escarp
#

but apparently (according to the scientist who wrote it) it works quite well

lusty scroll
verbal escarp
#

python as a glue-language does a good job at wrapping at a high level

flat gazelle
#

I would also be surprised if that actually worked

#

we may get some pytorch tensor-like higher level API as python, that is true

verbal escarp
#

yeah, that's what i'm expecting for the future

#

i'm thinking quantum computing could make a difference in db lookups like JOINS

spare furnace
#

So python might be interface but a lot of code might be written in other languages?

verbal escarp
#

which could be an optional optimization like with GPUs in tensorflow

verbal escarp
spare furnace
#

I am asking in terms of this distribution like if 90% of your written code is in python (not all the codebase). When we will shit to 50%+ of that time in other languages.

verbal escarp
#

depends on the level of abstraction you're working at

spare furnace
#

I mean just in general lol.

#

Like the average current python programmer.

verbal escarp
#

there's no such thing as "the average current python programmer" - python devs are across many different fields

flat gazelle
#

I do expect python to be used less these days as it is now competing with pretty cool languages, whereas in the past, you had... ecmascript 5, java and were trying to phase out perl

spare furnace
verbal escarp
#

just look at all the topical channels

spare furnace
#

I know this lol

flat gazelle
#

so now that you have languages like go, julia, rust, typescript, python is a lot less attractive

lusty scroll
lusty scroll
elder blade
#

It's very fast with no pain or afterthought

verbal escarp
flat gazelle
#

yeah, modern demands on performance are quite a shift

near coral
lusty scroll
#

it can do stuff js can't though, seems like

verbal escarp
near coral
#

but I'll take average ๐Ÿ‘

flat gazelle
#

but well, there are efforts to fix that, and there are still a ton of places which are IO bound enough that python is more than fast enough

near coral
#

I mean, is python really in some race to be The Programming Language

verbal escarp
near coral
#

if people want to use other languages, that is fine. All languages should have their pros and cons

spare furnace
#

Do you guys think GIL will be an increasing problem

elder blade
lusty scroll
#

and it doesn't require compiling some monstrocity like V8, LLVM, or chromium to get a working interpreter

elder blade
#

The GIL is a solution, not a problem

near coral
#

speed just isn't python's strong suit at the moment, but it is definitely quick when it comes to scripting up something quickly

flat gazelle
#

the GIL only matters for CPU bound workloads, which is things python will never be good at

#

at least not without removing a lot of runtime introspection

near coral
#

I wouldn't want to build an executable in Windows for it though for someone who doesn't use Python

peak spoke
#

it can be considered a problem in certain applications, just not as big as most people make it out to be as you can work around it if you want

verbal escarp
#

CPU bound matters don't matter nowadays really anyway

lusty scroll
#

how's that gilectomy coming along... ๐Ÿ˜…

verbal escarp
#

it's either GPU-bound or bus-bound

elder blade
peak spoke
#

I quite like it with Qt

elder blade
flat gazelle
#

and well, the GIL is great from a usability standpoint. No weird crashes from using thread unsafe data structures like in C++ or java, while keeping performance in single threaded workloads. Java tried having the approach where each data structure had a lock on it, and it was just too slow to be useful.

lusty scroll
#

he was gonna do lock-per-object iirc

flat gazelle
#

that's what the old Vector class does

lusty scroll
#

With his complicated buffered-reference-count approach he was able to get his โ€œgilectomizedโ€ interpreter to reach performance parity with CPythonโ€”except that his interpreter was running on around seven cores to keep up with CPython on one.

flat gazelle
#

I don't even have 7 cores.

lusty scroll
#

lol ne neither

verbal escarp
#

heh

elder blade
# lusty scroll he was gonna do lock-per-object iirc

Yeah if I've seen the 2 or 3 talks on it. The plan became like lock-per-object and maybe somehow statically figure out if an object is only ever used inside one thread which means that you don't need a lock on it

lusty scroll
#

slower python !

flat gazelle
#

and well, threads do introduce a whole host of issues in that you now have to think about context switches.

lusty scroll
#

like full-blown escape analysis

flat gazelle
#

you could just do a simple if refcount == 1: don't lock I think to start with

elder blade
flat gazelle
#

oh wait, weakrefs

lusty scroll
#

oh i didn't know about it

#

and now he's planning a tracing GC

elder blade
lusty scroll
#

talk about scope creep

elder blade
lusty scroll
#

don't try to remove python's GIL because it might be the last thing you do in your career

spare furnace
#

Sounds like removing GIL would require a pseudo new language/framework.

verbal escarp
#

@lusty scroll suppose we can tackle that after we reinvented the import system ๐Ÿ˜‰

elder blade
verbal escarp
elder blade
#

This is what I was thinking about I guess?

unknown.png
#

Aaaahh, read for yourself if you're interested haha I am confused https://speakerdeck.com/pycon2017/larry-hastings-the-gilectomy-hows-it-going?slide=16

Speaker Deck
PyCon 2017
Larry Hastings - The Gilectomy: How's It Going?

One of the most interesting projects in Python today is Larry Hastings' "Gilectomy" project: the removal of Python's Global Interpreter Lock, or "GIL". Come for an up-to-the-minute status report: what's been tried, what has and hasn't worked, and what performance is like now.

https://us.pycon.org/2017/schedule/presentation/118/

verbal escarp
spare furnace
#

Like Cpython interpreter to a Rustpython?

verbal escarp
#

i mean that CPython might not be the future but an interpreter written in a different language like Rust

#

yes

#

PyPy is another good contester

elder blade
#

PyPy is an incredibly good contester but it is very complex compared to CPython

spare furnace
#

Yeah you would need to adapt the python language to take advantage of Rust memory management I would assume though.

verbal escarp
#

moving to another default interpreter would be a major shift, definitely involving lots of PEPs

#

i'm imagining that people would even might ask for a formal specification before that happens

#

which is a big change from the status quo

elder blade
verbal escarp
#

i'm really not sure, i couldn't bet on either side

#

there are many good reasons why it might happen

elder blade
#

What I would like to see though, would be that someone can become a core-developer even though they're not directly a core-developer of CPython

verbal escarp
#

it could happen instead of a push for Python 4

wary harbor
#

Anyone have any good ideas for bash scripting projects?

#

Im not sure how well the general project repos would translate to bash, I have little experience with it

flat gazelle
wary harbor
#

oh gotcha

median palm
unkempt rock
#

lol

raven ridge
lusty scroll
median palm
#

lmao

magic nova
lusty scroll
#

CI: Failing
:(

surreal sun
#

What do you all think on the uh, adaptive interpreters thing

#

!pep 589

fallen slateBOT
#
**PEP 589 - TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys**

Link

Status

Accepted

Python-Version

3.8

Created

20-Mar-2019

Type

Standards Track

surreal sun
#

Wrong one

#

!pep 569

fallen slateBOT
#
**PEP 569 - Python 3.8 Release Schedule**

Link

Status

Active

Python-Version

3.8

Created

27-Jan-2018

Type

Informational

surreal sun
#

Oh my god

#

sorry

lusty scroll
#

not sure what to think tbh

surreal sun
#

I still don't understand it fully lol

lusty scroll
#

i see it's still a draft PEP, so it must predate the PEP i guess

#

my understanding was they replace parts of the bytecode (in existing code object) once some counters tick dowm to some "hotness" threshold

raven ridge
#

it's an Informational PEP - which AFAIK means that it's not a proposal, but a form of documentation. It's something that will be happening, not something that they're soliciting community feedback on.

magic nova
#

(it's already happening as part of 3.11)

lusty scroll
#

are the replacement opcodes type-specific versions of existing ones?

magic nova
#

they can be, yes

#

for example there's BINARY_ADD_INT

fallen slateBOT
#

Python/ceval.c line 2086

TARGET(BINARY_ADD_INT) {```
lusty scroll
#

it seems like a half-step towards a profiling JIT compiler for bytecode? or perhaps an optimizer

magic nova
#

yeah I'm sure some people would not like the term JIT here because it's not emitting new machine code

#

but it's like a profiling optimizer

lusty scroll
#

just-in-time optimizer :)

charred pilot
#

Extensive experimentation suggests speedups of up to 50%.
๐Ÿ‘€๐Ÿ‘€

lusty scroll
#

that would be huge obviously

static bluff
#

I'm wanting to write a decorator class whose instances perfectly resemble the functioon it wraps, from the outside

# 
    class Decorator(object):

        def __init__(self, function: FunctionType) -> None:

            self.function = function
            self.name     = ''

        def __set_name__(self, owner: Type['Node'], name: str) -> None:
            self.name = name

        def __repr__(self) -> str:
            return str(self.function)

        def __str__(self) -> str:
            return str(self.function)

        @property
        def __class__(self) -> Type:
            return type(self.function)

        @property
        def __signature__(self) -> str:
            return self.function.__signature__

        @property
        def __text_signature__(self) -> str:
            return self.function.__text_signature__

        def __call__(*arguments: Any, **keywords : Any) -> Any:
          return self.function(*arguments, **keywords)
#

Am I missing anything? I know I need to make the decorator's __call__ method's return type the same as the decoratee, but other than that, am I missing anything?

paper echo
#

you can delegate almost everything to __getattr__, no?

#

!e ```python
class A:
def repr(self): return "A()"
def str(self): return "A"
def add(self, other): return 1 + 1
def radd(self, other): return 1 + 1

class B:
def init(self, wrapped):
self.wrapped = wrapped
def getattr(self, attr):
return getattr(self.wrapped, attr)

a = A()
b = B(a)

print(repr(b))
print(str(b))
print(b + b)

fallen slateBOT
#

@paper echo :x: Your eval job has completed with return code 1.

001 | <__main__.B object at 0x7f3a1744bfa0>
002 | <__main__.B object at 0x7f3a1744bfa0>
003 | Traceback (most recent call last):
004 |   File "<string>", line 18, in <module>
005 | TypeError: unsupported operand type(s) for +: 'B' and 'B'
paper echo
#

hooo interesting dunder interactions

#

@static bluff you forgot __annotations__

#

and yes you'll have to override __getattr__ of course even if it doesn't work with the dunders

static bluff
paper echo
#

not sure if you can hook into dir()

static bluff
#

I'd rather do it all explicitly (property by property) at least until I understand it

paper echo
#

oh you need __doc__ of course, and __qualname__ and __name__ and __module__ - not sure if you can override any of those last 3 without breaking something

#

basically go through the python reference

#

i don't even want to know why ๐Ÿ™ˆ

static bluff
#

Emulating dir() might be more than I need โ€” I just don't like how decorating a function returns something that complete disresembles the original

#

Especially the name. You've got a function with a name go in, and you get something like 'wrapped' or 'decorated' come out, its a bit primitive. I'd rather use an object and get at least a little more grace out of it

paper echo
#

!e ```python
from functools import wraps

def nicen(cls):
@wraps(cls)
class Wrapper(cls):
def getattr(self, attr):
print('nice')
return super.getattr(attr)
return Wrapper

@nicen
class Thing:
def one():
return 1

print(Thing.name)

thing = Thing()
print(thing.one())

fallen slateBOT
#

@paper echo :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 12, in <module>
003 |   File "<string>", line 5, in nicen
004 |   File "/usr/local/lib/python3.9/functools.py", line 58, in update_wrapper
005 |     getattr(wrapper, attr).update(getattr(wrapped, attr, {}))
006 | AttributeError: 'mappingproxy' object has no attribute 'update'
paper echo
#

heh

swift imp
static bluff
swift imp
#

You're trying to decorate methods?

# 

class Foo:

   @Decorator
   def bar(self, *args, **kwargs):
       ...
``` ?
#

Honestly man

#

I would customize __init__ and then rework your __call__ to take in the method you're decorating and return self

#

Then implement __get__ to return the decorated method

#

What you have right now is that upon decoration you are initializing the Decorator instance and the method is passed to that

#

If you instead do @Decorator() then the wrapping happens in __call__ and to actually get the method back out you would have to implement __get__

#

Descriptor decorator as it's described in Fluent python

#

Plus in your current state, how are you going to bind the instance

static bluff
#

I'm trying to build an object that decorates a function and resembles that function as closely as possible, particularly its name its type hints

swift imp
#

Like if you do it how you have it now then when __call__ happens you don't have access to the instance of the owner class to pass down

static bluff
# 
Return = TypeVar('Return')

def decorate(function: Callable[..., Return]):
    
    class Decorator:
        
        def __call__(self, *args) -> Return:
            return function(*args)
        
    return Decorator()
#

PyCharm's linter catches the return type with this, but

#

Not the arguments

#

Ellipsis notation can't catch the complexities of the signature. I can have the class return the signature programatically, but I don't think there is anything I can do for it linter wise

#

3.10 has ParamSpec, not 100% sure what that does but I think it might be more or less what I'm after โ€” but I can't use 3.10 sadly, since I'm relying on another library and I'm lucky it even has a 3.9 release

swift imp
#

Are you trying to decorate methods or functions?

static bluff
#

Both, really

swift imp
#

This is important

static bluff
#

Capturing the instance wouldn't be too much trouble, I know how to work that

swift imp
#

Well the thing you got going above, the one I originally responded to, won't work for methods

static bluff
#

I know how to convert a decorating object to to bound method form

swift imp
#

How are you gonna bind the method without __get__

static bluff
#

I've navigated worse. I say this because I don't know exactly how, but, I know the ins and outs of the process, I'm sure I can figure it out

#

The typing is what I'm not sure about

swift imp
#

I don't know much about typing either

static bluff
#

Its an artform unto itself ๐Ÿ˜›

native flame
static bluff
#

And I can't upgrade due to an important dependancy

native flame
#

hm

fiery stirrup
#

Hello guys, how i can include an inequality on my sistem?? ```python
import numpy as np
from scipy.optimize import nnls

A = np.array([
[-190,-30,-30,10,10],
[-27,-197,-27,3,3],
[-36,-26,-196,4,4],
])

b= np.array([-45,0,0])

x= nnls(A, b)[0]

lament sinew
fallen slateBOT
#

Lib/typing.py line 859

class ParamSpec(_Final, _Immutable, _TypeVarLike, _root=True):```
brave badger
#

ParamSpec lives in typing_extensions as a backport

lusty scroll
elder blade
elder blade
raven ridge
#

Granted only CPython core devs vote in the election, but it's not unreasonable to think that many of them would vote for a PyPy core dev if one were nominated

prime estuary
#

Even if it was only core devs, there are core devs brought on who's motivation isn't working on CPython itself.

peak geyser
#

I want to learn OOP in python any Good resources ?

gloomy rain
verbal escarp
median palm
#

oop

verbal escarp
#

that's why i mentioned it ๐Ÿ˜‰

#

but it's no contester to cpython (yet)

median palm
#

yep

verbal escarp
#

it was pretty extensive

unkempt rock
#

How do I become better at reading Python code? any suggestions?

paper echo
#

try to read code that's in big applications/libraries

#

and in general try to read other people's code

swift imp
unkempt rock
verbal escarp
#

i found it

verbal escarp
lusty scroll
static bluff
#

Picture sent so you can see how its all be highlighted

#

It seems that in my sleep deprived state last night, I couldn't really see the forest for the trees

#

PyCharm's linter has no trouble tracing the typing, and warning appropriately. The only issue is that it sees the ParamSpec argument as an error and highlights it in red. This doesn't actually throw an errors, but there is no way to turn it off

#

That's the kind of problem I'd simply solve with a comment

#

I'm dependant on wxPython and cefPython, which I don't think are supported on 3.10 yet, and its 3.10 that this support is native. Hopefully in a year or two they will be updated or maybe I'll just update them myself

#

With the typing issue 'solved' though, I think between this and functools.update_wrapper I can hit all of the important points of introspection. I don't want my decorators to be identical, I just wanted a bit more control than the traditional decorator pattern offered

verbal escarp
static bluff
#

Gonna give that a read when I'm on the bus. Only just waking up now

lusty scroll
lusty scroll
static bluff
lusty scroll
#

all good

worn lantern
#

Is there anything like typeshed's sqlite3/dbapi2.pyi for generic use in libraries? I can go write up a module full of Protocols but I'd rather not if there's a lazy option.

#

Kinda surprised we don't have something in stdlib, actually

hollow estuary
#

why does the help channels keep changing names

worldly relic
#

to keep it interesting, also wrong channel this is for advanced python discussion

unkempt rock
#

i need help
how do you pass an variable from a .py file to a .kv file
please help ๐Ÿ™

static bluff
somber bobcat
# 
TypeError: Argument 'real' has incorrect type (expected numpy.ndarray, got NoneType)
paper echo
sacred yew
somber bobcat
surreal sun
#

I didn't know you could override the main module to be a new module type

#

I just created a constants module that actually enforces constants

# 
from types import ModuleType
from sys import modules


class ConstantsModule(ModuleType):
    constant = property(lambda self: 11)
    

modules[__name__].__class__ = ConstantsModule
#

this is some pretty cool stuff that i have somehow just discovered

verbal escarp
surreal sun
#

oops I read it wrong

#

I wrote it wrong*

verbal escarp
#

we had exactly that question a while ago in here

surreal sun
#

I meant that you can override the module class to act differently, I think godlygeek posted something earlier about it

#

With the __setattr__ thing

verbal escarp
#

the conclusion was that python handles global variables differently than those that require an attribute lookup

#

and since module-level variables in __main__ don't require attribute-lookup, you can't replace it with a property

surreal sun
#

Ah yeah, the global state discussion right/

surreal sun
verbal escarp
#

exactly

surreal sun
#

because it's using attribute access since the module is basically a class

verbal escarp
#

then you need an attribute lookup, which you can manipulate

surreal sun
#

That's really cool tbh

#

I can imagine the gates this opens up to cursed code

verbal escarp
#

well, that's how justuse works ๐Ÿ™‚

surreal sun
verbal escarp
#

currently we add auto-reloading that way but it's also possible to add operators to modules

#

we considered the idea of using the @ operator for decorating stuff

surreal sun
#

reloading is when a file is imported again, or just that the module class in the sys.modules is reloaded

surreal sun
verbal escarp
#

or [..] on the module for filtering stuff

verbal escarp
verbal escarp
surreal sun
#

Ah

verbal escarp
#

you could think of all kinds of manipulations on that level, but it's hard to think of something that makes sense for multiplication or addition ^^

#

numpy + math = ? ๐Ÿ˜„

#

numpy * 3 = ?

surreal sun
#

I would wonder how you would implement module addition and module decorators, I've tried but it fails

verbal escarp
# 
class ProxyModule(ModuleType):
    def __init__(self, mod):
        self.__implementation = mod
        self.__condition = threading.RLock()

    def __getattribute__(self, name):
        if name in (
            "_ProxyModule__implementation",
            "_ProxyModule__condition",
            "",
            "__class__",
            "__metaclass__",
            "__instancecheck__",
        ):
            return object.__getattribute__(self, name)
        with self.__condition:
            return getattr(self.__implementation, name)

    def __setattr__(self, name, value):
        if name in (
            "_ProxyModule__implementation",
            "_ProxyModule__condition",
        ):
            object.__setattr__(self, name, value)
            return
        with self.__condition:
            setattr(self.__implementation, name, value)
#

that's the ProxyModule we wrap all loaded modules with, which allows adding any kind of dunders like __add__ or __mulmat__ (@)

#

to apply decorators on the module, we use the aspectize function i came up with

# 
            for (check, pattern), decorator in aspectize.items():
                _apply_aspect(
                    mod, check, pattern, decorator, aspectize_dunders=aspectize_dunders
                )
#

with

# 
def _apply_aspect(
    thing,
    check,
    pattern,
    decorator: Callable[[Callable[..., Any]], Any],
    aspectize_dunders=False,
) -> Any:
    """Apply the aspect as a side-effect, no copy is created."""
    for name, obj in thing.__dict__.items():
        if not aspectize_dunders and name.startswith("__") and name.endswith("__"):
            continue
        if check(obj) and re.match(pattern, name):
            log.debug(f"Applying aspect to {thing}.{name}")
            thing.__dict__[name] = decorator(obj)
    return thing
#

which lets you apply decorators to anything that matches a regex pattern (against the name) and a check (against inspect.isfunction or any custom check)

#

like mod = use(Path("some_module.py"), aspectize={(use.isfunction, ""): timeit})

#

we considered putting the aspectize argument in matmul, to have it like mod = use(Path("some_module.py")) @ {use.isfunction, ""): timeit}

#

but never got around to do it ^^

#

also i kinda forgot to make it work for recursive objects, but yeah..

#

we're now working on mass-testing, making sure that all conda packages etc. work with justuse so those crazy-ass ideas had to take a little step back ๐Ÿ˜‰

#

@regal tide is nuts enough to join our little team and make the effort ๐Ÿ™‚

dusk quartz
#

You will need to create a game where the user trys to break a code before the time runs out. The user must enter a 4 digit code that is compared to the randomly generated four-digit code (1000 - 9999). Your program should provide output containing the number of correct digits in correct locations, as well as the number of correct digits in incorrect locations. Have an * indicate that the number entered in that position is incorrect. Have $ indicate the number at the position is incorrect but still belongs somewhere in the code. If the number is in the correct position, then just print the number in its position.
Example 1:
Difusal code: 1234
User guess: 1365
Output: 1$**

Example 2:
Difusal code: 1234
User guess: 9867
Output: ****

Can anyone help me how to isolate each digit

sacred yew
verbal escarp
verbal escarp
#

@sly raft nice work!

shrewd onyx
static bluff
#

You know, it would have been lovely if they put somewhere in PyInstaller's documentation that it won't copy a directory over if its empty

#

Just spent three hours figuring that out

grave jolt
static bluff
#

PR?

#

Oh a pull request

#

I've never done that

#

You know guys

#

This graphics engine of mine is really coming along

#

Its taken a lot of work, but I think I can see the light at the end

warm junco
#

And there was an issue on this, but it went stale because no one finds licensing fun to work on

static bluff
#

What sort of licensing issues are there?

#

O.O This sounds like something I should know

warm junco
#

Dependencies + python licensing you have to disclose are annoying

#

Iโ€™ll see if I can find the GitHub issue

static bluff
#

Guh, this looks like an ugly thing to have to worry about it

#

You'll have to, what, track down the licensing of everything you use, recursively, and manually include them?

warm junco
#

Pretty much. Thereโ€™s a tool called piplicenses on PyPI that can help though

#

!pypi pip-licenses

fallen slateBOT
warm junco
#

Like a button or link

static bluff
#

Would a big document containing the licenses, and a function injected into the global scope of all modules license(), do??

#

Also, you seem to have used PyInstaller in the past

#

My project uses cefpython and wxpython. I was testing to make sure both of those would bundle without issue. They did, but the resulting application took about 20s to launch. I know this has to do with pyinstaller rewriting all the scripts

warm junco
static bluff
#

Any thoughts on speeding it up?

small tree
#

Hello, good morning

warm junco
small tree
#

how to use this help exactly? Don't understand how to call this directly

DeprecationWarning: `set_matplotlib_formats` is deprecated since IPython 7.23, directly use `matplotlib_inline.backend_inline.set_matplotlib_formats()`
  set_matplotlib_formats('svg')
static bluff
#

Unbundled cef takes no time to launch

#

Or like, half a second maybe

static bluff
warm junco
#

What do you mean by โ€œrewritingโ€? Pyinstaller doesnโ€™t embed your scripts in the app directlyโ€”it compiles them to .pyc files first

visual shadow
#

Don't distribute as onefile if you want speed perhaps. Just zip the bundle

#

Since onefile mode essentially needs to be unpacked again before running

still wolf
#

hello is there anyone?

#

ping me asap

astral gazelle
verbal escarp
#

i got one for you ๐Ÿ™‚

#

for our AST-jugglers, mostly

#

let's say i have a mod = use(...) call, i only get the module that was returned

#

if i'm only interested in a function or variable that lives in that module, i'd need to write a = use(...).a

#

but that's not very DRY, and it gets worse with multiples:

# 
mod = use(...)
a, b = mod.a, mod.b
#

so, i'm wondering

#

what if it was possible to write a, b = ~use(...) where ~ would mean "take the names of the variables that are to be assigned and turn it into an iterator of .get(name) calls

undone hare
#

with bytecode manipulation it should be possible

#

or at least bytecode observation

verbal escarp
#

i'm not set on the ~, it could be any operation if that'd make things easier

#

just to clarify

undone hare
#

I don't think it is possible to know what you are unpacking into otherwise

#

an __unpack__ dunder could be interesting, hmmm

verbal escarp
#

could you write me a little prototype example from where i could go on? i have no idea how to tackle this otherwise ๐Ÿ™‚

lusty scroll
#

how in the heck is mutilating licenses any concern of pyinstaller's

verbal escarp
#

a simple, contained example, 10 lines of code or somesuch that demonstrates how to go about the "take the names of the to-be-assigned-variables and turn them into a function call on the right-hand-side"

#

or maybe even better <= as "reverse assignment operator"

#

could that work?

#

a, b <= use(..)

#

usually that would raise a NameError, maybe that could be caught and handled somehow?

#

hm.. sad

#

yeah, it's not ideal, i know

warm orchid
#

Anyone help?

#

Okay

#

Message link?

verbal escarp
#

if it wasn't hard-coded, i'd go for "a, b from use(..)"

warm orchid
#

Ok

verbal escarp
#

hehe

#

for a, b in use(...): pass could work :p

#

but it got the same reverse-naming issue as the assignment

elder blade
#

Why not just use next()?

verbal escarp
#

how would that help?

elder blade
# 
a, b = next(use(...))
verbal escarp
#

that's not really the issue, i'm afraid. the problem is that the variable names "a" and "b" are unknown before calling the right-hand side

#

so next(use(..)) is meaningless because the order is arbitrary

elder blade
#

Oh so use() may return something else than an unpack-able iterable?

verbal escarp
#

yes

#

use() usually returns the whole module

#

the "proper" way would be mod = use(..); a, b = mod.a, mod.b

elder blade
#

Oh you want the attributes of that. Like JavaScript?

verbal escarp
#

i'd like to get as close to from mod import a, b as possible

elder blade
#

{a, b} = require('....')?

#

I don't think that's possible in Python though no

verbal escarp
verbal escarp
elder blade
#

The issue becomes that Python has no sense of variable names right?

verbal escarp
#

i was thinking an AST-transform of sorts could solve the conundrum

elder blade
#

You have no idea when being called what the name of the variables are.. even if you inspect the locals of the caller (because they haven't been defined yet ;-;).

#

Right?

native flame
#

yeah i was thinking if it was possible at the AST

verbal escarp
#

you could observe the globals dict

#

i'm wondering at which level that NameError is raised, actually

#

does that happen at the AST level?

verbal escarp
#

{a, b} = use(..) would be perfect

undone hare
visual shadow
elder blade
#

!e ```python

{a, b} = 1, 2

fallen slateBOT
#

@elder blade :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     {a, b} = 1, 2
003 |     ^
004 | SyntaxError: cannot assign to set display
undone hare
#

Don't think it is valid syntax

elder blade
#

Ah cool

undone hare
#

oh interesting

verbal escarp
#
  • yet
visual shadow
#

I don't understand the motivation when the from syntax already exists, could you elaborate?

elder blade
#

!e ```python
{'a': a, 'b': b} = {'a': 1, 'b': 2}

fallen slateBOT
#

@elder blade :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     {'a': a, 'b': b} = {'a': 1, 'b': 2}
003 |     ^
004 | SyntaxError: cannot assign to dict display
verbal escarp
visual shadow
#

That feels patma-ey

elder blade
#

I wonder if that would be more Pythonic. Being able to assign to a dictionary

visual shadow
#

From module import a, b already does it no?

elder blade
#

Then you'd just have to add __getitem__ to make it supported?

verbal escarp
#

use(..) is a complete alternative to import

visual shadow
#

But you said it's a module?

#

Oh. Well in that case pep20 says there should be one and preferably only one way to do it

verbal escarp
visual shadow
#

And in this instance, I strongly agree with the Zen. ๐Ÿ˜…

verbal escarp
#

and that "one way" doesn't do a lot of the things use() can do ๐Ÿ˜‰

visual shadow
#

Use is something you're making?

verbal escarp
#

yup

visual shadow
#

Maybe you could consider some kind of patch on globals, though I'm surprised why you opted to use use instead of patching the import system

verbal escarp
#

because the import system sucks big time

visual shadow
#

So something like expose(use(...), a, b, c) I suppose?

verbal escarp
#

that could work, but it would be confusing to IDEs that look for assignments

visual shadow
#

I mean, use is confusing to python Devs who look for imports

#

Aka every python dev.

elder blade
verbal escarp
undone hare
#

ooh, you made justuse

verbal escarp
#

it' pretty transparent and doesn't interfere with the rest of the import system

verbal escarp
#

why the surprise?

elder blade
undone hare
#

Nothing, just makes sense now

verbal escarp
#

ah ๐Ÿ™‚

undone hare
#

I wish to use justuse in some of my script to auto download libs, but it turns into a chicken egg problem, as you'd still need to manually install justuse

verbal escarp
#

bbl

visual shadow
#

So I guess I can visualise the issue clearer. You essentially want to hijack assignments a,b = blah such that blah actually sees the names

#

This however is fairly different from how python does assignments though , meaning I don't think the outlook is good without some probably python level hacks

#

Since python completely evaluates what's on the RHS first

elder blade
visual shadow
#

Ironically enough I guess your best bet is to use from use(...) import a, b probably

elder blade
#

But that won't unpack by name

verbal escarp
verbal escarp
#

hijacking the import system to override its behaviour when encountering a ProxyModule returned from use(..)

#

i'd still prefer the JS-ique {a, b} = mod as a more general solution

#

but the import .. from syntax might be a viable workaround

native flame
# 
In [79]: f("""
    ...: from types import SimpleNamespace
    ...: a, b = ~SimpleNamespace(a=1, b=2)
    ...: print(a, b)
    ...: """)
1 2

success, i guess

elder blade
#

!e ```python
def func():
return 'collections'

from func() import Counter

fallen slateBOT
#

@elder blade :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 4
002 |     from func() import Counter
003 |              ^
004 | SyntaxError: invalid syntax
unkempt rock
# 
print('hi')```
#

!e

print('hi')```
fallen slateBOT
#

@unkempt rock :white_check_mark: Your eval job has completed with return code 0.

hi
paper echo
#

What is a legitimate use case for __foo method names? I see them in e.g. motor and all they do is make it hard/impossible to extend

spice pecan
#

Preventing name clashes with subclasses

#

I think that's the intended purpose of mangling

zenith flint
verbal escarp
#

i agree that it's annoying as hell though

#

but i don't see any other solution for the name clashing really

verbal escarp
#

does it also work for b, a = ~SimpleNamespace(a=1, b=2); print(a,b) -> 1,2?

visual shadow
# elder blade Is that legal?

Oh, looks like it's not, eh? Hmm, I hadn't considered that. If the grammar itself stops it then we are back to square one, would require hacking through python itself I presume

verbal escarp
undone hare
#

oh, I see

verbal escarp
#

it would've been doable up to the point where it only handled importing single modules

#

but since we got into the whole package installation business, it's far too complex to do it all by ourselves

#

just yesterday i added pydantic as requirement to help deal with pypi json etc.. meh

native flame
native flame
#

don't think its entirely foolproof though

#

it'll fail for a = ~SimpleNamespace(a=1) because it expects the LHS to be a tuple; but that could be special-cased

verbal escarp
#

brilliant

#

how did you manage it?

native flame
#

also probably fails for x = a, b = ~SimpleNamespace(a=1, b=2) though idk what the expected behaviour for that would be in the first place

verbal escarp
native flame
#

it just modifies the ast from a, b = ~x to a, b = [(_ := x), [getattr(x, 'a'), getattr(x, 'b')]][-1]

verbal escarp
#

ah, i see the pastebin now

#

arrow-code ftw ๐Ÿ˜‰

#

wow.. that's alien

verbal escarp
native flame
#

wdym?

verbal escarp
#

if the user had another class that implemented ~ with a different meaning, would that be affected?

native flame
#

should work as expected unless there's a __inv__ implemented for that class

#

oh wait i misread

#

yeah it won't work if ~ is supposed to carry a different meaning

verbal escarp
#

could we do isinstance(a, ProxyModule) and modify the behaviour only in that case?

native flame
#

hmm im not sure
dont think the ast is aware of types

flat gazelle
#

I wonder if you you could use inspect to read the line use() got called in

#

and then parse that

verbal escarp
#

and then replace the code in place by an expanded version, you mean?

#

that's bold

flat gazelle
#

you can just return a tuple from inside use then

#

or well, you would do this for __inv__

verbal escarp
#

pretty crazy, i like it

#

basically pre-parse the code and replace it in place

#

would also avoid the type-checking issue on the AST-route

#

and, depending on how the introspection in the IDE is done, it might "just work"

shy sphinx
verbal escarp
#

i don't think that's legitimate python ๐Ÿ˜‰

flat gazelle
#

IIRC, the non-ascii digits don't work in literals

#

also, seems like you don't always get code_context, so the idea may not work sadly

#

it works when running a file

# 
from types import SimpleNamespace
import inspect
import ast
data = SimpleNamespace(a=4, b=6, e=7)
def use():
    frame = inspect.currentframe()
    finfo = inspect.getframeinfo(frame.f_back, context=1)
    line, = finfo.code_context
    target, = ast.parse(line.strip()).body[0].targets
    if isinstance(target, ast.Tuple):
        return [getattr(data, el.id) for el in target.elts]
    else:
        return data
paper echo
verbal escarp
#

thanks!

lusty scroll
#

like setting __class__ or having the builtin getattr overridden comes to mind

#

but i don't think it'd be worth trading features for performance... if you want that you can just use Lua ๐Ÿ˜…

verbal escarp
undone hare
#

Hmm, that's an interesting question

#

I basically have a utility script inside a Rust project that changes the value of an ELF symbol. Not that relevant, but it requires a library to be installed for that, and it would be nice if could just use that library and have the library installed if that's required.

#

As to whenever a zip file would be a good idea, maybe it is, I am not 100% sure how I want to manage my build system so far

#

I might just ensure we have a venv with the dep included before running the script

verbal escarp
#

let me write up an issue on our tracker for that suggestion, maybe we can come up with a solution after we mass-tested all conda packages ๐Ÿ™‚

undone hare
#

That'd be quite cool

#

By the way, would you perhaps have some good first issues you need contributions to?

verbal escarp
#

heh.. uhm..

#

sure, what's your area of expertise?

astral gazelle
#

Plug project again please? If mods are ok with it

verbal escarp
#

np

thorny haven
#

Does anyone know how to create a code that helps you write backwards?

fallen slateBOT
#

@round hull Per Rule 6, your invite link has been removed. If you believe this was a mistake, please let staff know!

Our server rules can be found here: https://pythondiscord.com/pages/rules

quiet crane
verbal escarp
#

https://discuss.python.org/t/how-do-we-want-to-manage-additions-removals-to-the-stdlib/10681 <- that might also be interesting to us here

Discussions on Python.org
How do we want to manage additions/removals to the stdlib?

What sort of requirements do we want to place on ourselves for adding modules to the stdlib as well as removing them (this is not about what to do with the stdlib in terms of what kinds of modules should be there, etc.)? Clarifying our views on this affects multiple PEPs such as 4, 411, and 594 (and the latter is the reason Iโ€™m asking now so th...

gloomy owl
#

I have some problems with a program thats a number guessing game where u have 3 attempts. If the player guesses the number correctly on the first attempt he wins *10 the money he bet, if he guesses the number on the second attempt he wins *5 the money he bet, on the third attempt he wins *3 the money he bet. After 3 attempts he loses the money he bet. The game should carry on as long as the player wants to play if he has money

Rules:
Number between 1 and 50

The program keeps looping after you get it right. I expect the game to stop btw a tired to use break but didnt work at all

verbal escarp
gloomy owl
verbal escarp
gloomy owl
verbal escarp
#

strange, well..

#

can you post the code you wrote so far? do use `` triple quotes

paper echo
#

huh, i just went through all of them

lusty scroll
#

either that or your break isn't bring hit

#

put a print right before it and make sure you see the print

#

if it's nested loops you have to separately break out of each loop

#

afaik :\

delicate sage
thorny haven
paper echo
flat sinew
elder blade
# 
To show this is not blackmagic, here is an example:

**main.py**

```py
from hata.ext.extension_loader import EXTENSION_LOADER
from datetime import datetime

cake = 'cake'
now = datetime.now()

EXTENSION_LOADER.add_default_variables(cake=cake, now=now)
EXTENSION_LOADER.load_extension('extension')
```

**extension.py**

```py
print(cake, now)
```
# 
After you ran `main.py` you should see the following (the date should be different tho):

```
cake 2020-03-14 09:40:41.587673
```

Because now we have the interpreter, you can change the variables.

```py
>>> EXTENSION_LOADER.add_default_variables(cake='cakes taste good, and now is:')
>>> EXTENSION_LOADER.reload_all()
```

And a different text is printed out:

```
cakes taste good, and now is: 2020-03-14 09:40:41.587673
```
fallen slateBOT
#

hata/ext/extension_loader/extension.py line 342

def _load(self):```
elder blade
#

This is pretty interesting stuff ngl

chrome flax
#

How much math is needed for data science ?

mild flax
chrome flax
#

Thanks

timber dragon
#

How to get the source of str.endswith for example? inspect.getsource(<what>)?

surreal sun
visual shadow
#

some things in python are written in C proper, so you wont be able to inspect them like that

surreal sun
#

You will wanna check out CPython source code if you want to find it

#

Let me try and find where it is

timber dragon
#

Okey, fine

#

How can I find out if method I'm interested in is implemented in bare C?

#

<=> so not possible to get the source in mentioned way?

surreal sun
fallen slateBOT
#

Objects/unicodeobject.c line 13660

PyDoc_STRVAR(endswith__doc__,```
surreal sun
timber dragon
#

In this case I really didn't even know that to type as argument ๐Ÿ˜„

#

Nevertheless - fine, thanks

verbal escarp
surreal sun
visual shadow
#

cpython* doesnt do issues on github iirc.

verbal escarp
#

where did i see that huge list of issues then..

#

i thought it was python

visual shadow
verbal escarp
#

i see typeguard will become unmaintained soon, would you say pytypes is a good replacement?

elder blade
#

Is typeguard a static type checker?

verbal escarp
#

no, runtime

raven ridge
#

this would be a better fit for #type-hinting. It's not on topic for this channel.

verbal escarp
#

there's a hook that decorates all functions and methods in a project to check against the annotation

candid lodge
#

looks like the indexes have not updated since 3.4 and the indexer only run on python 2 because at the time libclang wasn't available for py3

#

I used to monkeypatch inspect with it since it passes through, and you just get to be surprised sometimes when C code comes up from getsource

novel vector
#

Is there any performance plus from compiling your own python over using the precompiled one?

flat gazelle
#

Not a meaningful one probably.

true ridge
raven ridge
#

the best case, if you build yourself with PGO and LTO and target your machine's particular architecture, might be a 20% performance improvement over your distribution's build of the interpreter, if your distribution didn't use PGO and LTO and targeted an older instruction set.

#

which isn't a tiny amount, but in the grand scheme of things, it's not going to make your program run twice as fast, or anything like that.

hearty root
#

I'm new to GitHub and I've started my first real project. If I want only me and my affiliates to be able to use the project commercially, what license might I use? (Open source)

Ping me with your response please, also if this is the wrong place to ask this please direct me if possible

paper echo
#

I legitimately don't know these things, but if I can get 20% speedup by recompiling Python then I would be tempted to do so

paper echo
# hearty root I'm new to GitHub and I've started my first real project. If I want only me and ...

I'm not sure what channel this belongs in, possibly off-topic. But i will say that restricting use means that none of the "free software" licenses will work for you, and you might need to make your own. What you can do instead is use a copyleft license like AGPL, which does not prohibit commercial use, but forces commercial users (including you) to publish the source code and forces derivative works to be published under the same terms, and thereby generally acts as a deterrent towards other companies using the code.

#

Creative Commons has "non-commercial" licenses but CC recommends against using their licenses for source code

undone hare
#

recompiling is not what takes time, it is running the tests

#

haha

elder blade
paper echo
#

Oh that's a good point and very true

tulip jasper
undone hare
paper echo
#

I wonder what makes it take so long

#

I'm sure there's just a huge number of tests

deft pagoda
#

the nogil fork is pretty impressive

undone hare
#

I don't know why it takes so much time

#

I do wonder if it is just frozen

#

I guess I could strace it and see what is causing so much trouble

elder blade
verbal escarp
#

wow!

tulip jasper
elder blade
#

Yeah after hearing about the pains of the GILectomy it really does seem too good to be true

tulip jasper
#

I am guessing the secret sauce to success is using a thread safe memory allocator mimalloc

#

Pythonโ€™s built-in โ€œpymallocโ€ memory allocator is not thread-safe. This project replaces pymalloc with mimalloc [link], a memory allocator developed by Daan Leijen at Microsoft. Mimalloc generally has better performance than pymalloc and is thread-safe. Additionally, mimallocโ€™s layout of objects in the heap allows for finding all GC-tracked objects without maintaining an explicit list, which isnโ€™t possible to do efficiently with pymalloc or other allocators built on top of malloc. Finally, the design of mimalloc enables a thread-safe implementation of dictionaries and other collections that avoids locking during most non-mutating access.

verbal escarp
#

very cool

#

in that regard i wouldn't say it's too good to be true

trail basin
unkempt rock
#

LMAO

#

LMAO

#

!ot

fallen slateBOT
unkempt rock
#

Oh okkk

#

i think the only thing he should do in return is learn python 9ACOSP_pray

#

Yes, i dont even want him to learn python, he want to be a Skidder

verbal escarp
unkempt rock
#

if im having a hard time deciding whether i should learn django or typescript for my backend framework given ik MERN and python
i have a group of friends that just wanna built stuff (hopefully one will be succesful) and idk which framework i should use
every idea is somewhat diff and i will eventually learn both but idk what to start with if that makes sense
what do u think the pros and cons are for each

verbal escarp
#

what's a self-promoting bot?

verbal escarp
rapid rampart
#

do you have any experience with gzip on heroku?

raven ridge
# paper echo 20% is potentially really big though. Where would those gains come from? Startup...

More optimized performance on the eval loop of the virtual machine, mostly. Each of those things individually might buy you ~5% speedup if your distribution doesn't do them. PGO runs a big chunk of the test suite under an instrumented version of the interpreter to figure out which branches are taken most often, and then uses that information to rebuild the interpreter so that the branch predictor prefers the branches that tend to be taken most often. LTO allows functions to be inlined across translation units. And your distribution has some minimum version of the x64 ISA that it will target - it may ship binaries that work with every x64 processor from the AMD Opteron on, for instance. If you compile yourself, you can ask the compiler to produce code that will only work for Core i7 processors and newer, for instance, which means that it can take advantage of newer instructions that weren't available for the earlier chips in places where multiple instructions would have previously needed to be used.

#

of the 3, two of them can be done without costing you anything but build time. If your distro doesn't do PGO and LTO, you should file an issue about it and ask the maintainers to do it. They make the package faster for everyone, at the cost of making it take longer for the maintainer to build it, which is a huge win for distribution packages.

verbal escarp
#

so, we're testing to auto-install all the packages in the conda distribution via justuse (up to 80% passing now!), but we're stumbling over a few roadblocks, namely packages that need extra stuff like dll's that you would have to download manually and on-site compiled packages that might require adding cython (and/or some other build-environment) as an extra dependency to justuse

#

are there any elegant solutions to any of those problems that might come to mind?

#

for the second issue i could think of some trusted third party service that could compile packages to certain configurations, is there anything like that?

paper echo
#

Seems worth it for a production environment

paper echo
raven ridge
#

That's part of what auditwheel does.

verbal escarp
#

interesting

floral glade
#

I saw some answers on qoura that computer science can also be used in healthcare. So do you think we can also use CS to help with mental health? And in what ways can we use CS in healthcare?

gleaming rover
floral glade
raven ridge
#

#career-advice is likely the best fit, since that channel is broadly about Python and jobs

floral glade
azure pine
#

hi

lusty scroll
#

it just bails out and says "Oh, you needed Cython"

raven ridge
fallen slateBOT
#

pyproject.toml line 7

"Cython>=0.29.24,<3.0",  # Note: keep in sync with tools/cythonize.py```
lusty scroll
raven ridge
#

It is

lusty scroll
#

ahh

raven ridge
#

Did you run python setup.py install or something like that?

lusty scroll
#

i did pip install, but I don't know if wheel was installed at the time. It warned that it was "using legacy setup.py"

raven ridge
#

Yeah, sounds like pip was unable to use the modern PEP 517 builds, then, and fell back to directly running setup.py without installing the build dependencies in an isolated build environment.

lusty scroll
#

ok, so if I have an up-to-date pip, and wheel, I suppose it should work?

raven ridge
#

Yep

pallid delta
#

Hello. I start new project and search for guy that can work with me. If anybody interest just send me message. Thank you

lusty scroll
chilly fractal
paper echo
#

https://discuss.python.org/t/announcement-pip-21-3-has-now-been-released/11078

Editable installs for build systems other than setuptools! โœจ
๐Ÿ˜ฎ

Discussions on Python.org
Announcement: pip 21.3 has now been released

Hello Pythonistas! On behalf of the PyPA, I am pleased to announce that we have just released pip 21.3, a new version of pip. You can install it by running python -m pip install --upgrade pip . This is the last scheduled release for 2021, following our regular quarterly release schedule . Highlights This release has a lot of good stuff in it...

void flame
grave jolt
obtuse knot
#

Two questions:

  1. Not sure if I missed it here but is there a planned replacement for XMLRPC for using pip? I know it's been down for some time due to what basically amounts to DDOS attacks but the status page for it doesn't mention of any replacement

  2. I know on installation that Python can be set to have the standard library compiled into .pyc, but are their talks of speeding up the standard library further by compiling them into something like Cython's pyd for Windows and so files for Unix systems? Essentially move as much of the functionality that 99% of projects derive from to give an overall performance increase?
    The more manual method would be to rewrite the standard library in C but that could introduce so many more issues

elder blade
winter dawn
#

anyone who is good at apache nifi do let me know, I need help in intergration of Apache NIFI, will pay you if needed. DM

verbal escarp
#

any reasons why we couldn't have USE flags like gentoo?

#

if there was a unified way to globally manage flags we could do optimized on-site build/compilation

elder blade
#

Of Python?

verbal escarp
#

yeah

#

-> cython for example

#

or consider tensorflow

#

or other packages with specific hardware configurations

elder blade
#

Ooh, I see now yeah

verbal escarp
#

and if pypi was managing things, we could condense all flags into a single number ๐Ÿ˜„

#

like.. numpy-2.21.0-298 would correspond to a bitmask of 100101010

#

you could look up the flags set

#

if flags were never removed, only appended, that system would be backwards compatible forever

raven ridge
#

Previously only pip install . would work.

paper echo
#

Basically what godlygeek said ๐Ÿ˜†

elder blade
#

I thought that is what PEP 517 did

paper echo
#

the PEP 517 build-backend tells pip how to build a wheel, that's it

#

"editable" installs are like pip install -e - they leave the files in place and create a "link" that tells python where to import the modules from. entirely different system, and was a setuptools-specific feature until now

iron summit
#

Hello everyone! How do I display the image from Google Earth by giving the valid co-ordinates in ".png" help will be appreciated!!

verbal escarp
#
  1. that's the wrong channel for this question 2) that question doesn't make much sense in any case
iron summit
#

I mean I have to write all of this in Python code.

warm junco
delicate sage
#

Why is hash same?

>>> a=[1,2,3]
>>> object.__hash__(a)
116525477204
>>> a.append(2)
>>> object.__hash__(a)
116525477204
peak spoke
#

because it's the same object and object's eq goes by identity

delicate sage
#

oh

#

oooh

visual shadow
#

theres a reason dicts dont accept lists as keys. relying on hashes of mutable containers is a recipe for disaster

delicate sage
#

does it work on any object?

flat gazelle
#

object.__hash__ works by id

delicate sage
#

oh

#

does it like hash id?

flat gazelle
#

I think it just divides the id by pointer alignment and uses that directly as the hash

magic nova
#

yup pretty much, though it's not as precise as "pointer alignment" ๐Ÿ˜…

fallen slateBOT
#

Python/pyhash.c lines 137 to 145

Py_hash_t
_Py_HashPointerRaw(const void *p)
{
    size_t y = (size_t)p;
    /* bottom 3 or 4 bits are likely to be 0; rotate y by 4 to avoid
       excessive hash collisions for dicts and sets */
    y = (y >> 4) | (y << (8 * SIZEOF_VOID_P - 4));
    return (Py_hash_t)y;
}```
quiet crane
#

In class methods, we can use super() to call methods in base class (MRO). Is there any way to reference static properties of base clases in the same way?

#

Something like this (which is NON functional python):

class CommonStuff():
    INTERESTING_ITEMS = ["Banana", "Potatos", "Carrot"]

class MoreStuff(CommonStuff):
    INTERESTING_ITEMS = super().INTERESTING_ITEMS + ["Spinach", "Stew", "Caraway"]
#

Is the implementation of super() heavily reliant on there actually being a self? (Since the earlier invocation of super had to look like super(CommonStuff, self))

spice pecan
#

IIRC it gets transformed at compile stage

#

So the change is purely syntactic, it fills the call with the same arguments as before, except automatically

quiet crane
#

yeah that seems like what I would have guessed.

#

Alas, it would have been useful in the above example

#

That wouldn't be enough since MoreStuff is not defined yet.

>>> class MoreStuff(CommonStuff):
...     INTERESTING_ITEMS = super(MoreStuff, None).INTERESTING_ITEMS + ["Spinach", "Stew", "Caraway"]
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in MoreStuff
NameError: name 'MoreStuff' is not defined
paper echo
quiet crane
paper echo
#

oh i was thinking inside a classmethod, sorry

#

no, it's not defined

#

you have to refer to the class by name

#

unfortunately you have to write INTERESTING_ITEMS = CommonStuff.INTERESTING_ITEMS + ["Spinach", "Stew", "Caraway"]

quiet crane
#

Yeah I know that works, just trying to find if there was aonther way ๐Ÿ™‚

verbal escarp
#

but it can get tricky if you want to use a list as key in dicts either way

paper echo
lusty scroll
#

but it'd be fairly useless, no?

paper echo
#

it'd be nice if class definition scoping wasn't so fucking weird anyway

verbal escarp
#

coming from you!

lusty scroll
#

if it's immutable 1) use a tuple, 2) if not, why are you hashing it

unborn yoke
#

cls

#

tf

paper echo
#

tuples are semantically different from lists!

#

!e ```python
class frozenlist(tuple):
def repr(self):
return 'โ„๏ธ'+repr(list(self))

def __str__(self):
    return 'โ„๏ธ'+str(list(self))

lst = frozenlist([1,2,3])
print(str(lst))
print(repr(lst))

lusty scroll
#

than an immutable list?

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

001 | โ„๏ธ[1, 2, 3]
002 | โ„๏ธ[1, 2, 3]
paper echo
verbal escarp
#

a long time ago i actually had a case where i (iirc) used hash(tuple(self)) to use a list as key in a dict

lusty scroll
#

what? tuples are record, says who? :p

verbal escarp
#

but don't ask me about the context

visual shadow
#

salt did just now! (in all seriousness though, yes tuples can carry extra meaning in context)

#

eg coordinate systems anyone? (x, y, z)

paper echo
#

yep, that, or a row from a database, or the result of ziping things together, etc.

visual shadow
#

this does give them an inherent semantic meaning above and beyond just being an immutable list

paper echo
#

a tuple is a heterogeneous fixed-size record, a list is a homogeneous (modulo Union) sequence of arbitrary/unknown-until-runtime size

verbal escarp
#

yeah, tuples basically scream "record!" and a list screams "homogenous output of a generator!"

#

but who listens..

lusty scroll
#

๐Ÿค”

quiet crane
paper echo
#

it's just naming imo

lusty scroll
#

yes

paper echo
#

but semantics matter a lot

#

otherwise we'd all be using perl

lusty scroll
#

but don't confuse convention with semantics

visual shadow
#

essentially semantics can make the same code easier or harder to understand and maintain, simply because we tend to rely on meanings a lot more often than we realise.

paper echo
verbal escarp
#

until they aren't and it's all anarchy \o/

lusty scroll
#

sounds like an oxymoron. "codified convention"?

verbal escarp
#

the way you can hack anything..

lusty scroll
#

who knows

paper echo
#

it's not an oxymoron, the point is that generally-accepted convention is no different from semantic meaning

#

and the difference between "what the compiler/parser accepts" and "what people just like to use" is kind of arbitrary

lusty scroll
#

that's your opinion

paper echo
#

that is indeed my opinion

#

and i will continue to reject tuples-that-should-be-lists in code review!

verbal escarp
#

PR nazi!

#

<.<

#

oh wait, that's me, too

lusty scroll
#

ok, it's settled then

verbal escarp
#

:p

lusty scroll
verbal escarp
#

me

#

wait, no

lusty scroll
#

they deserve to be smacked

verbal escarp
#

ahhh!

paper echo
#

i've seen it before, people wishing we had const like in javascript

#

(not that const means what that means anyway)

#

i.e. people wishing we had frozen lists

#

using tuples instead

lusty scroll
#

oh lmao

quiet crane
verbal escarp
#

pull request

#

on github, usually associated with a code review

quiet crane
#

yeah I know ^^ thx

verbal escarp
#

yw ^^

quiet crane
#

I have only done 1 of those, so the abbreviation doesnt auto expand ๐Ÿ˜›

verbal escarp
#

you'll get used to it ๐Ÿ˜„

lusty scroll
#

pytjon lists are linked lists, right

feral cedar
#

dynamic array

#

you get O(1) indexing

lusty scroll
#

is that what a tuple has too?

paper echo
#

"arrays of references" is the mental model

quiet crane
feral cedar
paper echo
#

it's literally an array of PyObject* in cpython, right?

quiet crane
lusty scroll
verbal escarp
#

is there any linked-anything in python? deque comes to mind.. heapq maybe?

feral cedar
#

heapq works on any mutable seq I think

#

deque is a linked list (of constant size arrays (to save memory))

lusty scroll
#

so the pointer chasing that's bandied about is just one pointer dereference?

#

two, I guess, for a list

paper echo
#

i've never heard people complain about "pointer chasing" in python fwiw

verbal escarp
#

then you haven't used weakrefs

#

fucking gremlins

feral cedar
#

i think it's just one of the ways people complain about python being slow

lusty scroll
#

yeah

#

something like lisp that has "real linked lists" would seem to demand much more traversing

quiet crane
verbal escarp
#

oh god.. that's how it all starts

quiet crane
#

haha

#

we have a not so good decorator now, that keeps references to all instantiatons of a class, eating a lot of memory during the run of the script. (It's memoizing class methods)

surreal sun
#

What are weakrefs exactly? Are they just a reference that doesnโ€™t get counted in the ref counting?

prime estuary
#

Yep. So you can call them to retrieve the object if it still exists, but they don't keep it alive.

#

If the object dies they change to return None.

surreal sun
#

Ahh

#

That makes sense, thanks

verbal escarp
#

well.. you should try to keep the linked object and context as simple as possible and you'll need a refcounter handy and you'll probably need to look into memory leaks

lusty scroll
#

for breaking cycles, is a big use case right

prime estuary
#

It's one yes.

paper echo
prime estuary
#

The other is stuff like caching, so when the object is otherwise unreferenced your cache doesn't keep it alive.

quiet crane
verbal escarp
#

yeah, for debugging

quiet crane
#

bah, it will work without debugging

#

๐Ÿ™„

prime estuary
#

One thing to note is that an alive weakref implements equality and hashing based on the object value (so you can simply create another weakref to do dict lookups), but when it's dead you must use the same weakref object to do so.

lusty scroll
#

they click when they're near a reference leak

verbal escarp
#

since anything anywhere can be a property and thus a potential cache of the linked object, you might miss a ref or two and you'll wonder why your program suddenly runs out of memory although everything should be a weakref

paper echo
#

i'm fortunate that most of the code i write is "stupid" and i have never had to use a weakref

prime estuary
#

You can probably just use the WeakKeyDict / WeakValueDict and not worry about directly using the ref, but if you do you should probably look at that code for some subtleties.

verbal escarp
#

gremlins, gremlins everywhere! crazymumble

lusty scroll
#

zombie qremlins?

verbal escarp
#

damn moonlight makes them all go haywire and sneak into your beautiful code, messing everything up while you're not looking

#

btw, python is #1 now - wonder how long that'll last :p and i'm curious what features people will come up with to keep the throne ๐Ÿ™‚

#

is there any big new change in the pipeline i haven't heard of yet? py311 seems boring compared to 310

quiet crane
#

Do you mean the swedish car registered with PEP310? (Volvo 850 Sedan 2.5 Automatisk, 170hk, 1995)

verbal escarp
#

i mean python3.11 :p

quiet crane
#

oh

verbal escarp
#

oh, there's a poll

#

https://discuss.python.org/t/should-loops-be-in-their-own-scope-poll/10593

Discussions on Python.org
Should loops be in their own scope? [poll]

In Python today, for loops run in the same scope as their enclosing block. Some languages have for loops create their own scope, as functions and classes do. Pythonโ€™s current behaviour is discussed here. Some people would like Python for each iteration of a loop to run in its own scope. That would probably solve the issue described in the top...

#

muaha

#

wait, how did that escalate from lambdas to loops

quiet crane
#

Well it's because the i in [lambda: i for i in range(10)] is "locked" and inaccessible after the comprehension is completed, right?

verbal escarp
#

but nobody would write that

quiet crane
#

what's wrong with the syntax?

verbal escarp
#

the lambda: i inside the comprehension is ambiguous

quiet crane
#

ambiguous?

verbal escarp
#

yes, it could mean [lambda: [0,1,2,3,4,5,6,7,8,9]]

quiet crane
#

oh, right

verbal escarp
#

but actually it means

#

[<function <listcomp>.<lambda> at 0x000001868E2C7430>, <function <listcomp>.<lambda> at 0x000001868E2C74C0>, <function <listcomp>.<lambda> at 0x000001868E2C7550>, <function <listcomp>.<lambda> at 0x000001868E2C75E0>, <function <listcomp>.<lambda> at 0x000001868E2C7670>, <function <listcomp>.<lambda> at 0x000001868E2C7700>, <function <listcomp>.<lambda> at 0x000001868E2C7790>, <function <listcomp>.<lambda> at 0x000001868E2C7820>, <function <listcomp>.<lambda> at 0x000001868E2C78B0>, <function <listcomp>.<lambda> at 0x000001868E2C7940>]

visual shadow
#

for the computer it's not ambiguous similar to how 2 + 42/2 is not ambiguous.

verbal escarp
#

actually, i'm tempted to post that as bug - it's absolutely not clear which should take precedence here

visual shadow
#

ie. it has a priority of parsing and it sticks to it. that we don't come across this priority much is on us, not on the computer

paper echo
#

!e ```python
funcs = []
for i in range(10):
funcs.append(lambda: i)
print([f() for f in funcs])

fallen slateBOT
#

@paper echo :white_check_mark: Your eval job has completed with return code 0.

[9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
paper echo
#

i don't see why you'd expect that to be any different in a list comprehension

#

loops don't have local scopes

#

get over it people, go home

quiet crane
#

its the same in both

paper echo
#

i am big -1 on local scopes in loops anyway

quiet crane
#

oh, thats what you mean

paper echo
#

but a local keyword that introduces a local scope, yes please

quiet crane
#

agreed

paper echo
# 
x = 10
local:
    x = 1
    y = 2
print(x)  # prints "10"
#

hell, give me a proper let

quiet crane
# 
for local i in range(10):
    funcs.append(lambda: i)

?

paper echo
# 
x = 10
local x = 1, y = 2:
    print(x)  # prints "1"
print(x)  # prints "10"
#

basically i want LET* from lisp

#

you can even call it let x = 1, y = 2:

verbal escarp
quiet crane
#

I would like where from haskell.

banana = x + diff * times where:
    diff = start - end
    times = total // step

or something

visual shadow
quiet crane
#

I guess it's similar to let

verbal escarp
visual shadow
verbal escarp
#

i don't accept it

visual shadow
#

fair. and all i know is, i'd rather type 2 + 42/2 because i prefer making life easier, not more difficult.

verbal escarp
#

the problem with the lambda and the comprehension is that it's too rarely used in conjunction so that people couldn't tell which one gets precedence - codified convention and all

quiet crane
#

Hmm, I saw a suggestion from someone in r/ProgrammingLanguages on a language that gave syntax errors if operators with different priority were used in the same expression without parenthesis. It's a nice idea I think.

#

It would help with the lambda vs comprehension-for, for example. Forcing you to provide parentheses to not get syntax error

visual shadow
#

i think practically it adds complexity to the language for very little benefit. clearly as established this scenario doesnt come up much at all, why should we have to add and maintain a special case for it. additionally, if you're not sure, use brackets to make things unambiguous. that's an advice that has always been good

verbal escarp
#

yeah, i see what you're getting at, but it might lead to a mess of () in normal circumstances..

visual shadow
#

essentially, this is a "nice to have" thing if you completely ignore the big picture and look at only this one instance

#

but there's always tradeoffs when taking decisions like this, they are worth considering.

quiet crane
#

It would be nice when mixing with shifters >> or % which I'm never 100% sure about precedence with

visual shadow
#

i'd say use brackets for those anyways

quiet crane
#

yes, but why not enforce it in the syntax?

#

Only way to control other programmers ๐Ÿ˜›

#

Of course you don't have to enforce it when just using /*-+

visual shadow
#

perhaps. without offering my personal opinion, i know python's stance on it is generally more towards open-ness and simplicity in the language, and conventions in cases where ambuigity needs to be resolved or intent needs to be clarified.

quiet crane
#

right

verbal escarp
# 
>>> lambda: i for i in range(10)
  File "<pyshell>", line 1
    lambda: i for i in range(10)
              ^
SyntaxError: invalid syntax
visual shadow
#

So at least in python it wont come up. basically, there is complexity and cost to these decisions that are worth thinking about

quiet crane
#

in that case, Im a bit surprised that pycodestyle doesn't complain about it, because I don't think it does?

visual shadow
#

without saying whether one is better or worse, just saying it's important to note that the goal of language itself needs to be simple too

visual shadow
quiet crane
#

Yes, it's just an interesting idea. I'm not really advocating for it in Python.

#

But what I have missed a few times is comprehension-like filtering in regular for-loops:

for f in fruits if f.size < 5:
    pass
verbal escarp
#

oh, that's a cute idea

raven ridge
#

all that saves is one line and one level of indent...