internals-and-peps

[nova iris]

an ABC is kind of different, it's like a blueprint with methods the subclass should supply and stuff (i'm kinda confused here)

well not really, a metaclass can do way more stuff

like hooking into class creation

although there are some times where a class decorator, __init_subclass__, or just plain old inheritance would do the job

so you should only learn metaclasses to understand python classes' inner workings

since you would almost NEVER use them

https://github.com/Objectivitix/dunder-magic/blob/main/easysubsuper.py here is a thing i made demonstrating a possible use for metaclasses

no idea. ask in #esoteric-python, this is beyond my knowledge 🤣

[static bluff]

I'd like ask you guy's advice

And if I'm not. I need to collect potentially hundreds of configuration settings from the user when they launch my framework. I think I might be up the creek in terms of static checking these settings, at least as far as things are configured now

class Application(object):

  def __init__(appSettings: AppSettings = {}, consoleSettings: ConsoleSettings = {},
      serverSettings: ServerSettings, cefSettings: CEFSettings = {}, cefSwitches: CEFSwitches = {},
  ):
  
    appSettings = AppSettings(appSettings)
    consoleSettings = ConsoleSettings(consoleSettings)
    serverSettings = ServerSettings(serverSettings)
    cefSettings = CEFSettings(cefSettings)
    cefSwitches = CEFSwitches(cefSwitches)

The idea, the original idea, would be that each of these annotations is a special subclass of TypedDict that's rigged up to enforce annotations on values at runtime. Each packet of settings would be passed in as a normal dictionary. This way when appSettings is provided, for example, the static checker would issue warnings to the IDE if there were any incorrect values.

And then when the normal dictionary appSettings was converted from a normal dict to an AppSettings object, runtime enforcement would occur. Runtime enforcement is as a secondary safety measure to keep the framework running properly, and to assist 'beginner' users of Python who have no concept of a static type checker

But static-checking on TypedDict's is, at least for PyCharm's linter, abysmal.

An example of a warning I get when I feed in a bad value is: expected type 'AppSettings', got 'dict[str, int]' instead

This is not a help message.

So I beseech you, Python Jedi of old. Is there a better way?

[nova iris]

hey, does anyone know why __prepare__ in a metaclass is supposed to be a class method? i found this from PEP 3115:

The __prepare__ method will most often be implemented as a class method rather than an instance method because it is called before the metaclass instance (i.e. the class itself) is created.

however, i still don't see why it's a class method and even less why it should be explicitly declared so. couldn't it be a static method? and couldn't it be like __class_getitem__, where it's implicitly transformed into one?

[verbal escarp]

i can give you a solution how i solved my config needs with attrs

from ast import literal_eval
import attr

class ConfigurationError(Exception):
    pass

class boolean:
    def __init__(self, x):
        if isinstance(x, str):
            try:
                self.x = bool(literal_eval(x))
            except ValueError:
                raise ConfigurationError(f"{x} is not a Value of True or False!")
        if isinstance(x, bool):
            self.x = x

    def __str__(self):
        return str(self.x)

    def __repr__(self):
        return repr(self.x)

    def __bool__(self):
        return self.x

@attr.s(
    auto_attribs=True,
    on_setattr=attr.setters.convert,
    field_transformer=lambda cls, fields: [
        field.evolve(converter=field.type) for field in fields
    ],
)
class Config:
    # "False" is a non-empty string -> True :|
    first_start: boolean = True
    lucky_num: int = 1
    count: int = 1
    telegram_user: int = 0
    telegram_token: str = None
    tictoc_volume: int = 50
    activity_color_body: str = "darkred"
    activity_color_mind: str = "darkblue"
    activity_color_spirit: str = "indigo"
    generated_faces_token: str = None
    tutorial_active: boolean = True

    def write(self):
        with open("config.stay", "w") as f:
            try:
                f.write(dump(attr.asdict(self)))
            except Exception as e:
              ...

and have Config(**d) from a json/toml/whatever

advantage is that everything is nicely typed and checked and defaulted, IDE-compatible

disadvantage is that it's a bit verbose, but i guess the boilerplate could be imported

[static bluff]

DMing

[paper echo]

Big +1 to attrs for this

You could translate the attribute names from kebab case to snake case if you wanted

Or implement your own __getitem__ that does the translation

That said I think the typeddict also makes sense

In the latter case you will need to figure out a way to "prove" to the type checker that a particular dict is in fact an instance of that specific typeddict

Working with a dependently typed language has given me a much better intuition for how type checking works

The alternative is something like Clojure spec, which you could build around TypedDict

https://youtu.be/Xb0UhDeHzBM

This is giving me some fun ideas, I will draft them up when I get home

[verbal escarp]

hehe

[paper echo]

Like an attrs class that also inherits from typeddict!

[verbal escarp]

salt rocks and their ideas

[paper echo]

Now the real magic of either syntactic macros or dependent types is that you can introspectively generate data types like that, without having to write out every attribute by hand

But you could also do something like generate python code from a jsonschema specification

I would much rather statically generate code than try to dynamically dispatch, but I think python developers tend to be afraid of build pipelines and code generation

Because if you have statically generated but properly annotated code you get to keep all your IDE tooling

[elder blade]

Isn't that just dataclasses?

Does anyone know what funny business TypedDict does? I want to do something like the following: ```py
from typing_extensions import TypedDict

class MyTest(TypedDict):
title: str
description: str

__getattr__ = __getitem__
__setattr__ = __setitem__

t = MyTest(title='Title Cased Title', description='See this one is a bit longer yeah')

Why is there so much magic involved for TypedDict?

[grave jolt]

@elder blade when you call MyTest, it returns a dict

[peak spoke]

I think it works similarly to NmaedTuples where you have a function that returns a type which directly inherits from dict to make it a normal dict at runtime, but restricted to only annotations

[elder blade]

Hmm, yeah thank you

[paper echo]

No because you don't have getitem

[elder blade]

Ah, right

[paper echo]

Does that example you posted actually work?

That's pretty slick if so

[elder blade]

No sadly not, __getitem__ is not defined

[paper echo]

Interesting

It's probably better to write it out anyway

[elder blade]

Honestly a bit disappointed more types don't incorporate the bitwise operators

They're super useful

Like lists would be awesome in my use-case

[peak spoke]

what functionality would you give them?

[elder blade]

[5] | [10] would become [5, 10]

I realize I can use addition now

🤦‍♂️

I guess you can AND like this as well: ```py
a = [5, 10, 20]
b = [5]

tempt = a - b
c = a - temp

[charred pilot]

how does subtraction on lists work?

[elder blade]

Am I just imaging stuff now? Maybe it's time for bed

!e ```py
a = [5, 10, 20]
b = [5]

tempt = a - b
c = a - temp
print(c)

[fallen slate]

@elder blade :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 4, in <module>
003 | TypeError: unsupported operand type(s) for -: 'list' and 'list'

[elder blade]

Yup

I'll go to bed now, see you all guys

[nova iris]

https://docs.python.org/3/reference/datamodel.html#customizing-class-creation details how classes are created. i noticed that nowhere in there says anything about MRO resolution... there is the resolution of MRO entries, but at what point are the __mro__ and mro attributes of the class defined?

i need this information to make a graph detailing every step of class creation, i'd greatly appreciate any help!!

[grave jolt]

!e
😉

from collections.abc import Set

class ListSet(Set):
    def __init__(self, iterable):
        self._values = []
        for item in iterable:
            if item not in self._values:
                self._values.append(item)

    def __contains__(self, value):
        return value in self._values

    def __iter__(self):
        return iter(self._values)

    def __len__(self):
        return len(self._value)

    def __repr__(self):
        return "ListSet({!r})".format(self._values)

a = ListSet([1, 2, 3, 4, 5])
b = ListSet([1, 2, 3, 6])
print(f"{a | b=}")
print(f"{a ^ b=}")
print(f"{a & b=}")
print(f"{a - b=}")

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

001 | a | b=ListSet([1, 2, 3, 4, 5, 6])
002 | a ^ b=ListSet([4, 5, 6])
003 | a & b=ListSet([1, 2, 3])
004 | a - b=ListSet([4, 5])

[nova iris]

oh god why

[grave jolt]

example stolen from Raymond Hettinger

it doesn't allow duplicates

[elder blade]

Please stay away from me and my family

[grave jolt]

in the __init__

I guess it's useful when you have a small amount of things, and those things aren't hashable

You could also shorten the implementation a little bit by using magic

@autorepr("_values")
@delegate("_values", ["__contains__", "__iter__", "__len__"])
class ListSet(Set):
    def __init__(self, iterable):
        self._values = []
        for item in iterable:
            if item not in self._values:
                self._values.append(item)

[nova iris]

https://docs.python.org/3/reference/datamodel.html#customizing-class-creation details how classes are created. i noticed that nowhere in there says anything about MRO resolution... there is the resolution of MRO entries, but at what point are the __mro__ and mro (method) attributes of the class defined?
i need this information to make a graph detailing every step of class creation, i'd greatly appreciate any help!!

wait a second- are they even defined at all in the class
NO WAY THEY'RE ATTRIBUTES OF THE METACLASS

AH THIS IS MAKING ME CRAZY

OH GOD WTF

__name__ is not in a class's __dict__???

[nova iris]

Python's absolutely convoluted af class creation system has led me to make this monstrosity of a graph

just how-

and i still don't understand 🤷

finally, 40 minutes of writing down to this final line:
"CLASS OBJECT CREATED"

i probably got a few things wrong

but eh, this was my best attempt lmao

[paper echo]

@nova iris if you can draw this up in a more "digital" format it might be a very useful resource for future readers

[robust anvil]

hello?

how can i make a website using C# and python?

[grave jolt]

Why do you want to use both?

[robust anvil]

my friend and I wanted to make a website but he's using c# and mine is python

and we dont have any idea on how to make one so

can anyone give me idea on how we can make it

[high pewter]

Python isn't generally used to make websites

Majority if not all websites will use an HTML base with python, C#, or Java(Rest in peace) elements

[grave jolt]

I think they meant the backend of the website

[grave jolt]

You can have two processes that communicate via sockets

either via HTTP or with your own protocol

[high pewter]

Is there anyway to add extra 'layers' of security to windows using a python program running in the back ground?

I'm looking to add both security to people physically getting my device and running malware, as well as network security

[charred wagon]

It's better to ask in #cybersecurity

[high pewter]

Cheers, There's so many channels I have no idea where to put things

[robust anvil]

oh, can we use c# and python for the elements?

[high pewter]

No elements, I meant that as in some sections of the websites

Not elements in the programming sense

Sorry, Poor word choice

Parts might have been a better word choice

[robust anvil]

oh okay thanks man!

uhm, one more favor, do you know any tutorials for it?

@high pewter

[high pewter]

Nothing for the backend stuff sorry

Try stack overflow for the python stuff, and W3 schools for html

[paper echo]

is it possible to hook into the import system somehow to keep track of which module a name was imported from?

[charred wagon]

Maybe... My memory of the internals of the import system are a bit vague right now but I think it might be at least aware temporarily of what is importing it

[pliant tusk]

You could also traverse stack frames to find out

[paper echo]

hypothetical output:

import my_magic_import_tracking_thing as mmitt

import ast
import math as py_math
from datetime import datetime as DateTime
from random import Random

assert mmitt.imports == [
    mmitt.ImportedModule('ast', path='/path/to/ast.py'),
    mmitt.ImportedModule('math', path='/path/to/math.py', alias='py_math'),
    mmitt.ImportedName(
        mmitt.ImportedModule('datetime'), 'datetime', alias='DateTime',
    ),
    mmitt.ImportedName(
        mmitt.ImportedModule('random'), 'Random',
    ),
]

the path= would be nice for debugging purposes but isn't necessary in the case i have in mind

the case i have in mind is, reloading individual top-level names by importlib.reload-ing the module, and then re-binding the appropriate names in globals() (or maybe even inside a smaller scope)

[pliant tusk]

You could just overwrite __builtins__.__import__

[paper echo]

that is mildly terrifying

[pliant tusk]

like inside mmitt.py you could do ```py
old_import = import
def my_import(name, *args, **kwargs):
print(name) # do something here
return old_import(name, *args, **kwargs)

builtins.import = my_import

[paper echo]

yeah, that makes sense

and overriding __builtins__ like that is truly global? and not just "module global"?

so i can do it in mmitt.py and it will affect all modules?

[pliant tusk]

yea it will affect all modules

[paper echo]

imports = []
old_import = __import__
def my_import(name, globals=None, locals=None, fromlist=(), level=0):
    imports.append((name, globals, locals, fromlist, level))
    return old_import(name, globals=globals, locals=locals, fromlist=fromlist, level=level)
__builtins__.__import__ = my_import

that's alarmingly easy

[pliant tusk]

yeah

[paper echo]

if i really wanted to be thorough i'd probably want to resolve the relative imports to absolute imports when storing this data

is there a function for that?

[pliant tusk]

as long as you call the original old_import you dont need to do anything else

oh

i dont know if there is

[paper echo]

e.g. if i'm inside the foo package, and in foo.utils i have x = 5, and in foo.things i have from .utils import x, i want to be able to do mmitt.reload('foo.utils') and and have x be re-bound when foo.utils is reloaded

i'd need to get the value of __name__ in the user's currently running module probably

which might be tricky (impossible?) if the code to do this is being imported from another module

maybe you'd have to pass the __name__ of the current module in order to resolve relative imports, as an extra feature

[pliant tusk]

old_import = __import__
def my_import(name, *args, **kwargs):
    frame = sys._getframe(1)
    print(frame.f_globals['__name__'])
    return old_import(name, *args, **kwargs)

(__builtins__ if isinstance(__builtins__, dict) else __builtins__.__dict__)['__import__'] = my_import
``` or use stack frames

[paper echo]

ooh

that could be useful for a logger setup function too

def new_logger():
    frame = sys._getframe(1)
    name = frame.f_globals['__name__']
    return logging.getLogger(name)

[pliant tusk]

yea

[paper echo]

(i actually really wish this was the default for getLogger() instead of the root logger)

this is a good start, thanks

[raven ridge]

You probably could do this all with an import hook. https://docs.python.org/3/reference/import.html#the-meta-path

[paper echo]

sys.meta_path would be good for tracking the actual filename of the imported module i think

[pliant tusk]

you could also just pull frame.f_globals['__spec__'] but import hooks are much cleaner

[paper echo]

i don't see how you could use the import hook machinery to track the names imported from a module, or the name that a module itself is bound to (with as)

in fact, can you even do that with __import__?

the docs suggest that you can't capture as bindings at all with the import machinery

[pliant tusk]

i mean you can cheat

[paper echo]

more stack frame magic?

[pliant tusk]

you can read the bytecode to look for the STORE_* or you could set a tracing function that'll get called later and look for the module in the namespace then

[paper echo]

thanks, i hate it

[pliant tusk]

yea like i said its cheating

but it would work

[paper echo]

wdym a tracing function?

[pliant tusk]

sys.settrace

you could also store the module and the namespace from the frame and lazy evaluate when you read from the log

so when you read from the log, search the namespace for the module

but then if del {name} is called you lose it

!docs sys.settrace

[fallen slate]

sys.settrace

sys.settrace(tracefunc)```
Set the system’s trace function, which allows you to implement a Python source code debugger in Python. The function is thread-specific; for a debugger to support multiple threads, it must register a trace function using [`settrace()`](https://docs.python.org/3/library/sys.html#sys.settrace "sys.settrace") for each thread being debugged or use [`threading.settrace()`](https://docs.python.org/3/library/threading.html#threading.settrace "threading.settrace").

Trace functions should have three arguments: *frame*, *event*, and *arg*. *frame* is the current stack frame. *event* is a string: `'call'`, `'line'`, `'return'`, `'exception'` or `'opcode'`. *arg* depends on the event type.

The trace function is invoked (with *event* set to `'call'`) whenever a new local scope is entered; it should return a reference to a local trace function to be used for the new scope, or `None` if the scope shouldn’t be traced.

[paper echo]

!eval ```python
from dis import dis
def f():
from foo import bar as BAR
dis(f)

[fallen slate]

@paper echo :white_check_mark: Your eval job has completed with return code 0.

001 |   3           0 LOAD_CONST               1 (0)
002 |               2 LOAD_CONST               2 (('bar',))
003 |               4 IMPORT_NAME              0 (foo)
004 |               6 IMPORT_FROM              1 (bar)
005 |               8 STORE_FAST               0 (BAR)
006 |              10 POP_TOP
007 |              12 LOAD_CONST               0 (None)
008 |              14 RETURN_VALUE

[paper echo]

so i'd have to look for IMPORT_* immediately followed by STORE_*

!eval ```python
from dis import dis
dis('from foo import bar as BAR')

[fallen slate]

@paper echo :white_check_mark: Your eval job has completed with return code 0.

001 |   1           0 LOAD_CONST               0 (0)
002 |               2 LOAD_CONST               1 (('bar',))
003 |               4 IMPORT_NAME              0 (foo)
004 |               6 IMPORT_FROM              1 (bar)
005 |               8 STORE_NAME               2 (BAR)
006 |              10 POP_TOP
007 |              12 LOAD_CONST               2 (None)
008 |              14 RETURN_VALUE

[paper echo]

!eval ```python
from dis import dis
dis('import a as aaaa; from a.b.c import hello as HellO')

[fallen slate]

@paper echo :white_check_mark: Your eval job has completed with return code 0.

001 |   1           0 LOAD_CONST               0 (0)
002 |               2 LOAD_CONST               1 (None)
003 |               4 IMPORT_NAME              0 (a)
004 |               6 STORE_NAME               1 (aaaa)
005 |               8 LOAD_CONST               0 (0)
006 |              10 LOAD_CONST               2 (('hello',))
007 |              12 IMPORT_NAME              2 (a.b.c)
008 |              14 IMPORT_FROM              3 (hello)
009 |              16 STORE_NAME               4 (HellO)
010 |              18 POP_TOP
011 |              20 LOAD_CONST               1 (None)
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/ijewemosip.txt?noredirect

[paper echo]

so i'd write a tracing function that looks for specific opcode sequences

seems very flaky

[pliant tusk]

!e ```py
import dis, sys

old_import = builtins.import

def my_import(name, *args, **kwargs):
frame = sys.getframe(1)
code = frame.f_code
co_code = code.co_code
lasti = frame.f_lasti
next_op = dis.opname[co_code[lasti + 2]]
next_arg = co_code[lasti + 3]
if 'IMPORT' in dis.opname[co_code[lasti]] and 'STORE_' in next_op:
if next_op in ['STORE_GLOBAL', 'STORE_NAME']:
print(name, 'as', code.co_names[next_arg], 'in', frame.f_globals['name'])
else:
print(name, 'as', code.co_varnames[next_arg], 'in', frame.f_globals['name'])
return old_import(name, *args, **kwargs)

builtins.import = my_import

import os as OS```

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

001 | os as OS in __main__
002 | abc as abc in os
003 | sys as sys in os
004 | stat as st in os
005 | posixpath as path in os
006 | os as os in posixpath
007 | sys as sys in posixpath
008 | stat as stat in posixpath
009 | genericpath as genericpath in posixpath
010 | os as os in genericpath
011 | stat as stat in genericpath
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/ukewogaloy.txt?noredirect

[pliant tusk]

@paper echo ^

[paper echo]

@pliant tusk that's horrifying but fascinating

makes sense though

i wonder if there are edge cases involved where it would pick up the wrong things

doesn't quite work for from ... import but i can start there at least

[vivid heath]

what's a good resource to learn how python works under the hood?

[elder blade]

There's a book from a core developer iirc

[verbal escarp]

github

a lot of details aren't even documented, so there's no substitute to looking at the source

[brave badger]

Anthony Shaw has a book on the matter
https://realpython.com/products/cpython-internals-book/

[verbal escarp]

there's so much deprecation going on in there.. it's surely nice to have a book as a reference, but i wouldn't count on it

well, at least it says 3.9, so it's fairly recent

[nova iris]

Really? I'll get started working on it today! Then maybe you can help me find some errors (the documentation was very convoluted, I worked with what I have and most of the time I needed to run a bunch of experiments to confirm the docs)

[nova iris]

Oh god frame objects

[verbal escarp]

you might want to check the regular help channels

[verbal escarp]

inspect.currentframe().f_back.f_back.f_code.co_filename - gotta love code like that ^^

[paper echo]

I guarantee you know more about it than I do, diagrams are always valuable resource

[unkempt rock]

D

[elder blade]

Dammit https://www.python.org/dev/peps/pep-0655/

I needed this aaahh

[nova iris]

right. i don't really have a good flowchart creator though...

oh wait, perhaps google slides can work

[charred wagon]

The syntax with the operators would've been pretty nice

[cyan elbow]

Im.having a issue with Jupyter I installed metpy but it's not detecting it

[nova iris]

part one is done! i'll work on part 2 later today, gotta go check out 3.10 now
sneak peak vvv

[paper echo]

nice

anything is better than nothing!

there are some nice diagram/flowchart creators online

you could use graphviz or mermaid.js

[nova iris]

yeah, i tried some but i somehow think that google slides is the easiest 😂

perhaps someone could yeet it into a full blown graph after i finish this prototype

[nova iris]

python's new __match_args__ is gamebreaking

indeed

and me being the python-breaking object-apprentice i am

i must find your dunders

lmao yeah, just yeet __match_args__ everywhere to mess things up

imagine: the order of __init__ is different than the order in match syntax-

whaaaat

c'mon

i might be doing it wrong

i can already see the match case enthusiasts making whole discord bots whose code is only match case

[paper echo]

thanks, i hate it

i think one of the big design innovations of "modern" programming languages is everything-is-an-expression

[flat gazelle]

Yeah, it is quite nice. One more good thing from functional programming

[paper echo]

the more i try to write code with match/case the less i want to

i think it'd be a great idea in Hy because of syntactic macros, or in Coconut where the syntax is already a superset of python

[verbal escarp]

what about powerpoint (no joke)

[nova iris]

hm, true

flashbacks to when print became a function

[nova iris]

lol, soon you're going to get global and nonlocal and similar statements as functions

[nova iris]

hm, could work, but i don't have powerpoint-

[verbal escarp]

i also produced flowcharts with inkscape a while ago

it has a nice connector tool

[true ridge]

how so? global or nonlocal are just directives to the compiler, so it doesn't make sense to treat them like functions

[verbal escarp]

it might actually be a great tool for this job with layers, infinite zoom, gradients etc..

[verbal escarp]

how come?

i really should give match/case a spin myself 😒

[paper echo]

this. print() should always have been a function, like open(), because it represents a "computation". global and nonlocal don't really represent "computation". and import kind of needs to be not-a-function because it needs to capture raw "symbols"; otherwise you'd be stuck with strings

yet another reason why syntactic macros are so nice - fewer special cases in the parser

[verbal escarp]

import really should be a function 😉

[paper echo]

no, import should be a macro that expands to importlib.import_module !!

[verbal escarp]

import should be a function, otherwise you need to change the syntax every time you want to change something, which is really a big issue

if you want it as a kind of macro, you also need special syntax for that

[paper echo]

but then you're stuck passing strings for import

which a lot of languages have (go, js) and i think it's anti-pythonic

import numpy

versus

numpy = import('numpy')

[verbal escarp]

it's not anti-pythonic, it's explicit

[paper echo]

unless you have import() also modify the current frame's globals

[verbal escarp]

there is a very specific point in the zen of python about this, btw

[true ridge]

I wouldn't expect something anything like that (e.g tracing a frame back, updating the locals) from a builtin function

[verbal escarp]

Special cases aren't special enough to break the rules.

[paper echo]

yes, and the rules are "no capturing unevaluated expressions / raw symbols" and "no dynamic scope"

[true ridge]

import as a statement is a wise choice, considering that both there are actions that are required by the compiler to recognize it is changing locals (in the symbol table pass) and also working with raw identifiers (as @paper echo commented)

[verbal escarp]

why is import so very special that one needs to invent a whole DSL just for this case?

[paper echo]

because it's a fundamental operation and numpy = import('numpy') is ugly and repetitive

and by "is" i mean "is, from the perspective of guido in the 90s"

look at javascript, i don't think imports are ugly there at all

or lua

and they do use the explicit version you propose

i personally like it too

the one downside is that you can't do from foo import *

one could argue that's a good thing, star imports are bad

but i don't know how else you'd do it other than something like import_all('foo', globals())

[verbal escarp]

well, your conundrum could be solved in a different way

[paper echo]

import('foo', to=globals())

[verbal escarp]

exactly

[true ridge]

And another thing is that, you can't do local imports

[verbal escarp]

take the name, assign it as a global var within the import func

[paper echo]

you can do local imports but the modules are loaded and cached globally

[true ridge]

if the compiler doesn't know the imported symbols

it can't lexically analyze the scope (per PEP 227)

[paper echo]

wait, what do you mean by "local" in this case

[true ridge]

def something():
    import x, y

[verbal escarp]

x, y are names, what's the problem with that?

[true ridge]

x and y needs to be explicitly available to the compiler, so that it can construct the scope of something and know it defines x and y at the compile time

[paper echo]

def die(status, message=None):
    import sys
    if message is not None:
        print(message, file=sys.stderr)
    sys.exit(status)

this works fine in python

but sys is executed and loaded once, globally, for the entire application, and cached

[true ridge]

what I mean is that, if you make import an expression like this

[verbal escarp]

yup

[paper echo]

even if it's only bound to sys in the die namespace

[true ridge]

and pass import('foo', to=locals())

[paper echo]

ohhh yeah

[true ridge]

it simple wouldn't work

[paper echo]

yep

[verbal escarp]

sure, caching is a problem

[true ridge]

since you can't change the local variables defined in the function's scope (per PEP 227)

[paper echo]

at least it can't be analyzed statically anymore

[true ridge]

this is not about caching

[verbal escarp]

it kind of is, at least that's what the internal code is all about

[paper echo]

@verbal escarp i think they're saying that with import('foo', to=...) it becomes impossible to statically determine where the imported module names should be bound

[verbal escarp]

oh

[paper echo]

you can add a special case check for literal globals(), but otherwise you're on your own

which... maybe is fine

maybe we should have __globals__ instead? idk

[verbal escarp]

sorry, i don't understand the issue with the statical determination

[true ridge]

unlike the global scope, locals are lexically scoped which means that anything that defines new locals should be available to the compiler

e.g a = b is explicitly defining a, or import x as y is explicitly defining y

if you make import a function, then you need to be bound to the python's mechanics, which would mean that it would be impossible* to statically determine which names are imported

unless you continue to use assignment syntax, e.g a, b = import('a', 'b') which brings back the issue of being redundant and repetitive

[paper echo]

well hold on

you can add special cases for: 1) string literals, and 2) globals()/locals()

[true ridge]

what do you mean by 1 and 2?

[paper echo]

so ```python
def foo():
import('a', to=locals())

can be statically analyzed, it's isomorphic to
```python
def foo():
    import a

but

def foo(mod):
    import(mod, to=locals())

is no better than

def foo(mod):
    l = locals()
    l[mod] = importlib.import_module(mod)

[true ridge]

no, not really. How could you understand import is indeed the builtins.import

[paper echo]

make it a keyword 😛

like True

the argument here is that it shouldn't be a statement with custom syntax, it can still be disallowed to assign to that particular name for safety

[true ridge]

then, you still need to syntactically represent it and differ it (like True is a Constant but not a Name), which would hit this issue

but yes, if you make it a keyword

then you can analyze it

and once you do that, you can actually keep using raw identifiers

instead of strings

since it would be handled by a separate rule in the parser (and in the compiler/vm), which I think would be more troubling then the current version + the illusion it causes

[peak spoke]

how would from imports and renames work with it being a func? Having to handle that manually through a sequence sounds messy

[pliant tusk]

this cannot work with how local variables work. local names have to all be known at compile time, thats why py def foo(): from FOO import name is valid but py def foo(): from FOO import * is not

[paper echo]

i never actually tried the latter

so then fine, don't add the special case for to=locals() and just treat it as a dynamic import

[pliant tusk]

!e py def foo(): from FOO import *

[fallen slate]

@pliant tusk :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 | SyntaxError: import * only allowed at module level

[pliant tusk]

you cannot currently add names to a local namespace at runtime

[paper echo]

great, that's one less special case to worry about

[pliant tusk]

(without shenanigans like rebuilding the frame object at runtime)

[paper echo]

but you can do something along the lines of

def foo():
    l = locals()
    l.update(importlib.import_module('foo').__dict__)

[pliant tusk]

nope you cannot

the dictionary returned by locals() is a snapshot, not a live mapping like globals()

[paper echo]

interesting, also TIL

[peak spoke]

In [7]: def foo():
   ...:     l = locals()
   ...:     l.update(a=1)
   ...:     print(a)
   ...:

In [8]: foo()
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-8-c19b6d9633cf> in <module>
----> 1 foo()

<ipython-input-7-615daf686f26> in foo()
      2     l = locals()
      3     l.update(a=1)
----> 4     print(a)
      5
NameError: name 'a' is not defined

[paper echo]

in that case there's no need to even bother trying to analyze that case

[pliant tusk]

its due to how local variables are implemented

[paper echo]

that's a good thing

[pliant tusk]

to make them faster they are essentially just stored in a c array instead of a whole python dictionary

[verbal escarp]

one less special case to worry about ^^

[paper echo]

so the special cases you have to check for are:

1. import('foo', to=globals()) at the top level only
2. foo = import('foo')
3. a = import('foo', 'a'), or fun combinations thereof like (a, b, c), (x, y, z) = import('foo', 'a', 'b', 'c'), import('bar', 'x', 'y', 'z')

if you get those 3, i'm pretty sure import() is isomorphic to the existing import statement with respect to what can be statically determined

maybe i missed a case or two

otherwise you can treat it as importlib.import_module

[verbal escarp]

well, there is a different problem

thinking of packages with long chains of a.b.c.d.e

would it be import("a", "b.c.d.e") or import("a.b.c.d.e") or import("a.b.c.d", "e")? and what should the variable in the namespace look like?

[paper echo]

yeah well now you're starting to see why sometimes you just want special syntax even if you don't need it 😉

but the point isn't import isn't a statement because it has to be, it's because if it weren't a statement it'd be a lot less ergonomic

unlike nonlocal which kind of has to be a statement for fundamental technical reasons

conversely, print was less ergonomic as a statement than as a function

[verbal escarp]

you know how strongly i think that import is non-ergonomic :p

one simple example

import assumes to know what you want, which makes it very hard to convince it to do something different if you need it to

like.. import something from a parent folder

or import a module instead of a similarly-named package

[paper echo]

well "folders" are an entirely different can of worms

as is the precedence between foo.py and foo/__init__.py

those are design decisions unrelated to the import statement syntax

the #1 biggest problem with packages is the name "package"

[verbal escarp]

"distribution" is the name used on pypi

[paper echo]

"distribution package" really

versus "import package"

that's what the python docs glossary says, i don't know when that was formalized

[verbal escarp]

which is quite confusing to me, coming from a linux background, but that's a different issue

[paper echo]

yes, well that's the problem

the name is totally fucking confusing

[verbal escarp]

yup!

[paper echo]

and misleading

again, another case where lua did it right

[verbal escarp]

which term are they using?

[paper echo]

"module" for everything

foo.lua and foo/init.lua

and require('foo') i think prefers the latter over the former like in python, but don't quote me on that

[verbal escarp]

hm.. yeah, okay. not sure if using a single term for everything does away with all ambiguities but ymmv

[verbal escarp]

i think so, sounds right

btw, another thing i got annoyed with, regarding imports - they are case-insensitive

i thought it was just a convention to use lower-case module names, but it's actually enforced

which might actually be another reason why import-as-a-function with the (on first glance) redundant mod = import("mod") syntax would be a good thing

you could kick most naming restrictions to the bin and allow filenames like ".testA.py" etc without interfering with the internal naming

[paper echo]

wait what

[verbal escarp]

what what?

[paper echo]

i don't think they're case-insensitive... are they?

on my mac, the filesystem is case insensitive

[verbal escarp]

last time i checked they were

[paper echo]

so maybe that's what's happening

cat <<< 'x = 3' > a.py
python -c 'import A'
# Traceback (most recent call last):
#   File "<string>", line 1, in <module>
# ModuleNotFoundError: No module named 'A'

cat <<< 'x = 4' > B.py
python -c 'import b'
# Traceback (most recent call last):
#   File "<string>", line 1, in <module>
# ModuleNotFoundError: No module named 'b'

[verbal escarp]

i stand corrected. although, the leading-. convention still doesn't work with .py files

[paper echo]

wait, why would it?

you would want .foo.py to be "hidden" from the import system?

or you would want it to be not hidden?

pretty sure python can't import .foo.py

[verbal escarp]

exactly

[paper echo]

would you want python to import foo.py as foo? or _foo? or something else?

[verbal escarp]

for example, i'd like to have .test files that aren't run by my testrunner but that i still could import if i explicitely ask to

i don't think that's too unreasonable, is it?

with import-as-a-function that would be simply doable via foo = import(".test.py") or somesuch, no problem

[paper echo]

meh

just tell your test runner to ignore it 🤷‍♂️

[verbal escarp]

let's be real for a second. most times, you don't just import foo

BUT you do import numpy as np

or import tensorflow as tf

and that isn't any better than `tf = import("tensorflow")

[paper echo]

imagine writing this with import():

from sklearn.feature_extraction.text import HashingVectorizer

you'd have to add a special case for a.b.c imports otherwise users would go nuts

HashingVectorizer = import('sklearn.feature_extraction.text', 'HashingVectorizer')

of course if you have syntactic macros you can write a macro like the former that expands to the latter

[verbal escarp]

hmm.. hold on a sec

how about a special syntax for import modules like..

import("sklearn.feature_extraction.text") * "HashingVectorizer"

it's a bit wonky, but might get the job done

i stole the idea from pathlib

path / "somefile" -> path

import("sklearn.feature_extraction.text").HashingVectorizer would also work, but that doesn't assign the variable to the name

so i figured * vs .

sadly : is not a valid operator, that would fit better

well, you see i'm on the fence myself ^^

[nova iris]

hmm... why not? don't functions perform a particular task?

[nova iris]

yes please... and as syntax can be just an optional parameter

[verbal escarp]

but then i'd like to have as as a general syntax for any assignment 😉

[clever shoal]

Hey ma dudes

Does anyone know any dunder method cheat sheets

Like im tryna become an expert at oop

[unkempt rock]

https://docs.python.org/3/reference/datamodel.html#special-method-names The docs has them all

[clever shoal]

May you produce healthy offspring

Lol

👍

[verbal escarp]

actually, it might be feasible to pass in a regex or simplified pattern matching, and with ~ instead as operator a "star-import" might look something like import("sklearn.feature_extraction.text") ~ "*"

import("sklearn.feature_extraction.text") ~ "*Vectorizer" could import all kinds of Vectorizers

[verbal escarp]

meh. ~ is unary 😦

can't even overload

[paper echo]

we hashed this all out above 🙂

nonlocal and global are more or less declarations or compiler directives

[verbal escarp]

"*Vectorizer" in import("sklearn.feature_extraction.text") on the other hand..

wished "from" would map to a method..

no, "from" as a keyword, like "in"

not what i'm asking 😉

[pliant tusk]

!e ```py
import dis, sys

old_import = builtins.import

def my_import(name, *args, **kwargs):
try:
frame = sys.getframe(1)
except:
return old_import(name, *args, **kwargs)
code = frame.f_code
co_code = code.co_code
lasti = frame.f_lasti
next_op = dis.opname[co_code[lasti + 2]]
next_arg = co_code[lasti + 3]
if 'STORE' in next_op:
if next_op in ['STORE_GLOBAL', 'STORE_NAME']:
print(f'importing {name} as {code.co_names[next_arg]} in {frame.f_globals["name"]}.{code.co_name}')
else:
print(f'importing {name} as {code.co_varnames[next_arg]} in {frame.f_globals["name"]}.{code.co_name}')
else:
print(f'importing {name} in {frame.f_globals["name"]}.{code.co_name}')
return old_import(name, *args, **kwargs)

builtins.import = my_import

import os```

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

001 | importing os as os in __main__.<module>
002 | importing _io in _frozen_importlib_external.get_data
003 | importing abc as abc in os.<module>
004 | importing sys as sys in os.<module>
005 | importing stat as st in os.<module>
006 | importing _io in _frozen_importlib_external.get_data
007 | importing _stat in stat.<module>
008 | importing _collections_abc in os.<module>
009 | importing posix in os.<module>
010 | importing posix in os.<module>
011 | importing posixpath as path in os.<module>
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/godecikoko.txt?noredirect

[pliant tusk]

@paper echo ^

[charred wagon]

This is not a help channel.If you need help please see #❓｜how-to-get-help

[paper echo]

this is getting hairy 👀

[pliant tusk]

!e ```py
import dis, sys

old_import = builtins.import

def my_import(name, globals=None, locals=None, fromlist=(), level=0):
try:
frame = sys.getframe(1)
except:
return old_import(name, globals, locals, fromlist, level)
loc = frame.f_globals["name"]
code = frame.f_code
co_code = code.co_code
lasti = frame.f_lasti
next_op = dis.opname[co_code[lasti + 2]]
next_arg = co_code[lasti + 3]
fl = ', '.join(fromlist) + ' from ' if fromlist else ''
if 'STORE' in next_op:
if next_op in ['STORE_GLOBAL', 'STORE_NAME']:
print(f'importing {fl}{name} as {code.co_names[next_arg]} in {loc}.{code.co_name}')
else:
print(f'importing {fl}{name} as {code.co_varnames[next_arg]} in {loc}.{code.co_name}')
else:
print(f'importing {fl}{name} in {loc}.{code.co_name}')
return old_import(name, globals, locals, fromlist, level)

builtins.import = my_import

import os```

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

001 | importing os as os in __main__.<module>
002 | importing _io in _frozen_importlib_external.get_data
003 | importing abc as abc in os.<module>
004 | importing sys as sys in os.<module>
005 | importing stat as st in os.<module>
006 | importing _io in _frozen_importlib_external.get_data
007 | importing * from _stat in stat.<module>
008 | importing _check_methods from _collections_abc in os.<module>
009 | importing * from posix in os.<module>
010 | importing _exit from posix in os.<module>
011 | importing posixpath as path in os.<module>
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/goqakahewe.txt?noredirect

[pliant tusk]

it got worse

[paper echo]

what's the try at the top?

[pliant tusk]

oh because __import__ is sometimes called from c code where there is no upper frame

[paper echo]

ahh

all this just to resolve import as'ed names for module reloading

the ultimate idea would be something like, invoke a reload() function or %reload ipython magic, and then it would:

1. re-exec the reloaded module
2. go through all the tracked imports and overwrite all top-level bindings w/ the new versions from the newly-exec'ed module

possibly something @verbal escarp might want to pursue ☝️

[pliant tusk]

that should be fairly simple now that we can get all the info we need from hooking __import__

[grave jolt]

!ban 750923801885409321 It seems like you're only here to post a referral link.

[fallen slate]

:incoming_envelope: :ok_hand: applied ban to @rugged cape permanently.

[paper echo]

@pliant tusk this is actually really helpful, this would be a nontrivial productivity boost in notebook environments. do you know if there's a way to figure out which modules were loaded into the "current" globals? i wouldn't want to go around re-binding names that shouldn't be re-bound

it looks like i might also have to do a topological sort of the import tree and reload each imported module from the bottom up

i do not have the stomach for such a project right now

[pliant tusk]

you could check if the __name__ of the globals is __main__

[paper echo]

hmm

i think ipython uses something other than __main__ for the name, but yeah that seems reasonable

i could make it a callable class instead of a function and check __name__ against self.module_name

ooh it's missing one more edge case

from datetime import datetime as DateTime, time as Time

importing datetime, time from datetime in __main__.<module>

it's missing the aliases for each of the imports... i'll try to take a crack at it using the pattern you laid down

[pliant tusk]

@paper echo loop over fromlist and increment lasti by 2 each time

wait not that simple, python does something weird here

ah my code doesnt handle from foo import bar as Bar either

[verbal escarp]

heh :p well, my version of module reloading works, and i'm happy with it 😉

i find it much more convenient to just go ahead and attempt to reload than requiring the user to trigger it

going through the bound names also doesn't resolve the lingering-object issue

checking if the module is function-only is a nice assurance

it also helps to not use __import__ for this because of all the caching

[boreal umbra]

@stuck valley @unkempt rock This channel is not an alternative to esoteric python. Be sure to stay on topic for this channel.

[stuck valley]

Understood. Thank you.

[nova iris]

"The line between advanced and esoteric is thin."
--- me, 2021

man, metaclasses are really messing with my head

[boreal umbra]

We make an effort to keep this channel especially on-topic. Feel free to leave any thoughts on that in #community-meta

[nova iris]

oh yes lol, i just said that as a meme

sometimes it's easy to yeet into esotericness (well, not really but)

[boreal umbra]

What are you finding confusing about meta classes?

[nova iris]

...a lot of stuff lol
first off, according to the class creation process documentation, __prepare__ can accept optional keyword arguments; however, those kwargs are also passed to the immediate superclass's __init_subclass__. then what happens when we want to pass kwargs into __prepare__ but not into the latter, since it might not be implemented for the specified kwargs?

quick and dirty fix could be to overwrite __init_subclass__ of a superclass to do nothing

but that is unelegant

secondly, why is it that __bases__ and __mro__ and such attributes aren't stored in a class's __dict__, but instead i see them as slot members of the metaclass? this is super weird, why are they descriptors?

thirdly, when in the class creation process is a class's MRO determined? in the metaclass? before during MRO entries resolution? the documentation does not say explicitly

and there are probably some more that i can't think of at the top of my head right now

ah yes, the entirety of this weird section https://docs.python.org/3/reference/datamodel.html#special-method-lookup

Incorrectly attempting to invoke an unbound method of a class in this way is sometimes referred to as ‘metaclass confusion’
you don't say 🤣

OH WAIT NEVERMIND I UNDERSTOOD THIS LAST ONE

[fallen slate]

Hey @unkempt rock!

It looks like you tried to attach file type(s) that we do not allow (.zip). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a.

Feel free to ask in #community-meta if you think this is a mistake.

[chrome summit]

Hello

Print(“Hello Guys”);

[nova iris]

MAN THESE ERRORS ARE SO GOOD IN 3.10

[cyan elbow]

Finally got some good results

[charred wagon]

Interesting that at one point long ago, there were plans for access modifiers https://github.com/python/cpython/commit/a75d306e2b799aa891666899ca973bec82b2362b#diff-bf5934ae010cae0082fec9a020b9bd2deb9f39721b6886332a35f26d73845adcR472-R482

[main ginkgo]

wow

now thats some deep lore

[elder blade]

I don't understand this, what would access modifiers mean to Python?

Is it like the difference between importing type definitions / symbols and running the file?

[lusty scroll]

sounds like they meant "visibility" -- which, I'm very glad we didn't get

[elder blade]

Ooooh, so like private, public stuff?

[charred wagon]

Did you read the documentation in the link? It states it's for class attributes. Like private/protected in Java

[elder blade]

Yeah that's a part I really love about Python

[elder blade]

Oh that was documentation? I thought that was changes to the parser or like grammar file

[lusty scroll]

just part of a commit message that i saw

yeah underscores do a fine job of communicating the stability of the interface, imo

[stuck valley]

meanwhile in 3.9.1

https://cdn.discordapp.com/attachments/746994822593642616/875635624395280424/unknown.png

https://cdn.discordapp.com/attachments/746994822593642616/872718285702066176/unknown.png

(to be fair the last one was excactly what I should have expected)

[cloud crypt]

yeah well

it should be c_uint32 or c_uint16 depending on the platform I think

due to int representation in memory

[stuck valley]

that's not the issue

I don't recall it precisely

[cloud crypt]

nope, it definitely is

you're setting higher bits to 5

[stuck valley]

>>> ctypes.cast(id(4),ctypes.POINTER(ctypes.c_int))[6]=5
>>> 2+2
5

the better way

[nova iris]

what.

[nova iris]

LMFAO WTF

[lusty scroll]

offset of 16 would be two 8-byte fields, right? for ob_type and ob_size ?

[nova iris]

what.

[lusty scroll]

curious what craziness is happening in datetime to break dir...

[lusty scroll]

the REPL running your code through some kind of socket, and the remote process crashed, would be my guess

[prime estuary]

It sorts the return value, looks like someone used a function as a __dict__ key somehow…

[nova iris]

didn't break for me lol

[nova iris]

ah, i see, this happened with anonymous as well

welp, guess i can't perform any ctypes dark magic lol

[lusty scroll]

rip lol

if there's a python on your box you could always rough it with regular python i guess

[stuck valley]

yeah the ctypes voodoo occasionally crashes

[lusty scroll]

occasionally haha

[stuck valley]

this is from a personal #esoteric-python project

[verbal escarp]

soo.. what's the big advantage of subinterpreters? ( https://www.python.org/dev/peps/pep-0554/ )

[elder blade]

As far as I know, right now the advantages are small and the PEP seems kind of forgotten.

There's a potential to have separate GILs for each subinterpreter, and that's when you get advantages in concurrency.

[verbal escarp]

basically threading with GILs

hm.. ok

[tulip cloud]

I need help, I want to distribute a shared library with statically linked the python interpreter + a statically linked cython extension. I already found how statically link the python interpreter to the shared library but I'm unable to link the cython extension to it. Anyone knows how do that?

[main ginkgo]

if there ends up being a way to share objects directly between the interpreters without any kind of serialization thatll be a pretty big boost. youll be able to get the same kind of parallelism that multiprocessing provides minus a lot of overhead

[verbal escarp]

as far as i'm aware their current approach to that is to put everything into strings, which is the contrary of efficient ^^

[main ginkgo]

yeah, at the moment the design is based on serialization and message passing. but the potential is there, and the groundwork is at least being done.

[verbal escarp]

maybe some kind of transactional memory sharing?

[main ginkgo]

that would be ideal yeah

[verbal escarp]

@lusty scroll you hear? make a note on our list of problems-to-be-solved 😉

it should be put right after "save the climate" ;p

[lusty scroll]

who knows, maybe we remove GIL too while we're at it 😂

[verbal escarp]

sure!

what could possibly go wrong 😄

i wonder if it would help if we had more people paid to work on python for these kinds of tasks

or.. we could build a system that uses genetic programming and ML to solve those issues for us! muahahaha

[main ginkgo]

think the difficult aspect would be dealing with the refcounting in a way that doesnt significantly degrade performance. might require a sizeable rework of the current gc. sharing semantics would be pretty straightforward. just a mutex and some kind of api for a subinterpreter to say 'pls let me acquire this object, okay im done with it here you go'

[verbal escarp]

actually, i'm only half-joking regarding the code-generation. a machine might come up with a totally novel solution humans couldn't think of since we're stuck in a rut

it would definitely be an interesting project 🙂

[broken furnace]

definitely agree

[stuck valley]

like what, skynet?

[verbal escarp]

pff :p

[stuck valley]

#esoteric-python would have a field day if interpreter goes live

[main ginkgo]

a method like this would have to be massively tweaked for maintainability in the case of something like cpython, otherwise we'd end up with code that would be indecipherable to humans. in which case, it might be less effort to just let humans figure out the problem in the first place.

you see this already sometimes in things like neural networks and auto generated fpgas. they work, but the designers dont have a clue how or why because of how complex the result is.

[verbal escarp]

i don't think that would be the biggest issue

just mutate the AST until you're happy with the performance, then run black over the AST, then sourcery for style, then another pass of black

done

[main ginkgo]

prettifying some code isnt enough to make it maintainable or understandable. mutating code without consideration for thing like adherence to the conventions of the codebase, ease of extensibility, code smells, etc, will lead to some ugly outcomes despite performance gains

also isnt black a python formatter? this would have to be done on the c level

[verbal escarp]

sourcery takes care of code smells, black takes care of conventions

ahh..

urgs.

[main ginkgo]

and by conventions i mean things that might be specific to the cpython codebase itself, e.g. the use of certain macros where necessary

something that wouldnt be easily generalized to all c code

[lusty scroll]

well it would be very, very slow but you might be able to do it in pure Python? at least as a proof of concept

and start with

import ast

that code practically wrote itself

[verbal escarp]

:p

i'm not actually sure if it's slower in python than in c

the notion of an AST in python makes this endeavour actually feasible

in c you could mutate goto and get nowhere

[main ginkgo]

i think theres a misunderstanding. this is something that would have to be explicitly supported at the c level to really work. the machinery needed to safely share an object between subinterpreters just isnt there at the moment. though you could maybe hack together something via ctypes but thats all it would be, a hack.

[broken furnace]

My thought was that if everything was built on top of the subinterpreters channel interface, maybe you could hack together enough marshaling/un-marshaling to make it somewhat approachable, though there would be no actual shared memory space of course. I don't know how useful that would be in practice.

[elder blade]

I can see uses for it with passing strings and what-not

For example APIs, just don't deserialize the payload on the "main" interpreter

[broken furnace]

yeah exactly

[elder blade]

Are you sure, subinterpreters have been in Python for a long time through the C API but never had a standard library for it or much attention at all

[main ginkgo]

afaik cpython's reference counting has no awareness of which interpreter(s) are currently using an object, so there is no synchronization for updating a shared refcount. more like each subinterpreter gets its own separate garbage collector.

that being said https://docs.python.org/3/c-api/refcounting.html

The following functions are for runtime dynamic embedding of Python: Py_IncRef(PyObject *o), Py_DecRef(PyObject *o). They are simply exported function versions of Py_XINCREF() and Py_XDECREF(), respectively.
you could access Py_IncRef/DecRef at runtime via ctypes.pythonapi

and if you an pass an object's id to a subinterpreter, it should in theory be accessible via ctypes.cast(id_of_object, ctypes.py_object).

im actually not sure how ids and addresses would in the context of subinterpreters

do object like 1 have the same id in both for example?

[peak spoke]

I think the integer cache was already changed to be per interpreter, with the goal of every object not being shared

[mighty pasture]

So i am getting the _tkinter.TclError and i don't really understand what is wrong. So could someone please tell me ?

[grave jolt]

@mighty pasture You should see #❓｜how-to-get-help and claim a help channel, or ask in #user-interfaces

[broken furnace]

surely not; they don't share objects, at least that was my impression from the PEP.

[main ginkgo]

was thinking that immutable singletons like 1 wont really even need reference counting to begin with, so maybe theyre shared

can see how it would probably be easier to deal with a separate int cache for each from a interpreter initialization/destruction point of view though

[main ginkgo]

do you know if there's a pep or tracker issue or something that covers this

https://bugs.python.org/issue39511 nevermind there it is

[lusty scroll]

ah, not sure

[verbal escarp]

heh.. i googled just for fun, this is curious - https://blog.christianperone.com/tag/genetic-programming/

i guess the secret sauce to genetic programming on the AST is to use a restricted set

lmao https://2021.pycon.org.au/program/mfunxt/

Sat September 11, 2021

in a month this is going to be topic in the pycon australia 😄

funky coincidence

the title is "Doing stupid things with ASTs"

lol

[verbal escarp]

actually, now that i'm thinking about it, there may be some cases where GP might be an interesting and useful approach in python

like instances where you have a slow but correct solution to a problem and you're wondering if there are better algorithms that have the same results

then you have a very simple way of testing and comparing

[paper echo]

the search space is still enormous

you can do a lot better at code gen with dependent types, see https://www.youtube.com/watch?v=nbClauMCeds

[nova iris]

match is not a hard keyword

say less

[charred pilot]

is there a downside to having everything be a soft keyword? i guess it would make parsing more annoying?

[nova iris]

https://www.python.org/dev/peps/pep-0622/#use-a-hard-keyword i'm just reading all the scraped ideas, i find this very interesting

YO NO WAY I THOUGHT OF THE SAME THING
i agree though, str(x) or str() as x is better syntax than x: str

BRUH NO WAY-

THESE ARE ALL THINGS I'VE THOUGHT "wait why didn't they add this"

...

at this point python is just reading my mind

how. are all these rejected/changed ideas ones i've once thought about

also, super cool addition, some of the standard libraries will have automatic support with structural pattern matching using __match_args__

[summer hedge]

Not sure if this is the right place, and forgive my ignorance, but most ORMs you have to define the model. Why couldn't the ORM be able to inspect the database at load and define the models from there?

I mean, I know some features of ORMs is model creation and updates through some type of tooling, but assuming an existing database or someone that wants to roll their own creation scripts etc

[spark magnet]

many ORMs have a way to introspect existing databases. "legacy" is what they'll call it.

[summer hedge]

I might be just dreaming here, but a solution where you could plug in an ORM for basic crud stuff, but still basically rely on regular SQL to form your more complex queries sounds good

so far with an ORM, I felt I was bound by the ORM. Then with raw sql, I felt it was too much boilerplate. Haha

ok I see sqlalchemy has some type of automap

[paper echo]

rejecting else was a mistake imo and i am not convinced by the PEP

the syntax could have been

match:
    x + 5
case pattern1:
    ...
case pattern2:
    ...
else:
    ...

instead we have this double indented monstrosity and special-case handling of _ which is elsewhere a normal python identifer and lack of symmetry with if/for/while/try

match x + 5
case pattern1:
    ...
case pattern2:
    ...
else:
    ...

[nova iris]

that is true

the thing is, indentation level of else would become unobvious to a beginner

[paper echo]

i realize that, but maybe that's because the double indentation isn't nice 🙂

[nova iris]

and the wildcard character _ is used, well, as a wildcard in more complicated patterns

so there's a consistency of using _

[paper echo]

but it could have been a naked *

case *: is like "unpack everything and drop it into the void"

[nova iris]

hm. i personally prefer double indentation just because it's easier to see that all the cases are within the match

[paper echo]

yeah i assume that's why they went with it

[nova iris]

but then in that case i'd prefer else over case * lol :P

[paper echo]

also it helps restrict the original case to an expression

shit even case else:

[nova iris]

LMAO YES

consistency of both worlds :D

[paper echo]

match x + 5:
    case pattern1:
        ...
    case pattern2:
        ...
    case else:
        ...

i like that as much as i can like case in python

[nova iris]

why are case and match inverted?

[paper echo]

because i mistyped

[nova iris]

ah lol

yes, case else would really be the merge of two consistencies and two ways of doing it

i like it

[elder blade]

!e ```py
else = 'test'
print(else)

[fallen slate]

@elder blade :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     else = 'test'
003 |     ^
004 | SyntaxError: invalid syntax

[elder blade]

Oh that's nice

[main ginkgo]

wouldve been cool if it was an expression too

[paper echo]

https://github.com/hylang/hy/pull/2084#issuecomment-855305814

(match (Point -1 1)
  (Point 0 0) "The Origin"
  (Point 0 :y y) f"One on the Y axis at {y}"
  [(Point :x 0 y1) (Point 0 :y y2)] f"Two on the Y axis at {y1}, {y2}"
  _ "some place off in the ether")

now we're talking

[paper echo]

this is kind of a fundamental problem in python. there's basically no sensible way to turn statements into expressions. i came up with something like this:

x = list(map(begin def(x, y):
    u = x + 3
    v = y / 10
    return u * v
end, data)

but it's hideously ugly, and the semantics are funky, because python doesn't currently have any semantics for how to handle multi-expression statements. do you allow return inside begin blocks?

result = begin
    result = None
    if x > 0:
        result = fetch_stuff()
        result = process_response(result)
    return result
end

[main ginkgo]

eh? was just thinking along the lines of

x = (
  match thing: 
    case some_case: ...
    case some_other_case: ...
    case _: ...
)

[paper echo]

yeah, then you're overloading the meaning of ()

[main ginkgo]

not really

[paper echo]

is that even possible to parse?

[main ginkgo]

you can already break expression into multiple lines with brackets

[paper echo]

that's a compound statement, not a compound expression

[main ginkgo]

e.g.

my_string = (
  'part 1'
  'part 2'
  'part 3'
)

[paper echo]

that's because () makes newlines not-special and treats them as whitespace

you'd have to change how () works to infer where the statement breaks are, or manually require ; inside ()s

and again you have to assign new evaluation semantics to if, where, etc. - what's the value of a where expression? is it the value of the last expression executed?

i don't know if the infer-where-the-statements-end is possible from a parsing perspective. maybe it is!

in which case yeah () would be great

the current behavior of () supports stuff like:

x = (
         foo
 .thing1
     .thing2()
)

which is the same as foo.thing1.thing2()

what does this do? and is this even unambiguously parseable?

x = (
         x = foo
 .thing1
     .thing2()
  if x:
   x -= 7
)

it'd be nice if it "expanded" to something like

__tmp1_x = foo.thing1.thing2()
if __tmp1_x:
    __tmp1_x -=7
x = __tmp1_x

at which point we're semantically in lisp territory, but again i don't know if the new parser can handle this sort of thing

[main ginkgo]

so then in your view, what is the prescribed 'meaning' of () in python?

and what's a where expression?

like haskell's where?

[paper echo]

sorry i meant while 😆

[paper echo]

in my view it still wraps a single expression, with the difference that \n is just regular whitespace

i'm not saying it can't be different, but if you change it, there will be some hard design decisions in the process

[nova iris]

By default, when in the __main__ module, __builtins__ is the built-in module builtins; when in any other module, __builtins__ is an alias for the dictionary of the builtins module itself.
from py3.9 docs ^

i'm wondering why is the language designed this way? i ran some tests, and the documentation is correct - accessing custom_imported_module.__builtins__ or printing __builtins__ within that module itself does indeed give me a dictionary instead of the builtins module object. also, within built-in modules, the attribute __builtins__ doesn't exist. probably to be more efficient? i'm not sure

the main question is why is there this difference?

[sand python]

from some brief reading, __builtins__ isn't part of the language design. it's just an implementation detail of cpython. Other implementations of python may not have it at all

[nova iris]

oh yes, sorry about that, i meant why does this behaviour exist in cpython

it just seems weird to have different things that __builtins__ point to depending on whether or not the module is __main__

[sand python]

mm, I think I found the reason for it

from https://mail.python.org/pipermail/python-dev/2005-December/058652.html

it's to prevent the entire __builtins__ dict from being printed in interactive consoles when you do vars()

ideally it would always have been a dict

[main ginkgo]

btw have you checked out rust and kotlin's pattern matching?

both of those treat it as an expression, though since they have explicit blocks using {} its a different situation

[red solar]

Python 3.10 also has os.eventfd 🙂 (maybe useful for #async-and-concurrency )

Linux only I think

[elder blade]

What does that do?

[undone hare]

Ooh

eventfd is basically a Linux IPC method to design concurrent softwares

https://man7.org/linux/man-pages/man2/eventfd.2.html

[nova iris]

after hours of work

https://docs.google.com/presentation/d/16c3ErFZ8a5YHwF_ERErGQ23jyWgsj6s8v1Z1HD3iewg @paper echo it's here

this is my best attempt at explaining the class creation process with a flowchart - advanced discussion people seeing this, if you're experienced with this topic, don't hesitate to correct me at parts where i'm wrong! it'd be greatly appreciated, i really want to make this a valuable resource like salt lamp said :D

if there are any cpython developers seeing this and would like to contribute, it'd be amazing lol

[stuck valley]

Thanks! This looks very useful!

[nova iris]

thank you for the feedback! i was going to do that at the start, but then i realized how HORRIBLE google slides is with diamonds

i would try, for sure

oblongs though? what are those?

[nova iris]

thank you for taking a look! i'm glad it could help!

[stuck valley]

Do you have any resources about MRO?

[nova iris]

got it

[nova iris]

mro and c3 linearization? i'll add a slide for all the resources later :D

[nova iris]

for now, check this https://en.wikipedia.org/wiki/C3_linearization

alright, i'll go take a look!

https://github.com/Objectivitix/ccc-flowchart <- btw, got a better link for sharing

[stuck valley]

How does the __prepare__ dunder work?

Also, how does one pass a metaclass to type()?

[nova iris]

1. https://docs.python.org/3.9/reference/datamodel.html#preparing-the-class-namespace
2. https://www.python.org/dev/peps/pep-3115/
   these are sources i used, along with a lot of testing

once again, i'll put all the resources together in the next version (and the graph is meant to be an organizer, after you already read the resources)

i need to go for now, cya!

[stuck valley]

sure!

[sonic dagger]

is anyone numpy smart? if yes #help-apple

[summer hedge]

Been looking at differnet ways people approach things. Saw this in a FASTAPI Project template. Am I right thinking this is just a glorified repository pattern that wraps an ORM?
https://github.com/tiangolo/full-stack-fastapi-postgresql/blob/master/{{cookiecutter.project_slug}}/backend/app/app/crud/crud_user.py

Typical usage looks like

@router.post("/", response_model=schemas.User)
def create_user(
    *,
    db: Session = Depends(deps.get_db),
    user_in: schemas.UserCreate,
    current_user: models.User = Depends(deps.get_current_active_superuser),
) -> Any:
    """
    Create new user.
    """
    user = crud.user.get_by_email(db, email=user_in.email)

Interesting as it uses DI directly in the route to get the database context, and calls directly on the repository from the route

[buoyant island]

not sure if this qualifies as "advanced," but i am using an implementation of the entity-component-system and would like to use dot notation to access entity components (i.e. entity.meta.name instead entity["meta"]["name"]. i have made the following class that i initialize with entity components to do this: ```pythonclass ObjectLite(dict):

def __getattr__(self, key):
    try:
        return self[key]
    except KeyError as k:
        raise AttributeError('attribute ' + str(k) + ' not found')

def __setattr__(self, key, value):
    self[key] = value

def __delattr__(self, key):
    try:
        del self[key]
    except KeyError as k:
        raise AttributeError(k)

def __repr__(self):
    return '<ObjectLite ' + dict.__repr__(self) + '>'```

i essentially want people's opinion on if this approach is "pythonic." i really would like to use dot notation personally but ideally this engine will be used by more people than just me at some point, and if forcing dot notation onto what are essentially large nested dictionaries is bad for performance / readability / maintainability etc., i'd like to know now before i run up against it later in the process

[summer hedge]

I've subclassed Dict before as well, but I remember hearing a lot of caution in doing so as some behavior becomes unexpected due to optimizations of the types in the standard library.

https://treyhunner.com/2019/04/why-you-shouldnt-inherit-from-list-and-dict-in-python/

haha I like how he changed his article name to be less aggressive but didn't change the slug

[stuck valley]

just don't inherit from dict or list

[buoyant island]

like, keep what i sent the same but remove the inheritance? or just don't use this approach at all

[stuck valley]

not sure what the solution is but inheritance from dict is awful

[nova iris]

@stuck valley i've updated https://github.com/Objectivitix/ccc-flowchart to now contain resources and contributing

[stuck valley]

thanks!

[nova iris]

no problem :D

lol, this looks more like a tree than a flowchart

[stuck valley]

it is amazing

python is complex

[nova iris]

Yep, it sure is

Still waiting for the first person to find an error lol

If there are no errors then

My dream would be for a cpython dev to see this and maybe give a review :D

[boreal umbra]

I wish there was a way for non-function, non-class objects to know their name. Things like UserId = NewType('UserId', int) are not elegant.

Perhaps there could be an optional dunder method that gets called after assignment, but I'm not sure what effect that would have on performance.

Would it be any worse than __del__?

[charred wagon]

How would that work considering an object can be assigned multiple names?

[boreal umbra]

you could have an instance variable like self._is_named = False, set it to True the first time it learns its name, and not make any subsequent changes if it's True.

[charred wagon]

That's implicit and unintuitive behaviour.

[boreal umbra]

I'm just thinking of how one could do it without any semantics changes other than calling a newly created dunder after assignment.

[charred wagon]

__set_name__ is kind of like that, but for class attributes

[pliant tusk]

its possible to implement this using some bytecode parsing but its hacky

!e ```py
import sys
import dis

class name_aware:
def init(self):
frame = None
level = 1
while frame is None or 'STORE_' not in (op := dis.opname[
(bc := (code := frame.f_code).co_code)[(idx := frame.f_lasti + 2)]
]):
try:
frame = sys._getframe(level)
except ValueError:
self.inst_name = None
return
level += 1
if op in ['STORE_GLOBAL', 'STORE_NAME']:
self.inst_name = code.co_names[bc[idx + 1]]
elif op == 'STORE_FAST':
self.inst_name = code.co_varnames[bc[idx + 1]]
else:
self.inst_name = None

def __repr__(self):
    if self.__inst_name__:
        return f'{self.__inst_name__} = {super().__repr__()}'
    else:
        return super().__repr__()

class a(name_aware):pass

class b(name_aware):
def init(self, arg):
...
super().init()

A = a()
B = b(1)
print(A, B)```

[fallen slate]

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

A = <__main__.a object at 0x7f2fee399d90> B = <__main__.b object at 0x7f2fee399d30>

[stuck valley]

Hi. How can I use a metaclass with the __prepare__ dunder to hook a classes' attributes dictionary?

[tawny current]

alright

!e ```py
class A(type):
def prepare(self, *args):
print(*args)
return{'lt':print}
class B(object, metaclass=A):pass
B() < "Does this work?"

[fallen slate]

@tawny current :white_check_mark: Your eval job has completed with return code 0.

001 | (<class 'object'>,)
002 | Does this work?

[stuck valley]

thank you

Why didn't it print the left operand?

[crystal nest]

import os
from discord.ext import commands
from discord.ext.commands import Bot
from keep_alive import keep_alive
client = commands.Bot(command_prefix='!')
my_secret = os.environ['token']


@client.event
async def on_ready():
    print('We have logged in as {0.user}'.format(client))

    if os.path.exists(os.path.join("winrate")):
            client.load_extension(f"winrate.cog")

keep_alive()
client.run(my_secret)

Main.py^

from discord.ext import commands

class Ping(commands.Cog, name="Ping"):
    """Receives ping commands"""

    def __init__(self, bot: commands.Bot):
        self.bot = bot

    @commands.command()
    async def ping(self, ctx: commands.Context):
        """Checks for a response from the bot"""
        await ctx.send("Pong!")

def setup(bot: commands.Bot):
    bot.add_cog(Ping(bot))

winrate.py^
Error :- command "ping" not found

Help someone ^

Stuck on this from hours

[tawny current]

#❓｜how-to-get-help @crystal nest

[crystal nest]

[tawny current]

!e ```py
type('A',(type,),{'prepare':lambda s,o:{'lt':print,'args':o}})(type('B',(object,),{}))() < 'Will this work?'

[fallen slate]

@tawny current :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | TypeError: type.__new__() takes exactly 3 arguments (1 given)

[tawny current]

bruh

[pliant tusk]

You are passing the metaclass a class instead of the args to make the class

Remove the call to type for B

[stuck valley]

>>> class fake:
    def __init__(self):
        self._={'a':'b'}
    def __setitem__(self,name,value):
        self._[name]=value
    def __getitem__(self,name):
        return self._[name]
>>> x=type('',(),fake())
Traceback (most recent call last):
  File "<pyshell#14>", line 1, in <module>
    x=type('',(),fake())
TypeError: type.__new__() argument 3 must be dict, not fake

this does not work either

[short heron]

Hi all. I'm trying to figure out how to handle differences in file locations while developing a project vs when it's deployed. I've got a config file in /path/to/project/etc/name/config.json, but when it's deployed, it'll be /etc/name/config.json or /usr/local/etc/name/config.json. Anyone know how to handle this the right way?

[pulsar iris]

can you just use a base or relative path? is it ultimately going to be in the same folder or folder tree as the script itself?

[visual shadow]

Take a look at importlib.resources

From python 3.7 it let's you import resources using top level folder in a project. That should now be the recommended way

[verbal escarp]

on the subject of typing - what's the point of ABCs if you can't even reliably tell if something is iterable :p

that's the only one thing i need from ABCs, ever, and not even that is reliably doable

[prime estuary]

If you're on earlier versions it's available as importlib_ resouces on PyPI.

You can check if something is iterable though?

[verbal escarp]

by trying to iterate over it, yeah, but not just by looking at the ABC

[prime estuary]

No, isinstance(x, Iterable) would work.

[verbal escarp]

class collections.abc.Iterable¶
ABC for classes that provide the iter() method.

Checking isinstance(obj, Iterable) detects classes that are registered as Iterable or that have an iter() method, but it does not detect classes that iterate with the getitem() method. The only reliable way to determine whether an object is iterable is to call iter(obj).

[visual shadow]

Well, that's a good recommendation isn't it

Iter won't actually iterate

[prime estuary]

That's because iter() implements a fallback, doing obj[0], obj[1] etc up until IndexError.

[visual shadow]

But I suppose you specifically want ABC to be able to tell you

[verbal escarp]

well, yeah. what's the point of the ABC if i can do it better in a different way?

[prime estuary]

Those cases should be fairly rare though.

Iterable could check, but it'd have to call __len__ and __getitem__, which is a bit unexpected for an isinstance() call.

Catching exceptions from iter() is more straightforward though.

[lusty scroll]

x = type("fake",(object,), fake.__dict__)

should work if you want to call type like that

[stuck valley]

thanks

however that's not what I'm looking for

[lusty scroll]

but

x = fake()

also worked

[lusty scroll]

you want to explicitly call type.__new__ or you don't

[stuck valley]

No, I think you misunderstand.

I believe I solved it in the end though.

[real turtle]

@unkempt rock żyjesz?????

[nova iris]

I mean, you have __set_name__ in classes

[paper echo]

How does Pip determine if a distribution has been installed?

[nova iris]

Ah lol, too late

[paper echo]

Does it keep a local database? There isn't always a 1:1 mapping between "import packages" and "distribution packages"

[nova iris]

Hey salt lamp :D
Did you see the class creation flowchart yet? I mentioned you yesterday and sent you the final product (I'd like to see what can be improved, if you find some errors/incomplete parts and tell me that'd be amazing!)

https://github.com/Objectivitix/ccc-flowchart here's the link

[paper echo]

This is excellent work! You should maybe post this on https://discuss.python.org and https://reddit.com/r/python as well, I'm sure people would appreciate it. You definitely know more about class creation than I do at this point so I don't think I'm qualified to make corrections.

Minor complaint: the colors are ugly

I would use a white background

[nova iris]

Thank you so much !! Yes, I do agree, it needs a good color scheme and better indicators of the flowing (which is what I'll work on today once I'm on my PC). Perhaps some time I could even migrate it onto a real flowchart site - I prefer google slides however since it's easy to access and I can still make good charts there.

[paper echo]

you could definitely clean it up visually but the real content is the research you put in

[nova iris]

Yep! But visuals are still a big part lol, i need to make it easier to understand and add a legend

[verbal escarp]

i'd still make it an svg in inkscape and from there pdf. if you use standard flowchart symbols, no legend is necessary either

also, you can probably merge all those type-errors into one

takes less space

rearrange the elements a bit

[paper echo]

i don't know if inkscape has good flowchart tools

what you really want in a flowchart tool is the ability to "attach" arrows to shapes

so you can drag the shape around and the arrow follows

i know for a fact you can do this in powerpoint, it's a surprisingly great flowchart/diagramming tool

[verbal escarp]

you can do that with inkscape, yep. i've used it to produce nice flowcharts a while back

[paper echo]

oh, really?

TIL

i'll have to try it

[verbal escarp]

there are some good tutorials for that on youtube, https://www.youtube.com/watch?v=G5LI695MiLY for example

(might not be the best tutorial in hindsight, but still works 😉 )

grml 😐

Deprecated since version 3.9: In the future, the population must be a sequence. Instances of set are no longer supported. The set must first be converted to a list or tuple, preferably in a deterministic order so that the sample is reproducible.

instead of making sets behave like dicts they just make things more annoying

[charred pilot]

random.sample?

[peak spoke]

can they make sets behave like dicts without losing performance on set operations?

[verbal escarp]

yep

[verbal escarp]

i'd bet on it

but i don't know for sure

[stone field]

no

[charred pilot]

could you elaborate? i'm curious now

[gusty marsh]

.bm

[paper echo]

huh... so dicts are allowed but sets aren't? or is it that now only instances of Sequence are allowed?

i would be ok with the latter, the former is just confusing (even if dicts technically are order-preserving now)

[stone field]

Currently sets use different algorithms from dictionaries that gets tiny performance boosts from the fact insertion order does not need to be retained. The simplest example of this is that set intersection can simply loop over the smaller of the two sets. This would obviously no longer be the case if insertion order had to be retained.

The current dictionary implementation of having a sparse index array coincide with a dense array is not suitable for sets. The biggest issue in my opinion is that deletions become an issue (like they did with the ordered dictionary), making it so the sets need to be re-organized often as they get changed. At least in my years of experience with Python, sets are much more prone to more complex mutation than simply insertion compared to dictionaries.

[paper echo]

what if it was just a warning instead of an error?

if not isinstance(data, Sequence):
    warnings.warn('Sampling is not reproducible on data structures that do not have a well-defined sequential ordering. Results might not be consistent even with the same random seed.', category=RandomReproducibilityWarning)
    data = list(data)

[peak spoke]

sets were explicitly allowed by converting to a tuple internally, no idea why that was the case though

[stone field]

Currently that is the behaviour, and there have been talks on whether this should be the permanent solution, but as it currently stands the plan is to raise hard errors

[paper echo]

see above, why not just make them all allowed and let the user decide if reproducibility is relevant?

ty for the explanation @stone field i've wondered about this too

[peak spoke]

Eh, I'd rather have the users take care of that with making it a clear choice on their part

[stone field]

Sampling is fairly trivial to implement yourself, but I also agree that it is inconvenient not to be able to sample sets with the standard-library provided method. I would simply have a warning in the documentation, akin to __del__ instead of even having a warning.

[paper echo]

right, if we have .pop on sets we should be able to have sample

[nova iris]

https://github.com/Objectivitix/ccc-flowchart has been updated to version 1.1 🥳
the color scheme is actually not so ugly now that i added other colors!

oh, and there's also the legend :D

[nova iris]

also posted it on r/python

[unkempt rock]

What's the right way to check if something has the fspath interface? i.e. check if something is a file path and matches the type Union[str, bytes, os.PathLike[Any]]

[crimson niche]

that is a great question to be honest.

[unkempt rock]

I can't just do isinstance(thing, (str, bytes, os.PathLike[Any])) surely isinstance(thing, (str, bytes, PurePath)) would work most of the time I guess

[crimson niche]

I feel like the answer I get in python most of the time is, "You don't need to check things; duck typing..."

But a lot of times I am writing that need to accept different types of paths.

[unkempt rock]

Though aren't Union[str, bytes, os.PathLike[Any]] and os.PathLike[Any] effectively the same since os.fspath works on strings and bytestrings? I'm still confused by this recommendation https://stackoverflow.com/a/61804884

(assuming you only care about things you can call os.fspath on)

[visual shadow]

they aren't the same, i think you ended up being bit by the same issue that you tried to avoid. if you want to see if they quack like a fspath, then one option could have been to just run fspath on it and catch errors if any. if you specifically want it to be like a isinstance thing, then os.PathLike has it's own hierarchy of parents and what not, even if str and PathLike could both quack like a path.

[verbal escarp]

since i know how awesome pathlib is, i refuse the temptation to accept str or bytes as paths

only pathlib.Path for me please

[crimson niche]

I somehow end up with strings and path objects in different places, its horrible. Its possibly just a testament to my awful organization skills.

[peak spoke]

I've had to work with some py2 compat code where Path objects are immediately converted to strings on receival and I didn't think I'd miss pathlib that much

[verbal escarp]

i think they have a lot in common to how bytes and str should be handled

bytes outside, str inside - str/byte outside, path inside

onion style

[cyan elbow]

How do I create this kind of program

[verbal escarp]

what is that o.O

[cyan elbow]

It's a noaa software

[verbal escarp]

oh

[verbal escarp]

are you asking for the process of creating or the structure?

[cyan elbow]

Yes

[verbal escarp]

both? ^^

[crimson niche]

There are so many steps and parts involved in making a GUI application that its a very general question.

[cyan elbow]

Yes

[verbal escarp]

well, for the process i'd say you create vertical and horizontal prototypes first

maybe do some powerpoint mockups

which is much cooler and more useful than it sounds

then you define the platform you want to deploy your software on and pick your gui framework accordingly

then you need to figure out which data you need inside and how it should be stored

prototypes for everything

then put things together

for the structure, there's not much to say ahead of time, since it depends strongly on the data, platform etc.

[cyan elbow]

I already created the GUI I just need help with the graphs and importing the data frome websites

[verbal escarp]

well, it looks like multiple graphs overlayed

[cyan elbow]

I am experimenting with matplotlib to import weather radar data as 3D but there are no pdf or resources

[verbal escarp]

rather try pyqtgraph

matplotlib isn't really ideal for use inside a gui

there are other frameworks that also are better suited for this than matplotlib

[cyan elbow]

There is a program called metpy but its only compatible with Jupyter I have tried with visual studio code but it wouldn't work :(

[unkempt rock]

Say is there a simpler way of optionally sending stuff to stdout or a file that still uses with? ```py
if output_file is None:
print_stuff_to_file(sys.stdout)
else:
with open(os.fspath(output_file), 'w') as file:
print_stuff_to_file(file)

This would nearly work but closes stdout py with open(os.fspath(output_file), 'w') if output_file else sys.stdout as file: print_stuff_to_file(file)

[deft pagoda]

you can always print to stdout, but redirect stdout only if there's another output

or use kw argument of print

print(text, file=output or sys.stdout)

[flat gazelle]

I would do sth like

from contextlib import ExitStack, redirect_stdout
stack = ExitStack()
if output_file is not None:
    a = stack.enter(open(os.fspath(output_file), 'w'))
    stack.enter(redirect_stdout(a))
with stack:
    print('hello, world')
```but it may be a tad overkill

[unkempt rock]

Interesting

[unkempt rock]

Why is 0.1 + 0.2 not equal to 0.3?

[grave jolt]

https://0.30000000000000004.com/

TL;DR: float values are stored as a finite binary fraction (like a decimal fraction, but in base 2 instead of base 10). Just like you can't represent 1/3 perfectly in a finite decimal fraction, you can't represent 0.1, 0.2 or 0.3 perfectly as a binary fraction.

I get autocomplete in the browser

[nova iris]

...

this domain,

is incredible

[unkempt rock]

if something errors between a = stack.enter(open(os.fspath(output_file), 'w')) and with stack: will the output_file still get closed properly?

[main ginkgo]

is a little bit of redundancy really so bad in cases like this? at most you write a few extra lines in exchange for simpler code

honestly the original snippet seems the simplest imo

very straightforward to follow

[boreal umbra]

@main ginkgo I'm not sure what code you're referring to, but readability counts

[main ginkgo]

#internals-and-peps message

[boreal umbra]

@main ginkgo I can't think of a good way to make that more concise

Salt's solution would work except that you have to open the file and then possibly not use it.

[main ginkgo]

yea its fine as it is imo

[unkempt rock]

Does open call os.fspath or do I have to myself? i.e. is open(file) the same as open(os.fspath(file)) ?

!e guess I could test it py class C: def __fsopen__(self): print('CALLED') open(C())

[fallen slate]

@unkempt rock :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 4, in <module>
003 | TypeError: expected str, bytes or os.PathLike object, not C

[unkempt rock]

!e oh, fspath, not fsopen```py
class C:
def fspath(self):
print('CALLED')
return 'foo'
open(C())

[fallen slate]

@unkempt rock :x: Your eval job has completed with return code 1.

001 | CALLED
002 | Traceback (most recent call last):
003 |   File "<string>", line 5, in <module>
004 | FileNotFoundError: [Errno 2] No such file or directory: 'foo'

[unkempt rock]

it does do it

[unkempt rock]

If you really do want to run moduleX directly, but you still want it to be considered part of a package, you can do python -m package.subpackage1.moduleX. The -m tells Python to load it as a module, not as the top-level script.
https://stackoverflow.com/questions/14132789/relative-imports-for-the-billionth-time
really wish this was in the tutorial docs for python

[lofty sierra]

This is definitely not advanced but is it possible to add custom buttons in tkinter?
Like maybe an image of a cup as a button
Or maybe a button shaped circle or something

[fallen slate]

:incoming_envelope: :ok_hand: applied mute to @unkempt rock until <t:1629458265:f> (9 minutes and 59 seconds) (reason: burst rule: sent 9 messages in 10s).

[elder crane]

hey guys i need help at multithreading / multiprocessing

stuck with something that cant figure out i want to start main function with different arguments but i put them at the same time and i want it to start with the top one and than with the others

[verbal escarp]

did you check #async-and-concurrency ?

[cyan elbow]

#armogorkon I got pyqtgraph working do you know how to import weather api

[stone field]

cm = nullcontext(sys.stdout) if output_file is None else open(os.fspath(output_file), 'w')
with cm as f:
    print_stuff_to_file(f)

[spark magnet]

That looks pretty good

[unkempt rock]

Oh neat, nullcontext enters but never exits so stdout won't get closed

[nova iris]

>>> 99999999999999999 is 99999999999999999
True
>>> a = 99999999999999999
>>> b = 99999999999999999
>>> a is b
False
>>> a = 1
>>> b = 1
>>> a is b
True
>>> a = b = 99999999999999999
>>> a is b
True
```i don't need sleep, i need answers

[charred wagon]

Not sure why the first one is true but the rest I understand

[sacred yew]

-5 to 256 is cached by cpython

for the 3rd one

[charred wagon]

Do you know why the 1st is true? Because it contradicts that

[nova iris]

i agree. i've heard something about caching all the way to 256 but the first disproves it

[charred wagon]

Well, on the surface it's a contradiction but it's probably not actually caching that

[nova iris]

yep

[neon tartan]

See #python-discussion message

[pliant tusk]

the first one is true because the line is compiled first, then executed. 999... is cached as a constant value

[neon tartan]

The easiest example of 256 is
#python-discussion message

[sacred yew]

!e __import__("dis").dis("999 is 999")

[fallen slate]

@sacred yew :white_check_mark: Your eval job has completed with return code 0.

001 | <dis>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
002 |   1           0 LOAD_CONST               0 (999)
003 |               2 LOAD_CONST               0 (999)
004 |               4 IS_OP                    0
005 |               6 RETURN_VALUE

[nova iris]

ohh!

[sacred yew]

yeah it refers to the same const

[nova iris]

wow, super cool

i mean, identity shouldn't be relied on anyways, i'm just curious of the implementation details

[neon tartan]

!e

a = 4
print(a+253 is a+253)
a = 3
print(a+253 is a+253)

[fallen slate]

@neon tartan :white_check_mark: Your eval job has completed with return code 0.

001 | False
002 | True

[nova iris]

ah, that's where the cache comes in

[pliant tusk]

its due to how the idle console works (it doesn't cache constants between blocks)

[neon tartan]

I didn't know about -5 to 0 though, is that true?

[nova iris]

!e

a = 4
print(a-9 is a-9)
a = 3
print(a-9 is a-9)

[fallen slate]

@nova iris :white_check_mark: Your eval job has completed with return code 0.

001 | True
002 | False

[pliant tusk]

yes

[nova iris]

yes, it's true

[neon tartan]

👍

Learning something new everyday

[nova iris]

but this might only be for cpython

[pliant tusk]

yea the behavior of is for literals is definitely implementation dependent

[nova iris]

>>> 1,
(1,)
```lol, i never knew you could do this

i mean, it makes sense, tuples are formed by commas

but bruh-

>>> 1,.0_0,0_0_0.0,.1
(1, 0.0, 0.0, 0.1)
```lmao

[pseudo cradle]

This isn't the right channel for this, this is more related to advanced topics regarding the python language itself. If that was a string, however, you could use .split("},{") I believe

[short ermine]

nvm

[crystal kindle]

I made up some new math for describing curves using secondary meshes
https://www.youtube.com/watch?v=XLHOisiIflw

[cyan elbow]

Can I use pyqtgraph for dna analysis

[verbal escarp]

i thought you wanted to plot weather stuff?

but yeah, if you can express something as a plot, pyqtgraph should be able to plot it, why not

[crystal kindle]

it's slow though

compared to using openGL / a uniform array

I would use pyOpenGL and a vertex shader / frag shader

https://pythonprogramming.net/opengl-rotating-cube-example-pyopengl-tutorial/

[mint dock]

pyqtgraph maintainer here; we don't support as many plot types as say matplotlib; but we do cover line plots, scatter plots, images, meshes, bar plots and graphs (like network graphs)... if there is a plot type you think would be good to have in pyqtgraph, but it's not there presently, let us know

[verbal escarp]

hey, good to know and thanks for the reply 🙂 i was just responding to an ongoing project of some kind by @cyan elbow where I recommended pyqt with pyqtgraph over a cobbled-together solution with matplotlib (and i'm guessing tkinter). The visuals in question looked like multiple graphs plotted over each other as overlays

@mint dock #internals-and-peps message

[mint dock]

I love matplotlib, but if you have a lot of data, or updating data, or want to get interactive w/ your data running in a local application, than yeah pyqtgraph should probably be the first stop 👍

@cyan elbow if you haven't used pyqtgraph before, we bundle an "example app" that has the code used to generate respective plots; most of the code is pretty minimal, it's not a great way to highlight how to build out a Qt application, but it does a good job to highlight how to create plot objects, how to update them, how to handle mouse events and so on; after installing pyqtgraph you can run the example app by running python -m pyqtgraph.examples

[dawn harbor]

why does the hubler equation take very long to calculate even on tensor cores

45 seconds a epoch out of 1000 for me

[dusty verge]

What is the hubler equatinon?

Google yielded bad results

[stuck valley]

#❓｜how-to-get-help

[dusty verge]

I was responding to TheGoodyGoat- no results were found on google

[stuck valley]

Apologies.

[cosmic lava]

In Python, if I have a_list = [1, 1, 1, 1, 1], each item is a reference to the same PyObject 1 right? Then, wouldn't this mean that I have one PyObject pointing to 1 with a reference count of 5?

Compared to C, if I made the similar array, I would have five units of 1 right? If say, I created a list and array of millions of 1 in both Python and C respectively, wouldn't Python use less memory than C as the only thing that increased is the reference count?

[boreal umbra]

@cosmic lava keep in mind that the python list of a million 1s is an array of PyObject pointers in C. This is unrelated to the reference count.

There would need to be a pointer at each index in the array, so the space complexity would be the same if it has been an array of ints

[cosmic lava]

So in this specific example, Python uses 2x more memory than C and the fact that there is still only one PyObject with millions of pointers, right?

[boreal umbra]

There's still the one PyObject struct, yes, but then there's still as many pointers as there are references to it in the array

If you wanted to optimize this for memory, you'd need an implementation that knows that all the elements within a certain range are the same.

[cosmic lava]

I see. ok. A PyObject has 3 parts - type, ref_count, value. So every Pyobject is at least 3x the memory of every data in C right? What about pointers?

[boreal umbra]

What are you asking about pointers? Their memory footprint?

[cosmic lava]

How much smaller is a pointer than a PyObject?

[boreal umbra]

I'm not sure. Crowthebird said that they're 8 bytes.

[cosmic lava]

Oh right.

[grave jolt]

depends on what "a data" is

[boreal umbra]

A PyObject struct and the values it has pointers for is going to be larger than a pointer to the struct itself.

[cosmic lava]

Okay, from all that has been discussed, lets say in a growth curve of instantiating 1, 10, 100, ... list/array of data, both Python and C will the same? But due to Python's implementation, it always uses more memory than C?

[grave jolt]

If you want to store lots of integers from 0 to 255, each of them is going to be 1 byte in C. But in Python (CPython) it's going to be 8 bytes

[boreal umbra]

@cosmic lava yes, the space complexity is still going to be O(n)

[cosmic lava]

Is this a true concept, or at least most of the time

ok thanks

[grave jolt]

yes, the time/space complexity will be the same

[cosmic lava]

Thanks

[boreal umbra]

@cosmic lava any other questions?

[cosmic lava]

No. Thanks

[boreal umbra]

Cool. I hope this was interesting for you.

[cosmic lava]

Yes indeed

[flat gazelle]

linear with respect to pattern width and number of patterns

it's more or less equivalent to an if elif chain, at least in CPython IIRC

[grave jolt]

yeah, you can't really make it more efficient

well, you can make a jump table if you only have ints

but that's a sort of a rare case

[flat gazelle]

There are a few other opts possible, but just like most python things, lotta freedom, little guarantees, so opts are hard

Contrasting to haskell, or even simpler rust