internals-and-peps

[crimson niche]

I really dislike the logging module generally. I often have some issue that to this day ive never resolved that makes it ignore my config and rather load logging with default config. To this day I don't know what causes it. Ive posted the code before the causes it and no one has ever replied so simply dont use it anymore..

I know that its my fault and Im using it wrongly in some way, due to multiple files and etc

But its easier to just not use it

[dusty ledge]

Wdym?

[native flame]

if len_ doesn't change in the loop there is no reason to check it inside the loop

[dusty ledge]

@native flame changed it

[prime estuary]

A nicer way would be to slice the list or use itertools.islice() to handle cutting it off after 10 values, or use enumerate and your existing break command.

[dusty ledge]

Why not just do somelist [:10]

[prime estuary]

Indeed that would be a nice way to handle it.

[dusty ledge]

I was actually gonna raise an error

Tim from (twt) said that checking the type of a passed value in a class constructor is a good practice

Since Python is not a typed language

[acoustic crater]

if you're concerned about typing in Python then use type hints and mypy, it's not good practice to just do type checking in some places and not in others

[acoustic crater]

you may have a reason for it sometimes but more generally ("in class constructors") you should be testing behavior of objects not what they inherit from

[dusty ledge]

mypy?

[pine wolf]

I have a list of pre-determined events (say A,B,C,D,E). Throughout the day, I'll have a collection of event a1, a2, b1, c1, b2, c1, c2, c3, d1 in any order
I want to store this chain of events in such a way that I can:

1. Fetch only the 'X' events [A or B or C or D or E] from the chain.
2. Fetch the entire chain of events
   What data structure would give me the most optimal solution in this regard? What is the best way to go about handling this?

[dusty ledge]

Tree I think? (please don't trust me)

[pine wolf]

The tree would simply turn to a linked list because the chain is singly linked. So, the solution might pivot around some sort of efficient/modified linked list I assume

[acoustic crater]

Just use a list and filter by event type? This channel is for the Python language not implementing your own data structures

[acoustic crater]

you probably don't need to worry about mypy right now, I just watched the youtuber in question write a class in which they didn't check the type of passed values so apparently they don't follow their own advice

[dusty ledge]

https://youtu.be/6thjSbJcoUc

[acoustic crater]

ohhhh it's a specific use case with a specific type they are testing for rather than a sequence of that type

that is fine in any circumstance not just in a constructor

[dusty ledge]

? I am confused

[acoustic crater]

they test for Address class

because it's either an Address or a sequence of Addresses

[dusty ledge]

that was my check was doing too..

I was just lazy to specify a name

[halcyon trail]

@pine wolf it kinda depends what other operations you need to support. If you get all the events at once, create the structure, never modify it, and only need to support 1 and 2, then it's really easy, you can just have a list of all the events and also a list for every category

If you need to efficiently add events during the day, that's actually still pretty easy as you just append the same object to two lists
A lot of these "compound" data structures only really get hard when you need to start considering removal

or modifications more complex than just "adding" a new thing

[sonic dagger]

I am currently making a k means cluster code which is only using numpy and matplotlib I want to color code the clusters can someone tell me how to do this?

My code:https://paste.pythondiscord.com/gabaporile.apache

help pls

[molten kayak]

in a program where performance is absolutely vital i pose a question
ive heard if (logic) statements are computationally expensive, with this in mind is it beneficial to performance to replace logic with maths where applicable
eg: if i have an integer thats constantly incremented but cant go over x (lets say x is 10), let me propose the following:

Count = 0
while True:
    Count += 1
    if Count > 10:
        Count = 10
```is my standard way of thinking about it, i guess the `if Count > 10:` could be substituted for `if Count == 11:` if that makes a difference
but look at this example:
```py
Count = 0
while True:
    Count += 1
    Count += Count // 11 * -1

this keeps count at a max of 10 without using an if statement
so to sum up and recap, of these 2 examples, is it beneficial to replace logic with maths where applicable in a purely performance focused program

[peak spoke]

while it may be true in a lower level language, in python it's not exactly true

[radiant fulcrum]

litterally doesnt matter in python

if you absolutely need the performance on the looping use Numba or pypy

[peak spoke]

You may get bit more performance with fewer opcodes but it really depends on what you're doing, doing more numeric operations instead of one if is not going to help in any way though

[grave jolt]

ive heard if (logic) statements are computationally expensive, with this in mind is it beneficial to performance to replace logic with maths where applicable
It's sometimes true in C/C++, but keep in mind that in Python mathematical expressions will do a bunch of if checks or pointer jumps

[radiant fulcrum]

if it's numerical stuff in the loop numba will give you highly optimised C performance anyway

[molten kayak]

okay, lets pretend the python was only for an example and say this is pseodocode for programming in general

[molten kayak]

oh

[radiant fulcrum]

I mean ig? But compilers are so optimised in todays world that they can already do this sorta stuff for you

[grave jolt]

Pseudocode doesn't have an implementation -- that's why it's pseudocode 🙂

[molten kayak]

focussing on the wrong bit 😂

[radiant fulcrum]

it's not really an excuse in the modern world to sacrifice the explicit readability for that little bit of insignificant performance when LLVM and GCC can already optimise to the extreme

sure? Maybe a couple decades ago when compilers wernt as good as optimising as they are now? but

[radiant fulcrum]

Sorta like the whole quake 3 algo for fast inverse root

[peak spoke]

you can't give a straight answer to it if it's for programming in general because it heavily depends on the implementation. There's rarely a need to do this in any language though, and python is definitely not the place for it

[molten kayak]

oo i heard about that, wasnt it able to increase speed with sacrifice to correctness (not the word but you know what i mean)

[radiant fulcrum]

well not really

at the time float operations wernt as optimised as they are now

[molten kayak]

oh, the maths takes longer

interesting

[radiant fulcrum]

So the quake 3 algo basically avoided using floating point arithmetic

but again, in todays world

quake 3 is slower than the general methods now because things have come along way

Highly optimised CPU instructions etc....

[visual shadow]

thing is, you can't generalise this well. for example these days the compilers in low lvl languages are also so good that they may be doing a lot of these optimizations much better than you could, for you. in which case, your attempts to optimize stuff would potentially hinder both performance and readability.

these types of tricks should be used as a last resort, and thoroughly battle tested for that specific scenario both before and after making the changes.

[flat gazelle]

The best general advice for performance today for languages with optimising compilers is do it the way that makes your goal as obvious as possible to the compiler, and wacky bit hacks are not the way to do that.

Also, it's much easier to invoke UB that way

[halcyon trail]

the best advice tbh is to measure

in the words of Andrei Alexandrescu "Measuring gives you an edge over experts that are too smart to measure"

[inland harbor]

that's a nice quote, I hope I remember it

[solar haven]

Hello, I have a quick question:
Given a function:
def f():
for i in (1, 2, 3):
pass
Is the tuple created every time you call the function? In relation, I have tried assigning the tuple in a variable and checked if it's created every time by using its id and found that the id is the same between calls. Is it the same case?

[grave jolt]

Why does it matter to you?

!e
If you really want to know something like this, you can use dis

import dis

def f():
    for i in (1, 2, 3):
        pass

dis.dis(f)

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

001 |   4           0 LOAD_CONST               1 ((1, 2, 3))
002 |               2 GET_ITER
003 |         >>    4 FOR_ITER                 4 (to 10)
004 |               6 STORE_FAST               0 (i)
005 | 
006 |   5           8 JUMP_ABSOLUTE            4
007 |         >>   10 LOAD_CONST               0 (None)
008 |              12 RETURN_VALUE

[solar haven]

!e
check = None
def f():
global check
t = (1, 2, 3)
check = id(t)
print(check)
f()
print(check)
f()
print(check)

[fallen slate]

@solar haven :white_check_mark: Your eval job has completed with return code 0.

001 | None
002 | 139827993051072
003 | 139827993051072

[grave jolt]

as you can see, it's loading a constant, so the tuple is created only once, before the function is ever called

[solar haven]

I'm just curious :)

@grave jolt Ahh, thank you!

[solar haven]

The deeper reason is optimization I think. To my thinking, if it's created every time, then it would slow down the function by creating a new object every time the function is called. Especially if you have a bunch of if i in {"red", "blue", "yellow"} -like checks inside functions

[spark magnet]

be careful about micro-optimizing, it's usually not worth it

[halcyon trail]

especially not in this language

[solar haven]

I try to 😩 Premature optimization is the root of all evil but I can't just help myself sometimes

[spark magnet]

i know, it's very tempting, and the mechanics of the language are fascinating.

[prime estuary]

There's actually an optimisation here in CPython, if you directly iterate over a set or list, it's swapped to a frozenset or tuple so it can be a constant.

[charred pilot]

as long as the behavior is the same, it'll do that for more than just iteration

[prime estuary]

!e

import dis
def func(x):
    print(x in {1, 2, 3})
dis.dis(func)

[fallen slate]

@prime estuary :white_check_mark: Your eval job has completed with return code 0.

001 |   3           0 LOAD_GLOBAL              0 (print)
002 |               2 LOAD_FAST                0 (x)
003 |               4 LOAD_CONST               1 (frozenset({1, 2, 3}))
004 |               6 CONTAINS_OP              0
005 |               8 CALL_FUNCTION            1
006 |              10 POP_TOP
007 |              12 LOAD_CONST               0 (None)
008 |              14 RETURN_VALUE

[prime estuary]

I believe this is the only way (without direct editing) to have a frozenset in a code constant.

[solar haven]

I've read about this but I haven't actually seen it in action. And it does do that! Thanks! I really should read about dis :)

[grave jolt]

!e
Do you know why in ```py
import dis
def f():
for x in {1, 2, 3}:
pass
dis.dis(f)

if uses a frozenset and not a tuple?

[fallen slate]

@grave jolt :white_check_mark: Your eval job has completed with return code 0.

001 |   3           0 LOAD_CONST               1 (frozenset({1, 2, 3}))
002 |               2 GET_ITER
003 |         >>    4 FOR_ITER                 4 (to 10)
004 |               6 STORE_FAST               0 (x)
005 | 
006 |   4           8 JUMP_ABSOLUTE            4
007 |         >>   10 LOAD_CONST               0 (None)
008 |              12 RETURN_VALUE

[grave jolt]

I mean, it's pretty strange to iterate over a set like that, but still

[prime estuary]

So it keeps the same behaviour, hashing the items and eliminating duplicates.

[grave jolt]

I don't get what's wrong with transforming this to for x in (1, 2, 3) -- if the items aren't hashable, Python will not do this thing at all, and duplicates can be detected as well.

[prime estuary]

It could yeah, probably just to avoid that work.

[halcyon trail]

The thing is that worrying about micro optimizations in python is like clipping coupons

While you use dollar bills for your bonfires

[prime estuary]

Here's the code, doesn't seem to have a specific explanation.
https://github.com/python/cpython/blob/main/Python/ast_opt.c#L577-L582

[fallen slate]

Python/ast_opt.c lines 577 to 582

/* Change literal list or set of constants into constant
   tuple or frozenset respectively.  Change literal list of
   non-constants into tuple.
   Used for right operand of "in" and "not in" tests and for iterable
   in "for" loop and comprehensions.
*/```

[raven ridge]

the iteration order would be different.

not that the iteration order is something that people should be depending upon, but switching it to frozenset probably involves the least risk

and the least work at compile time

[grave jolt]

but the iteration order for a set isn't defined

[raven ridge]

but it's consistent

[gleaming rover]

only for individual runs, right

[raven ridge]

yep.

[gleaming rover]

also IIRC if you want one frozenset you need to define it outside the function

[raven ridge]

!e Alright, better reason:

import dis
def f():
    42 in {1,2,3}
dis.dis(f)

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

001 |   3           0 LOAD_CONST               1 (42)
002 |               2 LOAD_CONST               2 (frozenset({1, 2, 3}))
003 |               4 CONTAINS_OP              0
004 |               6 POP_TOP
005 |               8 LOAD_CONST               0 (None)
006 |              10 RETURN_VALUE

[raven ridge]

if it turned that into a tuple, the __contains__ check would be slower than it needs to be.

and if it sometimes turned set literals into frozensets and sometimes into tuples depending on how they're used, it would take more smarts (and time) from the compiler.

[grave jolt]

@raven ridge Of course, here a frozenset is better. I wasn't talking about a __contains__ check, only about iteraiton, because iterating over a tuple is (marginally) faster than over a set

I guess the iteration order is why it's like that

so that py xs = {1, 2, 3} for x in xs: ... is the same as ```py
for x in {1, 2, 3}:
...

[raven ridge]

right.

[grave jolt]

also... why would anyone iterate over a set literal?

[raven ridge]

you'd usually use a tuple for that, yeah.

[halcyon trail]

When I'm checking membership over a literal I always use a set tbh

I realize it's slightly slower but, idk, my reasoning is if all I'm doing is checking membership, sets are what's idiomatic for that

[gleaming rover]

same

I think it's a very good habit to have

[halcyon trail]

List is fine too but I don't like using a tuple for that. To me, that's conceptually weird.

[gleaming rover]

and it makes your intention clear

[halcyon trail]

Right

Tuple to me is weird because tuples have heterogeneous types, but you're doing an operation that makes more sense over homogeneous types, so it's just rather odd

[gleaming rover]

yeah, I feel the same way

[grave jolt]

If you're doing ```py
if foo in {'a', 'b', 'c'}:
...

[solar haven]

I got more curious and did some more checking and it seems like container objects in functions other than tuples and strings are created everytime a function is called. Strings and tuples are, I think, cached. I guess it recreates these container objects so that when the function ultimately returns it, there are no gotchas like what happens when you set a default mutable object in function definitions.
!e
def test_1():
print(id(["hello", "world"]))
print(id({"hello", "world"}))

for i in range(5):
test_1()
print()

def test_2():
a = ("dog", "cat")
print(id(a))
print(id(("hello", "world")))
print(id("hello, world"))

for i in range(5):
test_2()

[grave jolt]

yeah, you can't cache mutable things

[solar haven]

That's weird (or expected?). In a for loop, the id is the same for test_1 in subsequent calls. Individual calls will show a different id.

*almost the same id between subsequent calls. What? I guess I will have to dig deep to the source in order to understand what's really happening lol

[static bluff]

Slowly I think I'm starting to get this encapsulation idea

Like, why its important

[grave jolt]

That probably has something to do with the way Python allocates objects. It probably reuses the object once its lifetime is over not to allocate new ones

on other versions it might be different

[raven ridge]

an object's ID is only meaningful for as long as the object is alive, after all.

[grave jolt]

yeah

there's no rule that says that IDs can't be reused

[raven ridge]

once that object has been destroyed its memory can be reused - possibly for another instance of the same type.

[solar haven]

I'm using iPython to kinda do these tests and maybe that's a factor too because id([]) in iPython is different on subsequent calls while it's the same on the Python repl

[prime estuary]

What happens is because you're destroying the object, then immediately creating it again, when it goes to allocate memory it sees that convenient block which is the exact right size.

If you did anything in between, it might use the memory for something else and then not work.

iPython has all of its code running, so that's probably using that memory block.

[signal tide]

class A:
  x = 5

A().__dict__
>>> {}

class A:
  def __init__(self):
    self.x = 5

A().__dict__
>>> {'x': 2}```
Why does `__dict__` only work when the vars are written to + is there a way to get around writing an init?

[native flame]

class A:
  x = 5

x is stored on A here, not on individual instances

[coarse urchin]

In python are string literals allocated only once for strings? So for instance

a = "abc"
b = "abc"

creates only one string, right?

[unkempt rock]

In Cpython, some strings are interned (such as that one) but its an implementation detail

[coarse urchin]

Some?

I mean, the ones that are known at compile time are?

Oh wait what compile time

Wait, is there some preprocessing before interpreting?

[native flame]

python gets compiled to bytecode before its interpreted

[coarse urchin]

Okay

In that stage, are all constant strings (present as string literals) interned?

[signal tide]

Shouldn't this work then?

class A:
  x = 5

A.__dict__```

[native flame]

!e

class A:
  x = 5

print(A.__dict__)

[fallen slate]

@native flame :white_check_mark: Your eval job has completed with return code 0.

{'__module__': '__main__', 'x': 5, '__dict__': <attribute '__dict__' of 'A' objects>, '__weakref__': <attribute '__weakref__' of 'A' objects>, '__doc__': None}

[signal tide]

Is there a way to clean that without manually deleting the dunders?

a_dict = {k, v for k, v in A.__dict__.items() if not k.startswith("__")}```

[sand python]

if you want a shorter notation look at dataclasses, they work well for many simpler cases: ```py
from dataclasses import dataclass

@dataclass()
class A:
x: int = 5

[signal tide]

that's what im using but I want default values while still having access to the __dict__
ok so I didn't realize typehints were so essential?

!e

from dataclasses import dataclass

@dataclass
class A:
  x = 5

@dataclass
class B:
  x: int = 5

print(A().__dict__)
print(B().__dict__)```

[fallen slate]

@signal tide :white_check_mark: Your eval job has completed with return code 0.

001 | {}
002 | {'x': 5}

[sand python]

oh you found that too lol

i'm actually looking into that right now

[signal tide]

it doesn't really matter, I've got types in my project I was just playing around in the interpreter but that's really odd behaviour

[sand python]

ah I see

yeah only fields with type annotations are considered fields

names without type annotations will belong to the class as normal I guess?

[signal tide]

ig that's somewhat intuitive?

[sand python]

!e yep that is indeed the case, good to know ```py
from dataclasses import dataclass

@dataclass()
class A:
x = 5
y: int = 5

a = A()
print(a.x, a.y)
A.x = 99
A.y = 99
print(a.x, a.y)

[fallen slate]

@sand python :white_check_mark: Your eval job has completed with return code 0.

001 | 5 5
002 | 99 5

[sand python]

it is documented but it is quite easy to gloss over

[static bluff]

Regarding 'encapsulation', something I'm really trying to get the hang of

Reducing the dependency of one 'hub' of code within my project on the other hubs

And reducing dependency in general

Would thinking of each main branch of my project as being its own standalone library, where possible, be a good way approach?

[halcyon trail]

not really tbh unless each branch of your library does totally different things and you really want to write each branch to have an API that makes sense generally and not just for your needs

[static bluff]

Hmmmmm

Thank ya!

[halcyon trail]

To get the hang of encapsulation I'd focus on writing good classes above all else

[hollow galleon]

Hey anyone who knows python threading can you please look at my question in #help-bagel

[polar gull]

When is the new season 4 of PythonPvp coming out?

[teal wing]

whats encapsualtion

[raven ridge]

encapsulation is the word for having a public interface for your modules or classes, separate from their private state. By having variables that are only updated in a known and controlled way, you can better manage complexity and make the system easier to reason about.

That said, that's well off-topic for this channel, and really belongs in #software-architecture

[iron perch]

what is making

            row = [str(item) if item is not None else 'X' for item in row]
            row.append(f"{index}\n")
            str_ += ''.join(row)```
better than
```py
row_str = ""
for item in row:
   row_str += str(item) if item is not None else 'X'
...```

is this because number 1 is looping twice because of ''.join?

[grave jolt]

First one is O(N), second one is O(N^2). (where N = len(row)).

[iron perch]

how was it n^2

[grave jolt]

Suppose that you have two strings, of size N and M.

n = "..."
m = "..."
``` then you create a new string, `k`, by concatenating them: ```py
k = n + m
``` To do that, you need to spend `N + M` "time units", i.e. the time will be proportional to `N + M`. Do you agree?

[iron perch]

on yeh

oh yeh

how does .join do stuff?

[grave jolt]

Now assume that every item in row has a fixed length L. The size of row_str grows like this:

0
L
2L 
3L
4L 
...
NL
```, so the amount of time units you're going to spend is `L + 2L + 3L + ... + NL = (1 + 2 + 3 + ... + N)L`, which is on the order of `O(N^2 * L)`

str.join is implemented natively (in C if you're on CPython). First it computes how long the string should be, then it creates a single mutable array and writes the contents into it.

[iron perch]

[grave jolt]

Something like:

row_str = []
for item in row:
    row_str.extend(str(item) if item is not None else 'X')
result = magically_turn_list_into_string(row_str)

but you can't really implement it in Python

[iron perch]

ok thanks!! I'll be back with more questions later!

[undone hare]

buf = ""
for e in my_list:
    buf += e```

[grave jolt]

That's still O(N^2) conceptually

[undone hare]

You didn't precise without any enormous flaws

[grave jolt]

By 'it' I meant 'algorithm used by str.join in CPython'

[undone hare]

Ah fair

I wonder how it eyeballs the buffer length actually

[grave jolt]

It doesn't eyeball it, it just computes it

[undone hare]

Because if it is a list of string that's pretty easy, but what if it is a generator?

[grave jolt]

Then it turns it into an array in memory first.

[undone hare]

Hmm

Yeah, fair

[grave jolt]

that's why "".join([x for y in z]) is faster than "".join(x for y in z)

[undone hare]

I guess it is twice the memory used by the final string, but it is way more efficient

That's interesting, I didn't know it was faster

[grave jolt]

if you're constructing a giant (megabytes) string in memory, there might be a better way, like lazily sending it over a socket or writing it to a file

[undone hare]

Yeah, if you have a string that large you are probably doing something wrong

[stuck valley]

uh oh

[iron perch]

@grave jolt ```py
str_ = ''

    for index, row in enumerate(self.board, 1):
        row = [str(item) if item is not None else 'X' for item in row]
        row.append(f"{index}\n")
        str_ += ''.join(row)
    
    return f"{self.IDENTIFIER[:index]}\n{str_}"

str_ +=

[grave jolt]

If you want it to be "fully optimized", write it in assembly 🙂

You could first collect all fragments into a list and then "".join them.

[native flame]

if you have no other references to the previous string the compiler probably optimises it anyway

[grave jolt]

CPython somewhat optimizes it at runtime, but you shouldn't really rely on it

[charred pilot]

if there are no speed issues then there's no need to optimize

[grave jolt]

also that

@iron perch how large is your string?

[iron perch]

not that much

but I am practicing to mostly write efficient code

[grave jolt]

Microoptimization is not very useful. If you want that function to run faster, use PyPy.

If you want to replace += with join, as I said, you can first collect a list of parts and then "".join it.

        parts = []
        
        for index, row in enumerate(self.board, 1):
            row = [str(item) if item is not None else 'X' for item in row]
            row.append(f"{index}\n")
            parts.extend(row)

        str_ = "".join(parts)
        
        return f"{self.IDENTIFIER[:index]}\n{str_}"

but you'll need to measure it to see if it's really faster, and how much faster

[iron perch]

https://paste.pythondiscord.com/ajirazucoq.rb is the design of this good?
I made it so objects in the program is like in the real world so I think the code is abstract

I will refactor some codes there

I think the implementation of the design is bad (I am currently refactoring stuff but I might not see the other ones that needs refactoring so I need pro's eyes)

[magic python]

wondering about the use of classes to wrap functions so that all functions which do a particular thing are accessed from the same place?

for example if the following was in some mod.py

class Funcs:
    @staticmethod
    def f(x):
        return x + 1
    @staticmethod
    def g(x):
        return x - 1
    @staticmethod
    def h(x):
        return x / 2

Then it might be used as:

import mod
mod.Funcs.f(1)
mod.Funcs.g(2)

and so on.
It feels as though it's probably wrong, but i'm not too sure. Basically it's the opposite of a dataclass i guess lol

perhaps it'd be better to just put these into a separate module and import that instead 🤔 Is there a functional difference between the two approaches tho?

[native flame]

why in a class? why not just mod.f(1); mod.g(2)?

[magic python]

@native flame yeah sure, i get that'd work, but this would work too... and i don't really understand whether there's a functional difference other than weird style

[native flame]

modules are the way to keep similar functionality together

[magic python]

right, so instead of having this Funcs class, one would have some _mod.py or whatever, and then in __init__.py have something along the lines of

from ._mod import f, g, h

that's more typical?

[spice pecan]

pretty much, yeah

it accomplishes the exact same goal without introducing a new class

unless you intend to do some metaclass voodoo or something similar, you'd just shove the functions into a module

[magic python]

no metaclass voodoo no lol, unless by accident 😅

are there any rules around where __all__ can go within __init__.py ?

I looked here: https://docs.python.org/3/tutorial/modules.html#importing-from-a-package but didn't see anything relating to where it can / should do within init

[visual shadow]

you dont plan to star import stuff yes?

iirc you only need to define __all__ if you plan to star import, which shouldn't be needed in the first place for most codebases*

[magic python]

its needed for some linting as well :S

think it gets removed as unused imports, can't remember by what, autoflake i think

[cobalt agate]

KivyMD vs React Native. Why do pro devs frown upon the former? Is it just the huge bulky app size? Anyone built professional android apps with KivyMD?

[radiant fulcrum]

Honestly if you're looking to seriously make an app use react Native over kivy

Python is just not the best language in the world to try ship apps with

Leave that job to kotlin n java for android apps

[swift imp]

just make a module

static methods are next to useless and the way you wrote that class essentially makes a nested namespace nothing me

it has zero state

[magic python]

yeah that's why it felt kinda daft - it worked, but felt silly. I was wondering if there was anything that didn't work as expected though i guess

[cobalt agate]

I was hoping to get away with not learning new languages and syntaxes.

has anyone incorporated canva/figma/dribble designs into KivyMD? Is it even possible?

[spiral willow]

That's not going to work if you want to learn Android development

[idle fossil]

what they mean by pseudo compiled code could smone explain it plz?

[spark magnet]

That should say "compiled to bytecode" . "pseudo-compiled" is a silly term.

[sand goblet]

!e ```py
def f():
x = y
print(y)

y = "123"
exec(f.code)```

[fallen slate]

@sand goblet :white_check_mark: Your eval job has completed with return code 0.

123

[sand goblet]

What does it mean “they don’t contain a reference to their global execution environment”?

[gleaming rover]

I think it means that functions are closures?

like they maintain references to variables outside function scope

[sand goblet]

And code objects don’t, right?
But that one seems like it accessed y fine

[gleaming rover]

that's true

[sand goblet]

Or maybe I still don’t get it

[gleaming rover]

I'm stumped

🥴

I mean

functions have __closure__

hm

https://stackoverflow.com/questions/14413946/what-exactly-is-contained-within-a-obj-closure

think this is relevant

[sand goblet]

!e ```py
def f(y):
def g():
x = y
print(x)
exec(g.code)

z = "123"
f(z)```

[fallen slate]

@sand goblet :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 8, in <module>
003 |   File "<string>", line 5, in f
004 | TypeError: code object passed to exec() may not contain free variables

[sand goblet]

It’s talking about this?

[gleaming rover]

ye

is what I would think?

[sand goblet]

I wonder why it doesn’t say “nonlocal execution environment” instead of “global execution environment”

[spark magnet]

code objects are immutable, and are just the result of compiling the code. functions are mutable, and have a reference to a code object. functions hold the default parameter values, the global references, etc.

[sand goblet]

!e ```py
def f(x=[2,4,6,8,0]):
print(x)

exec(f.code)```

[fallen slate]

@sand goblet :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 4, in <module>
003 | TypeError: f() missing 1 required positional argument: 'x'

[sand goblet]

Ok

[sand goblet]

@spark magnet why did it work with the global variable here?

[spark magnet]

isn't it because exec used the current globals as the globals?

[sand goblet]

It’s something like, exec has the globals dict as a default argument value, and it uses that when executing the code from the code object?

[spark magnet]

yes

[sand goblet]

Ok thanks

[magic python]

(removed)

[raven ridge]

that question would fit much better in #software-architecture than here.

[magic python]

ok cool thanks 🙂

[faint quarry]

what is this for?

channel

[elder blade]

Read at the top

[narrow flare]

Hi guys, I have a question in regards to environments (pip/conda/poetry that kinda stuff) and don't know where to go as I dont think any of the channels cover this. Does anyone have suggestions ?

[livid dove]

I’d go to #tools-and-devops

[sonic dagger]

my code:
https://pastebin.com/B84qGuME

How can I amke my code wrk with more than one centriod it is for k means cluster

[elder blade]

You may have better luck opening a help channel, read #❓｜how-to-get-help

[formal compass]

Not a help thing. Mostly just curious

Is there a way to get the position of n that returned true in this type of if check?

If x in n:

Where x is something that’s in the list n and n is a list

Actually. I guess you could just get the position of n[x] at that point.

Nvm

Ok so my question now would be. Does that if statement return true on the first instance of x in n or does the list not go in order?

[valid rose]

typically, classes provide a index or find method to help find the position

[formal compass]

My question is: If x in n… this is looping through n to find x is it not? If so. Does it stop at the first instance of x ?

[native flame]

for lists? yes

for sets/dicts? no

[formal compass]

Yea specifically lists

Curious how sets and dicts differ but I can look into that myself

[formal compass]

And just to make it easy. Assume x is an int and n is a list of ints

Is there a function to “get position” of x when it finds itself in n in that case?

While in an of statement?

If*

[spark magnet]

why wouldn't it stop at the first for a set or dict?

[formal compass]

That’s what I’m curious about

[elder blade]

n.index()

[spark magnet]

(full disclosure: it will stop when it finds it, but maybe @full nebula meant something else by "first"?)

[formal compass]

Forget I said this. I know how to get the index inside of the if check. Again I’m half asleep

I’m on my phone and half asleep or I would just test this lol

[dusty verge]

you aren't telling it what to find the index of

[elder blade]

Dicts and sets are hashmaps, that means that

• There cannot be duplicates
• Looking something up in a hashmap does not require iterating through it
• They don't have indices

[formal compass]

That makes sense

[spark magnet]

yes, ok. so "first" is misleading there

[dusty verge]

hah. Just found this

import ctypes
a = "hello world"
print ctypes.cast(id(a), ctypes.py_object).value

clearly its from python2, but I would assume it still works unless ctypes has been changed

[spark magnet]

works to do what?

[elder blade]

Yeah that's why hsp said it's not the case for those.

[dusty verge]

Return a python object from an id

[spark magnet]

why do you need that?

[dusty verge]

I don't- was just curious if you could

[spark magnet]

and it might not work: it could get you the new object that has the same id

[formal compass]

But just to clarify. If x in n returns true. X will stop iteration when it’s found and not cycle the whole length of n right ?

[dusty verge]

I thought id was the memory address

Oh, yeah

if its been GCd

[spark magnet]

right

[formal compass]

👍

[dusty verge]

I mean I would think thats unlikely, thats only if you are being really dumb with your use case

[elder blade]

Yes, it will stop at the first time it sees x because it doesn't matter if there's multiple

[spark magnet]

we don't have a use case. this isn't something people need.

[formal compass]

Makes sense. Just wanted to verify what I figured would be true

[native flame]

oh, that was in response to

this is looping through n to find x is it not?

[spark magnet]

got it

[spark magnet]

however you were keeping the id around, you can just keep the object.

[dusty verge]

Just to be clear, like I said before, I'm not actually using this

The reason I thought of it was because I wanted to store py objects in a set by id, and I was wondering if it were possible to access them without a dict

(Well thats not even true, I'm not even writing any code, I was just thinking how you would store mutable objects in a set 😅)

[prime estuary]

It technically works, but is quite dangerous since you're casting a raw pointer, you have no guarantees that's still a valid Python object, and it's also something that only happens to work in CPython.

[spark magnet]

btw, a way to store mutable objects in a set is to give them __hash__ and __eq__ methods, possibly by wrapping built-in mutables (like list) in your own proxy objects.

[visual shadow]

This seems like a strange premise. You could avoid all funny behaviours by just storing in a dict, Id as the key and value as the objects themselves

Now, I will caveat by saying that its extremely rare that you'd ever need to rely on id or use them in this manner.

[finite sparrow]

!e i implemented a probably very bad hashable list for fun. DISCLAIMER: use at your own risk```py
class hashablelist(list):
def hash(self):
return hash(tuple(map(hash, self)))
def eq(self, other):
return hash(self) == hash(other)

a = hashablelist()
b = hashablelist()
a.append("hey whats up")
a.append("idk if this is a good idea")
b.append("hey whats up")
b.append("idk if this is a good idea")
print(set([a, b]))

[fallen slate]

@finite sparrow :white_check_mark: Your eval job has completed with return code 0.

{['hey whats up', 'idk if this is a good idea']}

[finite sparrow]

caveats: the list is mutable but the hash is computed based on a tuple. generally, putting mutable objects in a set is a bad idea

[frail wasp]

Hi,
I have an issue with the following code

import os
print("Can I write into the tv folder?",os.access("/tv", os.W_OK))
print("Does the /tv/test file exists?", os.access("/tv/test", os.R_OK))
with open("/tv/test", "w") as f:
    f.write("toto")
print("Does the /tv/test file exists now?", os.access("/tv/test", os.R_OK))
with open("/tv/test", "r") as f:
    print("Content of the /tv/test file:")
    print(f.read())

prints out

Can I write into the tv folder? False
Does the /tv/test file exists? True
Does the /tv/test file exists now? True
Content of the /tv/test file;
toto

According to the first call to os.access, I should not have been able to write the test file...

I'm running Python 3.8.10 in a docker container, and the tv folder is a remote nfs mount done by docker

I know that os.access can give false positive, but false negative ?
Does anyone have already seen this ?

[terse orchid]

If I want from an associate to write a simple module, and considering writing a SDD is overkill for this task, how can I explain it to him briefly? Like, is there any standards for this? I want to explain him the expected inputs/outputs of the function

[dusty verge]

Yeh thats what I said earlier- I was just wondering if it were possible to without it.

Also- for the hash thing, why not just

def __hash__(self):
  return id(self) / 8

wait actually

(nvm, its 8. my bad)

[peak spoke]

that won't satisfy the condition that equal objects have equal hashes (at least how you want it to)

[visual shadow]

The general gist Is that usually people want the dict to match if a different object with same value is present.

[dusty verge]

Right, yeah thats fair

I mean I guess that means that you'll never really be able to hash mutable objects

which is the case already 🤔

[elder blade]

Why not hash the repr?

[peak spoke]

although I'm not sure what the benefit of hashing of all elements is instead of converting to a tuple and hashing that

[dusty verge]

would the repr be unique?

I guess it would be

[elder blade]

It would be the same for all objects that have the same values

And unique if the objects' values are unique

*by values I mean whatever is displayed in the repr

[dusty verge]

wait, can tuples have self references?

Because I think in that case there could be two interpretations of a repr

[peak spoke]

only with some ctypes hacks afaik

[dusty verge]

haha, back to ctypes hacks.

Ok hear me out: tuple with self reference will have the same repr as a tuple containing an ellipsis

now I wonder how to actually nest the tuple 🤔

[peak spoke]

ellipsis uses Ellipsis for the string representations

[dusty verge]

Oh! nevermind

[raven ridge]

If you want to store mutable objects that define __eq__ in a set, you could just define __hash__ to return a constant. That would be enough to satisfy the one actual constraint on __hash__ - that equal objects have the same hash code.

That would be a bad, inefficient hash, of course, but it would let you store them in a set without breaking anything

[halcyon trail]

what an evil genius idea

[paper echo]

def __hash__(self):
    return hash(id(self))

this probably has legitimate uses

or something like an ORM where you don't care about the "structure" of an object, just its "name"

def __hash__(self):
    return hash(self._id)

[deft pagoda]

i used something similar when grouping canvas instructions in kivy -- i would just use the id of one of the instructions and assign the rest to a group with that name

[paper echo]

yeah, as long as it's semantically reasonable to treat equality as object equality (as opposed to structural equality), i don't have any moral objection to hashable mutable things

[raven ridge]

In that case you get it for free, though. Just don't define a __eq__ or a __hash__ for your custom class and it is hashable and mutable, and an == check behaves like is

[paper echo]

true, i was thinking about sets and dict keys

[quasi hound]

def _(*a, **kw):
  pass

f = lambda: (
    _(a := 5),
    _(b := 6),
    print(a + b)
  )

f()
>>> 11
``` this is great

and it works

[paper echo]

that is some cursed goodness

is the evaluation order in tuple literals guaranteed to be left-to-right?

[quasi hound]

probably

man why can't they just add fat arrow functions to python

(i know why they said they don't want to)

but imagine

[paper echo]

why did they say they didn't want to? i remember a typing-sig mailing list thread where guido suggested that () -> syntax could be a candidate for lambda

[quasi hound]

something about {} being used already

[paper echo]

oh, yeah {} wouldn't make sense

[quasi hound]

and python relies on whitespace for function defs

and also

this example would be ambiguous

x = map(
  multilambda x:
     if x % 2 == 0:
        return x
    else:
        return x+1
  , [1, 2, 3, 4, 5])

[undone hare]

I believe so

[paper echo]

the big problem is the whitespace, as you said. i just had a thought, that they could add support for something like begin/end in order to turn a statement into an expression

map(begin def _(x):
    return x + 5
end, [1,2,3])

[quasi hound]

does it return (x+1, [1, 2, 3, 4, 5]) in the else condition or is that the second arg of map()

that's what the interpreter wouldn't know

but yeah that makes sense

[paper echo]

the rule would be to use the indentation of the line containing the begin as the base point

so it has to be indented at least 1 space beyond wherever the begin is indented

map(
    begin def (x):
        return x + 5
    end,
    [1,2,3],
)

maybe that ☝️ would be equivalent to lambda?

so begin def (x): ... end would be like lambda: ...
and begin def foo(x): ... end would be like foo := lambda: ...

[quasi hound]

they could also do

x = () -> print(5)
x = y -> print(y)
x = (y, z) -> print(y+z)
returnSum = (a, b) -> a + b
returnSumMultiline = (a, b) -> (
    ...
    ...
    return a+b
  )
x = (a, *b) -> (
    print(a)
    print(len(b))
  )

i think that covers all the examples

and yeah () makes a bit less sense than {} for lambdas

[paper echo]

() makes sense in python i think

i imagine that's more like what guido had in mind anyway

[quasi hound]

but this works

(lambda a, b: print(a+b))(5, 3)
>>> 8

[paper echo]

you might have a similar restriction to list comprehensions, where un-parenthesized tuples are not allowed in () -> () syntax

[quasi hound]

yeah makes sense

[paper echo]

or maybe you don't need it? since you can strip off the outer layer of ()s

i like my begin/end proposal because it also lets you wrap up loops and if as expressions

and i think the -> is a bit unpythonic

:= makes sense because we already have =

[quasi hound]

i personally don't like how begin/end looks

it gets really cluttered in languages like lua (i hope im not wrong)

python's whitespace/indents look great

[paper echo]

i haven't seen lua or ruby code where it looks really cluttered but i haven't seen a whole lot of either compared to python

it's also used in julia

[quasi hound]

oh yeah

but here's what i was gonna say

-> is already used for typehints, and the devs seem to be fine with reusing keywords and stuff (as and from are already used in several different places)

[paper echo]

true, but then you can't type annotate expression-def's

side note: it'd be cool if you could define custom infix operators, e.g. in R you can define a function called %hello% and it will become an infix operator

`%hello%` <- function(x, y) {
  cat("hello!\n")
  x + y
}

print(3 %hello% 5)

i really wish they didn't take @ for matmul, because you could have easily used it as a "guard" for such operators, e.g.:

def @*(x, y):
    return np.dot(x, y)

as much as i selfishly like having @ for the domain i work/worked in, i think it was a really bad idea to add it to python and call it "matmul"

[quasi hound]

oh bruh that's really cool

yeah it seems stupid

[paper echo]

at least call it "the user-defined binary operator"

[quasi hound]

same with ... idk why that got added

[paper echo]

... has specific uses

[quasi hound]

it has its own class and stuff and apparently used in arrays yeah

i just use it as a replacement for pass sometimes

and that's it

[paper echo]

semantically it's different from pass

pass is a no-op, ... indicates that you aren't meant to actually use the function/class in question

... is for protocols, abstractmethods, type stubs, etc.

[quasi hound]

oh ok

[paper echo]

that Ellipsis is a python language construct and not special syntax is probably a good thing, because you can write new APIs that use it if you want to (i think pydantic does this or did it in previous versions)

[paper echo]

where's that from?

[quasi hound]

https://stackoverflow.com/questions/772124/what-does-the-ellipsis-object-do

[paper echo]

i didn't realize numpy even supported ..., and i didn't realize that it was originally added to support numpy

[quasi hound]

neither did i lol

also weird that they gave a variable a name that wouldn't normally be supported

[paper echo]

wow that is really useful for high-dimensional arrays

[quasi hound]

like .... isn't a valid name

the only other ones like it are True False None

[paper echo]

that's probably why they did it, to avoid adding a new keyword + make it pretty

True, False, and None are syntactically valid names 🤔

[native flame]

... is also used in some typehints

[quasi hound]

nah nah i mean like those are the only 4 builtin instances of an object

and 3 of them have valid names

(i heard some tutorials refer to those other 3 as keywords which is wrong right)

[paper echo]

i'm actually not sure what they are

they are definitely reserved words, you can't assign to them

!e ```python
True = 3

[fallen slate]

@paper echo :x: Your eval job has completed with return code 1.

001 |   File "<string>", line 1
002 |     True = 3
003 |     ^
004 | SyntaxError: cannot assign to True

[quasi hound]

oh interesting

you could in past versions tho right?

[paper echo]

while you can assign to e.g. list

!e ```python
list = 3
print(list)

[fallen slate]

@paper echo :white_check_mark: Your eval job has completed with return code 0.

3

[paper echo]

no idea

[halcyon trail]

yes

[paper echo]

!e ```python
import ast
root = ast.parse("True")
print( root.body[0].value )

[fallen slate]

@paper echo :white_check_mark: Your eval job has completed with return code 0.

<ast.Constant object at 0x7f83b13dde80>

[paper echo]

not sure how to figure out what the constant "is" from here, if that's even possible

ah, that is what it is

a Constant

!e ```python
import ast
print( ast.parse("list").body[0].value )

[fallen slate]

@paper echo :white_check_mark: Your eval job has completed with return code 0.

<ast.Name object at 0x7f0f445c4e80>

[paper echo]

whereas list is a name

so True, False and None are constant literals that happen to also be syntactically valid names

[undone hare]

You can in 2.x, yes

[cloud crypt]

there is __debug__ also

I mean, it makes sense to have those immutable

[elder blade]

Yes, ellipsis are currently used for sentinel values.

[static bluff]

Hey all. I've asked for some general advice over in #help-croissant. I didn't want to clog up the chat here but it is a bit advanced. Just thought I send up a flare.

[elder blade]

That's because ... is often used as "etc." and whatever, so it reads nicely. I remember reading that you can also do: ```py
def whatever():
0

It's really just the same time.

!e ```py
def whatever():
0

[fallen slate]

@elder blade :warning: Your eval job has completed with return code 0.

[No output]

[elder blade]

No error ^

[elder blade]

Not really, that's more convention

[raven ridge]

Even conventionally, rather than pass it's better to give things with intentionally empty bodies (like protocols and abstract methods and custom exceptions) docstrings, and that's enough of a body to satisfy the Python grammar.

!e ```py
import abc
class MyClass(abc.ABC):
@abc.abstractmethod
def some_protocol(self):
"""Do something"""

class MyException(RuntimeError):
"""Raised when things fail"""

[fallen slate]

@raven ridge :warning: Your eval job has completed with return code 0.

[No output]

[limpid sigil]

Hello, where is a good resource to learn more about how python code is interpreted? Is there a tool that can break down .pyc files into readable mnemonics or something?

Like how it translates to actual underlying CPU actions

[paper echo]

CPython i think is conceptually a loop over bytecode instructions with a big switch statement in it. So python bytecode doesn't directly correspond to machine code as such. But all the opcodes are listed here if that helps https://docs.python.org/3/library/dis.html#opcode-NOP

I'm not sure where this is all defined in the C source, probably not too hard to find though

[limpid sigil]

Oh perfect, I didn't know python used bytecode. That will help a lot.

[paper echo]

And of course this is all totally different for Pypy, Graalpython, etc

[limpid sigil]

yeah I was talking about cpython. Thank you

[limpid sigil]

So the bytecode is non-standardized, right? So depending on which implementation you look at, things are done vastly differently.

Is the CPython VM similar time the JVM?

[grave jolt]

yeah, the language doesn't define the implementation in any way

[limpid sigil]

Perhaps I'm asking a dumb question, I haven't done much research yet, mostly looking at things briefly on my phone between IRL things

[grave jolt]

CPython is pretty slow in comparison, if that's what you're asking

[limpid sigil]

Oops, autocorrect. Meant similar to

So like, does it have a stack register heap etc

[spark magnet]

The big difference between JVM bytecode and Python bytecode is that Python doesn't support the bytecode as a supported interface. It frequently changes from release to release.

[spark magnet]

But yes, it's a stack machine (no registers though), with values on a heap.

[warm badger]

Is this of interest? Bytecode disassembler for pyc files. https://docs.python.org/3/library/dis.html

@limpid sigil love the nick https://xkcd.com/1814/

xkcd: Color Pattern

♫ When the spacing is tight / And the difference is slight / That's a moiré ♫

[limpid sigil]

:P

Not just a nick

[limpid sigil]

Thank you. I could swear I saw someone using a CLI tool to do that, but maybe it was just a script to do that, or in misremembering. I'll try to remember where I found it. In the meantime, thanks for this.

[warm badger]

You can probably build a CLI tool from that real quick! 👍

[limpid sigil]

True.

[finite sparrow]

bout the whole "bytecode is a implementation detail" thing. its a implementation detail to the point that there are python implementations without any bytecode

[limpid sigil]

Yeah.

Or implementations built on an entirely different platform, like Jython

[grave jolt]

well, Jython is stuck on Python 2

but, for example, Brython is a thing -- it compiles to JavaScript

There's also a GraalVM implementation, but it seems to be in early development

[limpid sigil]

Graal is deprecated I thought

[spark magnet]

There's also PyPy, and IronPython (dead), and RustPython (nascent)

[small quest]

in 3.9 (and maybe earlier) they compile to the same bytecode, which is new. I found that interesting

[elder blade]

Huh, that's cool

[small quest]

@warm badger for .pyc you probably want to look at the marshal"library, as pyc files are just python objects "marshalled" into a binary format. You'd need to parse the objects out, then you can dis them

Be warned, it's not documented well, if at all.

[halcyon trail]

the problem IMHO with the whole "X is an implementation detail" is that if your language doesn't have a spec, then the language is just in effect "however this one implementation behaves"

so, bytecode is an implementation detail, that's true, but CPython is still effectively python

[small quest]

That's why CPython is the "reference" implementation

[finite sparrow]

hmm, would nuitka be a python implementation or something else?

Nuitka is a source-to-source compiler which compiles Python code to C source code

not sure if that qualifies as a python implementation

[spark magnet]

yeah, i'm not sure about that one 🙂

[raven ridge]

I think it's certainly fair to call a Python compiler an implementation of the language

[limpid sigil]

Python with extra steps, if you will.

[spark magnet]

I just don't really know anything about nuitka, that's why I said that

[small quest]

I've heard interesting things about nuitka. It doesn't sound like pyinstaller or any of those other runtime bundlers

[limpid sigil]

I mean even saying "C source code" can be pretty vague because there are a lot of libraries that you might or might not take advantage of, and different ways to do the same thing

[undone hare]

source-to-source compiler
Why not just call it a transpiler

[true ridge]

The primary (and probably extremely outdated) reference we have is here about the classifications of different python implementations/compilers/extensions: https://wiki.python.org/moin/PythonImplementations

[elder blade]

I was gonna say haha

[iron perch]

What's the best way to store a I have a board class and I want to make a board state, but idk how to implement board state
I want to make a board from that board state and also be able to compare it from other boards what's the optimal way :pixels_emoji_2: :pithink:

[spice pecan]

IIRC nuitka isn't a transpiler in the traditional sense

It produces C code that makes calls to the CPython API

[elder blade]

That's Cython

Nuitka seems to simply take Python code and directly compile it to C.

[spice pecan]

Cython is the superset that partially compiles to native C code, while AFAIR nuitka bundles the interpreter and makes equivalent calls directly from C code instead of regular interpretation

[undone hare]

Does the if __name__ == "__main__ structure has a name?

I saw someone call it "name equals main guard" online

[astral gazelle]

https://stackoverflow.com/a/19578335/5607403
This guy calls it a main guard

"name equals main guard" is too long, unpythonic

[undone hare]

hmm okay

[undone hare]

hmm, that's interesting, thanks

[small quest]

Anyone familiar with import hooks? If I want to override source code parsing of normal source file packages/modules, do I want a meta path hook or just a path entry finder?

[halcyon trail]

I keep meaning to write a macro/template/snippet/whatever in my IDE that inserts

import sys
import argparse

def main() -> int:
    parser = argparse.ArgumentParser()
    return 0

if __name__ == "__main__:
    sys.exit(main())

This is how I always write python files usable as scripts

[spark magnet]

if you are looking for tweaks, you can pass sys.argv into main() to make it a testable function.

[halcyon trail]

That's a fair point though usually main just has argument parsing, and then calls a real function

and it's the real function that I would test

would you pass sys.argv or sys.argv[1:]

[small quest]

if you're making a template I'd pass sys.argv so you can keep it generic

[halcyon trail]

iirc when you pass an argument to ArgumentParser.parse_args it's not supposed to include the name of the script so you have to do the 1: either before you pass to main, or in main

but then if you were testing it you'd need an extra pointless entry in your list, at the start

[small quest]

oh hmm

[spark magnet]

excellent question. I think i waffle between the two 🙂

[halcyon trail]

🙂 Over the last 5 years, I've written a lot of python files usable as scripts, as I'm on a small team where we just had a ton of little utility pieces of code

[spark magnet]

but more often sys.argv[1:]

[halcyon trail]

and I waffled over all kinds of things, some of which were pretty annoying

the worst was probably, for a long time I was calling the parsed arguments ns

short for namespace

because the type of the returned object is Namespace

take notes; this is really bad naming

[small quest]

i always call it args lol

[halcyon trail]

yeah, I switched to args and I also waffled some between ap and arg_parser and parser

usually these days I think I call it parser

a lot of times now though I unpack args into individual variables with type annotations

if you do something like args.output_dir, it won't realize that its a Path, and you won't get any benefit from mypy or your IDE

oh, main in my code also somewhat often has the logic to do things like send an email, or trigger an alert, in case of failure, so then I'd need options to disable those things in order to run a test. Another reason why I'd probably never really test main

[spark magnet]

i hope the logic to send an email doesn't have bugs in it. that sounds like something i'd especially want to test.

[halcyon trail]

how do you test that

You either setup an automated email server or something like that, or you mock out the email sending completely. The first is a ton of effort, the second, well, you have a test now, but one that catches very few meaningful bugs

I found this with a lot of my very, hmm, struggling for an adjective, "system-y" python code. Like, I have scripts where part of what you do is ssh into boxes, using paramiko (or not), and do various things. You can test in certain places to make sure that your logic for creating the commands is sensible, or how you process files, etc, but you have all kinds of things there that are either a ton of work to test properly, or you can do a moderate amount of work to add tests that test almost nothing of interest

[safe hedge]

It's incredibly refreshing to hear someone else express this sentiment. I'm aware that sometimes it's an indication you should refactor your code, but I have often found myself tearing my hair out over the idea of how much I'd have to mock to make it easily testable!

[halcyon trail]

The thing is, when you end up doing all that mocking, you'll probably have tests that don't really test much. There was actually a blog post recently from a guy about what he had learned at google, and he specifically mentioned this point

[magic python]

I'm not too sure i follow what you mean, is this referring to the notion that the tests will be written using toy examples with known behaviour and fail to catch the real bugs / errors that would cause trouble?

I dont test any of the wrappers that I have around some google gcp utils, though that's mainly because i'm not sure how i would 😬

[halcyon trail]

well, pretty much

like, if you have some code that runs some commands over ssh, sometimes it was tricky for me to make sure that the commands worked exactly the way I wanted, especially once it gets tricky with multiple &&'s, sourcing a conda environment, etc

if I wrote a mock myself, what would the mock look like? Like, it would accept some string to run over ssh and then do what with it? Probably look for some known set of errors, but the errors that I can easily code the mock to reject are also probably the ones I can easily avoid by construction in the real code.

The bugs i encountered with that code were exactly the things that I didn't understand, like, ok, how exactly does bash precedence in a one liner work in this situation, ok

[hollow hill]

hey guy ! is no / low code development going to replace traditional coding development in the future ?

[spark magnet]

no

[hollow hill]

Can you tell me more details why ?

[spark magnet]

people have been talking for decades about some technology or other that is going to "make programming obsolete". Someone always has to tell the computer what to do, and that's always a complicated process. Programmers will be needed.

@hollow hill https://en.wikipedia.org/wiki/Fourth-generation_programming_language

[hollow hill]

Thank you !

[raven ridge]

The hard part of programming isn't the programming language, it's breaking down the problem and organizing the solution in a way that is resistant to changes over time.

[spark magnet]

or that can adapt to changes over time

[quasi hound]

damn that's really cool

and weird to think about

github copilot is 6GL

[raven ridge]

It's definitely true that we'll keep making tools that make it easier for programmers to focus on the important parts of the problem and spend less time on boilerplate and incidental complexity

but software development has lots of inherent complexity that's never going away.

[quasi hound]

another cool thing to think about

assuming we ever make an AI that can write programs well (not github CP), or just real artificial intelligence of some kind (which we don't have now btw it's just really good guesses and pattern matching trained from data), and we make it write its own code, it'll improve itself exponentially

[raven ridge]

except that we'd need to define what it means to "improve", which means explaining to it what is and isn't a good solution. Which will be a form of programming.

[quasi hound]

and never stop

https://en.wikipedia.org/wiki/Technological_singularity

kinda like this

*exactly like this

but yeah

we'll oversee our computers until they can do it themselves

and then who knows

btw i think a real AI could learn on its own (dk much about AI but i know sometimes they use two separate programs or whatever that outcompete each other and turn it into an arms race)

without any of that

without being force fed giant datasets before it starts making any good guesses

don't get me wrong what we have now is cool

but it's just pattern matching

and looking at more possibilities than we can at a time

for example the chess AIs

if you printed out every possible next move and a human had time to look at all of them and consider the options then they'd do no worse than an AI

only difference is the speed

tldr AIs only know what we know even if "omg this chess AI can beat every human on earth every time"

because we're teaching them

they need to start learning on their own

then they'll actually be intelligent

[raven ridge]

AI researchers have claimed that Artificial General Intelligence is only 20 years away for more than 50 years. I wouldn't hold my breath.

[quasi hound]

how do they know

maybe i've read too much sci fi but the AI isn't a slow discovery where we start putting together the pieces

maybe one day it just happens

you know what i'd want to see

an AI where you give it its own source code and it runs from the updated version

and it can modify it

and if it fucks up the generation or whatever ends

but AIs that are smart will start figuring out parameters

and adjusting them

dude that's scary

isn't it

could be done honestly

i'd fuckin love to see it

with some restrictions

like if it modifies a variable name in the code it refactors all of it

cause if you type a random character somewhere on the page with any source code file in any language there's like a 99.5% chance the code will break

probably more

so yeah there's gotta be some restrictions

[raven ridge]

mutation testing is a thing. That's something like what you're describing.

but it's off topic for this channel.

[quasi hound]

that's cool

[south oar]

Well

im working on scratc

The first thing i do when i learn c

is reverse every application out there

thats gonna be so fun

[stuck valley]

I'm working on scratch.
The first thing I'm going to do when I learn C
is reverse every application out there**.**
That's... ambitious.

Hi. I'm learning about dunders and magic methods. How can I fix this?

>>> class foo:
    def __concat__(self,other):
        return 5
>>> x=foo()
>>> ''+x
Traceback (most recent call last):
  File "<pyshell#5>", line 1, in <module>
    ''+x
TypeError: can only concatenate str (not "foo") to str

[raven ridge]

You're looking for the __radd__ dunder

!e ```py
class C:
def radd(self, other):
return 5
print('' + C())

[fallen slate]

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

5

[stuck valley]

Is there a complete list of everything dundered in python?

[raven ridge]

When you do x + y, Python first calls type(x).__add__(x, y). If that returns NotImplemented, Python calls type(y).__radd__(y, x) to give the other class a chance to try.

[stuck valley]

And what if type(y).__radd__(y,x) returns nothing? Error?

Also, I note the order of the operands is reversed - but that makes sense because of the self, other structure.

[raven ridge]

Yes, and that's also what the r in __radd__ stands for

[stuck valley]

I assumed it meant right - but that makes sense too.

[raven ridge]

Technically "reflected", not "reversed", but same thing

https://docs.python.org/3/reference/datamodel.html#special-method-names is the docs for all of the funders, but there's a lot of them.

[stuck valley]

Thanks.

[raven ridge]

https://www.google.com/url?q=https://m.youtube.com/watch%3Fv%3DS58rRIBi0A0&sa=U&ved=2ahUKEwj265u434fyAhUKXc0KHSOaDnsQtwIwAHoECAMQAQ&usg=AOvVaw14JxJkgpF2tAy-yJ3uZYU0 talks about the most common ones.

[stuck valley]

Thanks! I'm learning so I can participate in #esoteric-python .

[rancid pulsar]

https://stackoverflow.com/questions/68571148/aws-cognito-sso-user-authentication-with-django-rest-framework-react-js-and-al

Does anybody know an answer to this

[raven ridge]

That question doesn't fit this channel's topic. Try #web-development

[cobalt agate]

I wish to make a simple application using KivyMD. I want to display the latest block and pending txns on the ethereum blockchain automatically. I'm using infura url and web3.py.
pending_block = web3.eth.getBlock(block_identifier='pending', full_transactions=True)

What do I do to make the app automatically get this every time a new block originates? Listeners? Websockets? Or use a while loop that sleeps for a specified time and calls a function to fetch the latest data?

So that my app automatically updates the pending_block by pinging the infura url...

[eager trail]

guys, I'm team lead now and I've been thinking of instituting some required reading because some of my team are... lacking... in python and software engineering fundamentals

anyone got a recommendation for a concepts book?

had a friend recommend code complete but it's from 2004

[stuck valley]

What are the attributes of a frame?

[undone hare]

https://docs.python.org/3/reference/datamodel.html#frame-objects

[stuck valley]

thx

[peak spoke]

The inspect docs also have some information in them and similar objects

[charred pilot]

fluent python is great

[valid rose]

what is an import hook? i tried searching around and couldnt find any good stuff

[small quest]

You put a Finder class into sys.meta_path
When you import a module, the Finders in sys.meta_path are scanned, and the find_spec method of each one is called, with some data about the module. The first one that returns ModuleSpec is used, and the Loader that spec contains is used to load the module

[paper echo]

@verbal escarp ☝️ this is the thing i was wondering about, if it made sense to use this functionality in justuse

[iron perch]

https://paste.pythondiscord.com/ajirazucoq.py
Start_game here won't be abstract anymore

Because when I can't render it to pygame or anyother thing

What do I do with this?

[iron perch]

Someone said decency injection

Should I do it?

[tacit moth]

Hey folks! Facebook wants to create a metaverse. Now I was thinking: "Imagine if I can create an AI to be me in this metaverse, and I would be able to stay out of the matrix." You know, I hate Mark and Facebook. They are destroying our society.

[indigo violet]

One thing I've noticed with AlphaStar is it values minimizing the amount of inputs per frame or per minute

Pro players do 300-400 actions per minute in Starcraft where AlphaStar did about 250

[paper echo]

@iron perch this probably belongs in #software-architecture

!ot @tacit moth @indigo violet AI is interesting but off-topic here

[fallen slate]

Off-topic channels

There are three off-topic channels:
• #ot0-psvm’s-eternal-disapproval
• #ot1-perplexing-regexing
• #ot2-never-nester’s-nightmare

Their names change randomly every 24 hours, but you can always find them under the OFF-TOPIC/GENERAL category in the channel list.

Please read our off-topic etiquette before participating in conversations.

[verbal escarp]

hi, sorry i didn't see it earlier 🙂

well, we have 3 distinct cases in justuse - use-path, use-url, use-str

use-path and use-url are both explicit loaders for single modules, so that's out

use-str is where things get tricky with packages

what we do now, we zipimport pure python packages, which works great

but anything beside pure python packages is really hard

the first thing i declared as goal was to make numpy work with use, which was way harder than expected

because numpy itself monkeypatches itself and sys.path during installation

next thing i tried was sqlalchemy, which wasn't even doable so far because that one is compiling binaries on the target machine, which is crazy

we then thought maybe protobuf would be less problematic than numpy as a test

but protobuf hacks around package names, which also needs special casing

so now we ended up with a package_hacks.py where we try to collect those special cases until we find some pattern to properly handle everything

so.. importing really isn't the issue, installing is

[paper echo]

but why not use the existing setuptools or pep-517 infrastructure for installation?

i thought you were just calling out to pip with a subprocess anyway

[verbal escarp]

that won't work because pip strictly enforces the "one and only one version per package" policy

[paper echo]

i had previously suggested using a temporary site-packages for each installation

so pip wouldn't "see" the other versions

did that not work out?

like how nixpkgs works, kinda

[verbal escarp]

which i think was ruled out because virtualenvs can't be set up from within an interpreter, i think?

nixpkgs?

[paper echo]

rather, the nix package manager. as in https://nixos.org/

[verbal escarp]

does that work from within a python script?

[paper echo]

no, it's a whole separate thing

[verbal escarp]

if there's a solution to the installation conundrum, i'd gladly take it

[paper echo]

you can't create a venv with subprocess.run?

[verbal escarp]

i haven't looked at that venue to be honest

[paper echo]

subprocess.run([sys.executable, '-m', 'venv', temp_site_pkgs])
subprocess.run([temp_site_pkgs/'bin/pip', 'install', *pip_install_args])

then do whatever you need to do in order to import the installed module/package from the temp_site_pkgs dir

[verbal escarp]

hm.. if that works just like that..

let's give it a try 🙂

[paper echo]

otherwise you're literally reinventing pip

with all its backward compatibility and pep-517 forward compatibility

until/unless pip introduces a stable python API i don't see another way to universally install packages

[verbal escarp]

well, we've taken some pip code already 😉

[paper echo]

also this now supports wheels which is important

[verbal escarp]

hmm.. i think it could work that way. we know the folder, we only need pip to install things, not import

if we protect sys.path from alterations, we're golden

[paper echo]

what do you mean by "protect sys.path from alterations"?

you can use the machinery you wrote that bypasses sys.path

in fact arguably you should?

[verbal escarp]

well, as i said, numpy modifies sys.path during installation to find different parts

for example

[paper echo]

ah, is that in its setup.py?

[verbal escarp]

i think so

[paper echo]

https://github.com/numpy/numpy/blob/main/setup.py#L373 this doesn't affect the sys.path of the parent process if you run pip install with subprocess.run

[fallen slate]

setup.py line 373

sys.path.insert(0, src_path)```

[paper echo]

seems fine to me

the bigger problem is that if you run pip install --no-deps then you have to manually find and install deps in the parent env

or if you run pip install with deps then you reinstall all the deps for every installation, NPM-style (although with pip download caching maybe that's not so bad)

[verbal escarp]

if i knew how messy auto-installation would be, i might not have went for it in the first place ;p

[paper echo]

you could also use python -m venv --system-site-packages to work around the issue https://docs.python.org/3/library/venv.html

[verbal escarp]

with a subinterpreter? ^^

[paper echo]

no, with subprocess.run

[verbal escarp]

hm.. not sure if that fixes the issue

hmmm

i have an evil, evil idea

how about replacing sys.path by a property and serve exactly that path the package is looking for 😉

could make a kind of diff to see whatever the package tried to add to sys.path during installation, memoize that and whenever something from within that package tries to import something, serve that memoized path

i really wished i didn't have to do that, but many people resort to adding their stuff to sys.path to make installation work, there's almost no getting around it

now i forgot why i came on discord in the first place..

[paper echo]

You seem to be conflating installing and importing?

Install with pip in a new venv, prepend venv to sys.path or use your other functionality

Or do you need to do some wacky stuff like hook into every import statement and resolve it differently based on what module invoked it

I guess i still don't see why you can't just use pip with subprocess

[verbal escarp]

i'm not, the problem is following up with package-internal imports

[paper echo]

If you don't use it, you still have all the same importing problems, trying to import the right version of a module from the right place

Ah yeah

Yeah, I wonder if you need to monkey patch parts of the import machinery, if that's even possible

Or otherwise really deeply hook in there

The other issue is that things might break if you have two different versions of a module loaded in two different namespaces

Like if "justuse bar" imports foo 1.2, but at the top level you also have "justuse foo 1.5"

Even if you can resolve the technical problem of hooking sufficiently deeply into the import system, you might have some difficult decisions to make about situations like that

Could it lead to confusing error messages? Unexpected behavior and crashes? Memory integrity issues in C extension modules?

[verbal escarp]

well, i wouldn't even encourage loading different incompatible versions of the same package into the same program, but with the way pip and imports work at the moment, you can't even have different versions in the same environment

[paper echo]

Right

That's what the temporary virtual environment thing would hack around

You don't want to just say "don't load incompatible versions" because then there is a lot of burden on the end-user to audit dependencies, which seems kind of antithetical to the idea of your tool

That said, in a lot of use cases version incompatibilities probably just wouldn't matter

[verbal escarp]

i thought of checking call signatures similar to what i had in mind for reloading

then you could at least be sure that call signatures are compatible when loading different versions

into the same program, that is

and give an appropriate warning to the user

in the sense of "well, we warned you, do what you will" :p

but i really want at least to make it possible to have different programs in the same environment with different incompatible dependencies

that would be a huge step forwards

[paper echo]

Yeah, I think in most cases it might "just work"

Or maybe it would never work, because modules are loaded by name and cached globally

It might "just work" if you override enough of the import system...

I agree it would be pretty cool and kind of magical, although I question what the ultimate benefit is

[verbal escarp]

well, there's the idea of public and private dependencies that resonated and stuck with me since last time we talked about dependencies a while back

you could declare public dependencies that are managed by pip and imports across all code that "just works" like requests etc, very close to the stdlib

and stuff you want to deploy but you can't be sure it wouldn't interfere with other dependencies, you could handle as private dependencies, via justuse

that way you could avoid dependency hell where pip would upgrade a dependency to a version that is incompatible with other stuff

[elder blade]

How would private dependencies work with Python?

[tacit hawk]

Maybe that private dependencies should be vendored

[elder blade]

I don't think Python distinguishes between installed packages and your code

[elder blade]

Yeah

[tacit hawk]

If I am not wrong pip uses vendoring package to vendor its dependencies

[verbal escarp]

we were talking about https://pypi.org/project/justuse/

[grave jolt]

btw, are there plans for justuse to somehow integrate with typecheckers like mypy?

[verbal escarp]

you could use("your_package", version="1.23", hash_value="adsf463", modes=use.auto_install) to install the package with a specific version and hash without interfering with any pip-installed stuff

[elder blade]

That's cool!

[verbal escarp]

oh god, mypy 🙂

we tried really hard to make mypy happy, but it didn't make us happy.. we'll probably try another time, when the whole typing is a bit more mature

but that probably didn't answer your question

[verbal escarp]

what exactly did you mean by integration?

[grave jolt]

like, making editors aware of stuff inside the modules imported by use

for intellisense and such

[verbal escarp]

i'm not sure how justuse could help with that, tbh

do you have anything specific in mind? i'm always open to suggestions..

[grave jolt]

Editors (and tools like mypy) are aware of the contents of the modules you import with import. Does that work with use?

[verbal escarp]

heh..

that's really up to how the autocompletion is implemented on the editor side, i'm afraid. with some it does, others not so much, some execute the code partially, others just make a rough analysis..

if there is anything on my side that could help, i'd happily put some extra effort into that

[grave jolt]

mypy has a plugin system, maybe it's possible to make a plugin to make it work?

[verbal escarp]

you think editors use mypy to help with the autocompletion?

[grave jolt]

no, that won't help with autocompletion, but will make mypy happy 🙂

well, not happy, just aware of the stuff in the module, so it could e.g. point out errors

[verbal escarp]

so use() would run mypy before importing stuff? 😄

[grave jolt]

hm?

[verbal escarp]

i could run mypy before importing stuff 😄

[paper echo]

what would type checker integration do? fail to import if mypy fails?

or more like, generate type stubs on the fly for the imported module?

[verbal escarp]

at least tell the user they're trying to import a crappy module 😉

[grave jolt]

No, that's not at all what I meant, I meant creating a plugin so that mypy thinks that use returns a module with specific names with specific types that it takes from a stub file or something, instead of returning Any ModuleType

but yeah, I guess it would require downloading a stub file at inspection time

[tacit hawk]

like proxy type inference beyond module's content

[verbal escarp]

Use is a module now and returns ModuleTypes 😉

as i said, we already tried to make mypy happy, quite successfully, but it made our code look pretty ugly

[paper echo]

but their contents are not knowable statically

pd = use('pandas')
pd.DataFrame().index

does mypy understand this?

is that even possible?

[verbal escarp]

Optional[ModuleType]

sure, it knows use() returns a module or None

[paper echo]

but it doesn't know that pd has a DataFrame attribute

[verbal escarp]

hmm.. not sure, we only checked mypy on our own code

[grave jolt]

I guess the user could do ```py
if TYPE_CHECKING:
import pandas
else:
pandas = use("pandas", version="1.2.3.4.5")

[halcyon mural]

hey guys is there anyway in pycharm that i cant type without it deleting my code

[verbal escarp]

would creating stub-files on the fly help with that?

[paper echo]

im honestly not sure that problem is solvable now that i think about it..

or globally caching use invocations and pointing mypy to the installed modules

[verbal escarp]

that could work

not sure how, though..

does mypy know about modules loaded via importlib?

then i could check how importlib makes things mypy-able

[grave jolt]

I'm pretty sure that's not possible with mypy natively, that would require a plugin

[verbal escarp]

it's possible, yes

actually, that's pretty much equivalent to import pandas

it's useful in cases like use("pprint").pprint(stuff) where you don't want to add things to the global namespace

[grave jolt]

from importlib import import_module

module = import_module("math")

reveal_type(module)
reveal_type(module.pi)
``` this results in: ```
main.py:5: note: Revealed type is "types.ModuleType"
main.py:6: error: Module has no attribute "pi"
main.py:6: note: Revealed type is "Any"
Found 1 error in 1 file (checked 1 source file)

[verbal escarp]

oops. well 🙂

if it doesn't even work with builtins..

[grave jolt]

well, importlib is typically used with dynamic data, as I understand, so it wouldn't make much sense to account for import_module("math") and such

[verbal escarp]

dynamic data? it's just the functional counterpart to the import keyword as it is..

[grave jolt]

special/dynamic stuff like dataclasses, pydantic etc. always requires a plugin, mypy doesn't do any deep code inspection across function boundaries (i.e. it just looks at the types of symbols)

[verbal escarp]

just today we talked about static code analysis for justuse, https://securitylab.github.com/tools/codeql/ was one of the things that came up

well, i don't know enough about static code analyzers to make a good case for any one solution

[grave jolt]

I don't know anything about them either 🙂

I guess it's a good idea to wait until typing matures a bit?

[verbal escarp]

i'd love to revisit that subject in the future, yeah

or maybe you'd like to give it a try and experiment with one or the other and give me something to work with 🙂

salt rock lamp already helped a great deal with a prototype for threaded reloading

[grave jolt]

I'll give it a try after I'm done with my current project

[verbal escarp]

looking forward to it 🙂

[halcyon trail]

@grave jolt it's really annoying IMHO that dataclasses/attrs (less familiar with pydantic) and mypy, didn't introduce some kind of type annotation representing the generated class

they just completely hardcoded it in and there's no way to do anything else with it except write your own plugin

so, if you want to write say a decorator that even just changes some of the defaults of the @dataclass decorator

I don't know any easy way to write that decorator and have it work with mypy

Just silly, IMHO

it would be really nice if somebody wrote up a tutorial about how to actaully do this, as a toy example, using mypy's plugin system. I googled around and found some info but nothing very easy to walk through, and then just gave up on the whole idea.

[grave jolt]

@halcyon trail yeah I wish there was a way to express some more complicated concepts directly in code, but I suppose that's not an easy task...

there are some ideas specifically about dataclasses-like classes, though: https://github.com/microsoft/pyright/discussions/1782

[verbal escarp]

i wished the whole typing stuff was an actual DSL and not some tacked-on hack 😉

[grave jolt]

yeah I like TypeScript

[cobalt agate]

Has anyone tried importing web3 lib into their flask server file? Flask stops serving up on running.
#help-lemon message

[small quest]

I strongly recommend pyright over mypy

[grave jolt]

I am using pyright. Don't remember exactly why, but I liked it better than mypy for what I was doing. Why do you recommend it over mypy?

I guess the biggest disadvantage of pyright is lack of a plugin system, so an ORM/serialization tool won't play nicely

[halcyon trail]

so far all I know is that pyright seems to, iirc, correctly handle a recursive type definition

which is really useful for properly typing json.

like, that's probably the only place I would ever, in writing python code for the next ten years, use a recursive type definition

but it's a doozie

doing this with mypy breaks its skull

@verbal escarp not sure what you mean by "DSL", but it sounds like you just mean a statically typed language?

[verbal escarp]

not really, i was thinking more of {}, [], () etc as syntax for protocols, abstract types rather than concrete types

[halcyon trail]

not really sure what that means...

[verbal escarp]

i mean, why go with typing.Dict if you could have {} as a syntax

it doesn't make sense to me

and using a DSL where appropriate isn't new in python..

[small quest]

Pyright handles "proper" types way better than mypy in most cases. It has fewer false positives and edge cases

[raven ridge]

How would that extend to classes with no literal syntax? Would it be

def foo() -> [int]:

for a list of ints, but

def foo() -> deque[int]:

for a deque? Wouldn't it be confusing for the [] to mean different things in the two cases?

[verbal escarp]

hm.. maybe there could be syntax to denote specific types like <deque>[int]

[raven ridge]

If you do that, you're making the overwhelmingly common case much uglier

[verbal escarp]

isn't the common case to just say [int]?

btw, deque would be included in that

[raven ridge]

You think a list of ints is a more common type hint than, say, None or int?

[paper echo]

[] by itself is already syntactically different from []-as-__getitem__ so i don't see the issue

[int] and {str: int} could be very nice syntax

[raven ridge]

I don't think they're much nicer than list[int] and dict[str, int]

[paper echo]

[int] is already familiar from a lot of other languages

[raven ridge]

I'm not sure what you mean by this.

[paper echo]

fair enough. also one could argue that "blessing" list and dict this way might be a bad thing

[raven ridge]

it might encourage over-use of standard types over library defined types, for one thing.

[verbal escarp]

i mean [int] wouldn't have to be limited to list but could denote iterables in general

[raven ridge]

then how would you type a list?

As typing stands today, Iterator[int] and Iterable[int] and Sequence[int] and list[int] are 4 different types. How would your DSL denote each of them?

[verbal escarp]

no idea. there's a lot to consider and i'm not paid to work on that 😄

[raven ridge]

no one is 😄

[verbal escarp]

isn't guido basically?

i thought he was hired by MS to work on anything he likes

[paper echo]

Make [] be Sequence

[raven ridge]

Since coming out of retirement, I think he is getting paid to do nothing but develop Python again

[verbal escarp]

makes sense to me

[paper echo]

Or have mypy automatically pick up names from typing

[raven ridge]

oh, I totally agree about that one - if a name used in a type hint isn't bound in globals() but is found in typing, type checkers should assume that's what you meant

every type checker already needs to know the builtins - it would be as easy as adding each of the typing names to their builtins list.

[verbal escarp]

apropos DSL, i've noticed there were discussions about adding additional syntax to replace Optional

something like ?int

[raven ridge]

We'll already have int|None

[verbal escarp]

not sure how far the discussions are at this point

[raven ridge]

adding a 3rd way to do it seems excessive to me.

[verbal escarp]

wouldn't be the first time to have 3 or more ways to achieve the same thing 😉

[raven ridge]

usually they're introduced over the span of 3 decades instead of 3 years, though 😄

[verbal escarp]

xD

well said

[raven ridge]

and usually they have different tradeoffs, instead of literally being different spellings for the same thing.

[verbal escarp]

yeah..

[raven ridge]

hindsight being 20/20, they probably shouldn't have ever added Optional[X]. I think the goal was just to give a shorter and easier to quickly read alternative to Union[X, None], but X|None is shorter than both, and as easy to read

and Optional always confused people - functions that have required Optional parameters or parameters that aren't required but aren't Optional caused a lot of confusion.

[verbal escarp]

nodnod

[halcyon trail]

I think the literals are a mistake in the first place, they super hardcode specific types.

you're looking at the literal syntax, which special cases in a few types, and saying "hey, let's special case those types for typing also"

for no real benefit

the real question is the opposite, how much benefit do we actually get by special casing in literal syntax.

in python specifically you get decent benefit because if you didn't have list comprehension, set comprehension, etc, the alternatives are really ugly

but looking at different languages they manage to get things about as nice, without special casing in anything

[raven ridge]

It's hard to imagine how a language could function without at least some types of literals. int and string, at least.

[halcyon trail]

yeah, sorry, I guess I should have said, container literals

that is a pretty interesting point though

int and string literals are still literals, at the end of the day, so what if someone wants a different string? Their string is always second class because it doesn't have a literal.

But I think that's probably a more reasonable compromise to make than for contaienrs (and for integers, even more so)

C++ has user defined literals which probably is reasonable for C++ but for other languages probably not needed

actually, sorry, I was really thinking more of comprehensions, than literals

I really failed to make that clear 🙂

[native flame]

what

[fallen slate]

:incoming_envelope: :ok_hand: applied mute to @jade field until <t:1627616212:f> (9 minutes and 59 seconds) (reason: duplicates rule: sent 4 duplicated messages in 10s).

[eager trail]

thanks fam

[hard ravine]

Hi! want to scrape a website through XHR requests with the help of python inside Calibre E-book manager. I am not allowed to use selenium. I think json module can help me out, but I have no idea how to work with json module to get the necessary contents from a website. Any help would be greatly appreciated. Thanks in advance.

[acoustic crater]

#❓｜how-to-get-help

[unkempt rock]

Anyone

[unkempt rock]

Can I take data scientist after completing CSE???

[valid rose]

this is not the channel @unkempt rock , but yes. you can become anything if you're good enough

[unkempt rock]

Thanks dude 👍

[native flame]

definitely not metaclass, I would lean towards ABCs

[swift imp]

Does Java have metaclasses? I've never used it but I thought it was missing a lot of language features and is why so many of those patterns you read about for java arn't necessary for python

[cobalt agate]

I've built a webpage in Flask that shows details of the last 3 ethereum blocks mined, and all pending transactions in the current block:
https://github.com/zubin-madon/Ethereum-Block-Updater

[cobalt agate]

Challenges I faced: Heres the challenging part that I DIDNT KNOW:

• Flask is a synchronous app. So if you try to use web3 in the same .py file, the flask server will not load. Thank you to ANON in this server who helped me understand that.
  Even importing the web3.py file into the flask_app.py file gave issues.
  So the key to using web3 in tandem with Flask is to obtain whatever data you want within functions. And then calling ONLY those functions from your flask file, which return the final data.
  If you try to run flask and web3 concurrently from the same .py file nothing will work.
  Anyway I'm just a beginner so perhaps this is something most pros would know already I assume.

• I am saving all the data as a pandas dataframe and then displaying it in a table using jinja in html. By doing so, for some strange reason the damn CSS would not apply to the table when I was pointing to the .css file within .html
  So I just wrote all the css in html file and then it worked. Wasted 15 hours trying to link to the css file.

The page auto refreshes every 20 seconds. Thats a little more than what it takes to mine a new block on ETH.

[halcyon trail]

something like metaclasses is a lot more complicated in statically typed languages, very few statically typed languages have something quite like that

since python is dynamic, types themselves are really still values, there's no fundamental distinction, so talking about the type of a type makes sense

if you're statically typed then types and values are just completely different, so talking about the type of a type doesn't make sense.
The closest thing maybe, in some sense, is a generic, because a generic type in some sense is a blueprint for stamping out types, the way that a metaclass does. But it's a lot more limited.

and yeah interfaces == ABC's pretty much

[gleaming rover]

try #web-development

[boreal umbra]

In [3]: %timeit bob = []; [bob.append(1) for _ in range(1000)]
64.1 µs ± 825 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

In [4]: %timeit bob = deque(); [bob.append(1) for _ in range(1000)]
61.9 µs ± 688 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

In [5]: %timeit bob = []; [bob.append(1) for _ in range(10_000)]
624 µs ± 7.41 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [6]: %timeit bob = deque(); [bob.append(1) for _ in range(10_000)]
622 µs ± 8.12 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

I didn't expect these to be so close.

[gleaming rover]

isn’t it more that there’d be no need to support something like that

because it’d just be super reflection heavy

at the same time

I don’t think metaclass functionality is essential…?

[prime estuary]

It doesn't make sense there since the classes and their behaviour is defined at compile time, so the compiler has to know what they do...

[gleaming rover]

yes, it’d basically be building a dynamic type system in a statically typed language

but is it more complicated (assuming the language sufficiently supports reflection?)

[craggy jacinth]

Yeah it does - its a class called Class.

[blissful comet]

I made some usability improvements to typpete. Please give it a shot and send me feedback. Upstream has been inactive for ~3 years. Still trying to figure out if I should fork or not.

https://github.com/adsharma/Typpete

It's a SMT based type inference program (unlike mypy and pyright, which check types).

[craggy jacinth]

I'm not super versed with the limitations of manipulating class instance data at runtime within Python, but in Java there is an API called Reflection, such that you can basically access or invoke (anything that's executable) any property of a class object at runtime (constructors, fields, methods, etc) or load other classes via ClassLoaders and what not. You can't exactly modify the behaviour of classes though via reflection - you can penetrate the security of certain areas even if their access modifiers do not permit such activities, and you can, on a basic level, manipulate small, trivial areas of the class, but nothing major (you can also prevent this too). To do such a thing (truly edit with complete freedom), you'd have to use the instrumentation library, and create a Java Agent (most people use the ASM framework). This would allow you to inject and/or manipulate the bytecode of classes whether that be only on their runtime instance, or their .class equivalent (basically the same as Python's .pyc bytecode files - cpython impl), and many more features. I assume there is something pretty similar in Python though?

[grave jolt]

Why?

[magic python]

Is it possible to create a decorator that'll validate the types of a function argument?

I thought pydantic did this, but it coerces.

[grave jolt]

can you elaborate what you want? do you want to validate type hints?

[swift imp]

yes but its quite limiting. You can very easily get the annotations from a function to check the inputs

[grave jolt]

yeah, I think this is a somewhat frequent question here

you inadvertently run into limitations like type-checking a list (just traverse the two million items and check each one, right), function, generator etc.

[charred pilot]

typecheck an infinite iterator 😔

[magic python]

:(

Yeah I guess you could do anything that isn't a list or a function or a generator 🤔

[grave jolt]

or dict

etc.

[magic python]

Or that, hmm. I've been using attrs validators instead of dicts recently and really like them

I'm wondering whether there'd be any value checking the simple types like int, string etc at run time, maybe not

[grave jolt]

I think static type checking (mypy/pyright/etc.) gets rid of most type errors tbh

unless you're using something really dynamic

[magic python]

Yeah just it doesn't check runtime

Maybe if there was a cli hook to run mypy on a script then the script that could be useful lol

[grave jolt]

Well, the whole point of static typing is that you find bugs before you hit runtime