#internals-and-peps

1 messages Β· Page 149 of 1

raven ridge
#

eh, it's not stupidity - it's just a feature that you never needed and didn't need to learn about. It's a feature that's used much more in enterprise Python than open source Python, because enterprise stuff needs to pick names for everything that are guaranteed not to cause problems by conflicting with (current or future) PyPI packages

grave jolt
#

if you care about security shouldn't you be using a pypi mirror anyway? or is it about the case where you'd want to use a library that has the same name from pypi?

raven ridge
#

yep, the latter.

#

our PyPI mirror overlays our stuff on top of the OSS stuff, but obviously it's a problem if we were to want to depend on a public library that has the same name as one of our private libraries

#

which is bound to happen relatively often, if you have a few thousand Python developers in the company and let them choose their own names without any guidance.

grave jolt
#

we have a completely air tight pypi πŸ‘€

raven ridge
#

and the reason why implicit namespace packages are better than the old, explicit ones was the problem of who was responsible for creating the magic __init__.py file for the old, explicit ones. In practice, in order to package them for systems like Debian, you needed to create a package for each namespace package that contained only the __init__.py, so in my example above you'd have 3 explicit namespace packages, one containing each of py somecompany/__init__.py somecompany/team1/__init__.py somecompany/team2/__init__.py and the somecompany.team1 and somecompany.team2 packages would need to declare a dependency on somecompany, and the somecompany.team1.package_a and somecompany.team1.package_b packages would need to declare a dependency on somecompany.team1, and the somecompany.team2.package_c package would need to declare a dependency on the somecompany.team2 package, and the somecompany.team[12].package_[abc] packages would need to explicitly exclude somecompany/__init__.py and somecompany/team[12]/__init__.py from their sdist.

#

which was an absolute nightmare to explain to people, and to get right - and when someone got it wrong, the most likely result was that it would just break the ability to find the package across multiple directories on disk, and things that are installed would just stop being importable when you installed the new, incorrect package (though importing the new, incorrect package itself might work!)

#

so: PEP 420 made it much simpler to create and maintain namespace packages. The biggest downside to PEP 420 is that it also made it possible to accidentally create namespace packages - in fact, it's now easier to create namespace packages than the regular packages that almost everyone actually wants

grave jolt
raven ridge
#

nope. Different type of package, for an entirely different purpose. Lots of people's takeaway was "Now I don't need an __init__.py", but that's not right - namespace packages aren't a substitute for regular packages, and some feature that work for regular packages - like, say, importlib.resources - don't work for namespace packages.

#

spread the word πŸ™‚

grave jolt
#

the import system in Python is really confusing

#

it seems to be really customizable for some reason

raven ridge
#

yeah, it's very complicated

grave jolt
#

if you want a unique name, can't you just name it acme_string_utils instead of acme.string_utils?

raven ridge
#

sure, at the cost of not being able to use relative imports, and needing to use import acme_string_utils as string_utils instead of from acme import string_utils, and so on.

prime estuary
#

You end up with smurf naming where 'acme' is repeated everywhere.

raven ridge
#

"smurf naming" πŸ˜„ Love it.

grave jolt
#

well... that's just unavoidable, right? if you want to depend on that library

prime estuary
#

And also acme itself is also a module, so you can import that.

grave jolt
#

and what if you need a PyPI library called acme? πŸ˜›

raven ridge
#

it's quite nice, in an enterprise world, that if company.team.app wants to depend on company.team.library, it can do from .library import stuff instead of from company_team_library import stuff

grave jolt
#

I guess

prime estuary
#

If two packages exist with similar apis, it's possible you could import one, dynamically switch.

raven ridge
grave jolt
#

but is the nicity worth the complication?

prime estuary
#

Oh, that's another reason - you can vendor a library under say yourlib.thirdparty.whatever, and it'll be contained there and not affect the toplevel.

#

Stuff like pip does that for instance.

raven ridge
#

I can't imagine the nightmare of needing to deal with a project where every filename had the company and team and project name embedded as part of the filename.

grave jolt
#

Is there a mechanism like this in other languages?

#

How do they solve it?

#

most enterprise software is probably not written in Python

raven ridge
#

most, sure. But lots is.

#

and more every day πŸ™‚

grave jolt
#

I honestly would be okay with ```py
from acme_string_utils import left_pad

#

maybe I'm just weird

raven ridge
#

er - hm? That's what we have today, with namespace packages, right?

grave jolt
#

Fixed @raven ridge

#

or, well, acme_sales_string_utils

raven ridge
#

as a user, that's not much different, I guess - but it makes a big difference to repo layout. Instead of py acme/ sales/ string_utils.py you wind up with ```py
acme_sales_string_utils.py

grave jolt
#

or maybe I misunderstand again

raven ridge
#

I'm talking about the experience for the maintainer of the acme.sales.string_utils library

#

that repo needs to either contain acme/sales/string_utils.py (the namespace package route) or acme_sales_string_utils.py (your fake namespace proposal)

#

the latter is less nice to work with, in practice.

grave jolt
#

Why? Just the name being longer?

#

well, maybe I just lack the 'practice' part

#

I'll come back when I encounter a situation like this πŸ™‚

raven ridge
#

yeah, the name being longer, and harder to tab complete, and harder to recognize at a glance. It's just less convenient to deal with a file name that's 50 characters long than one that's 15 characters long, in a lot of tools - like, fuzzy-finding files in an IDE will be harder if every file contains a common prefix that your pattern will wind up matching against

elder blade
verbal escarp
#

by forgetting __init__.py?

#

it's a major pita, imo

raven ridge
#

a package without an __init__.py is a namespace package. Normally you don't want a namespace package - they're a niche feature. But by forgetting one file, you can accidentally make the wrong kind of package.

elder blade
#

So discord.ext is a namespace package?

raven ridge
#

what do you see if you:

import discord.ext
print(discord.ext)
elder blade
#

And that's why you can install other stuff and import it using discord.ext.xyz?

elder blade
#

Interesting

raven ridge
#

so yep, that's a real life example of a namespace package

#

CC: @grave jolt πŸ™‚

grave jolt
verbal escarp
#

it probably could

raven ridge
#

because multiple different packages want to install stuff under it

grave jolt
#

that's cursed

#

if you ask me

raven ridge
#

that's... namespace packages

#

that's their whole thing πŸ™‚

grave jolt
#

right

#

I think I'll never understand namespace packages

#

I'm giving up

#

I'll go write Rust

verbal escarp
#

just never forget __init__.py and you're fine, most of the time

#

you'll know it when you need it

lusty scroll
#

if you have a directory where you put your projects/repos, and the subdirectories have, that are the same as a namespace package names (an existing one installed elsewhere), you cannot run python from the directory and import a module having the same name as a project directory, without messing with PYTHONPATH ?

#

it's like the directory version of "Don't name a file io.py"

#

my workaround is to juat name the directory something else. but it seems like __init__.py were still required, then python wouldn't see those directories. I am aware ut will prefer protobuf.py over a folder named protobuf, though, which is good

verbal escarp
grizzled sleet
#

Guys I have a question and its:

#

how to link python code to GUI

#

For exempel add URL to a button

visual shadow
tame imp
#

I am doing a node graph where all the node inputs (input slots) are procedurally generated and filled based on the nodes execute() functions parameters and return type; using typing and __annotations__. I want to reload that execute function in a reload function on the node so that the user doesn't have to restart the application every time a node's execute() function has been changed (as in the source code of the node subclass).

The code is going to be used in a pipeline where the developers are actively developing the nodes while using the application, meaning that having to restart the entire application for the execute function to to reload on the node that has is currently being worked on is a MAJOR downtime.

There are two possible interface solutions that could work, ether the user reloads the entire active graph; which i want to avoid if possible. The other option would be to selectively update the execute function on the node(via. the user right click->refresh from source).

But this is where it gets a bit complicated, the logic i ideally would want would be something like this:

newModule = importlib.__import__("userNodesModule")
classToUpdate.execute = newModule[classToUpdateName].execute

But this has one fatal floor, now if i try to re-import this "userNodesModule" again, the base module is changed:

>>> import importlib
>>> thisModule = importlib.__import__("math")
>>> def dummyPrint():
...     print("Hello World")
...
>>> thisModule.acos = dummyPrint
>>> thisModule.acos()
Hello World
>>> otherModule = importlib.__import__("math")
>>> otherModule.acos()
Hello World
#

I want to update just one function in the current runtime without affecting the other parts of that modules, since that would almost certainly break stuff with active references.

Sudo code for the node base class:

class BaseNode(SomeGuiBaseClass):
   def reload(self):
      # Reload the execute function - this is what i can't figure out how to do.
      # Rebuild node input parameters from self.execute.__annotations__.

   def executeImplicit(self):
      # This is the only place execute() will ever get called from.
      # The args variable is pieced from the node inputs and verified against self.execute.__annotations__.
      args = [...]
      execute(*args)

   def execute(self, string: str, integer: int):
      # The definition and implementation of execute is overwritten in all inheriting classes.
#

A working example of the "front-end", user faceing side of the code.

class StringConstantNode(KSNodeItem):
    def execute(self) -> str:
        return "String123"
    
class PrintString(KSNodeItem):
    def execute(self, string: str, boolean: bool) -> None:
        print(string)

nodeGraph = KSNodeGraph(None)
nodeGraph.addNodeType(StringConstantNode)
nodeGraph.addNodeType(PrintString)
lusty scroll
tame imp
#

Could you clarify what you mean when saying "you need some scopes to execute in"?

verbal escarp
#

and don't reference those functions other than via module-attribute lookup

spice pecan
#

just return a proxy from attribute lookup on your module object and then update all proxies when a module is reloaded /s

#

Though that wouldn't preserve state, just handle assigned objects

verbal escarp
#

object proxies? hmm

#

interesting idea, but it's problematic if object instantiation has side-effects

spice pecan
#

It's incredibly problematic and still leaves a lot of things untreated, such as instances of reloaded classes still referencing the old class

#

You could maybe remedy the instances with a proxy metaclass, but something tells me it's going way too far

verbal escarp
#

hmm

#

in justuse we return a ProxyModule by default, so we could keep track of object instantiation.. if we'd keep track of those references via gc, we could replace those objects' class by the reloaded one behind the scene

#

it's really.. no idea what it is

#

i'm not even sure if it could work

spice pecan
#

You could replace types with your own proxy that would store weakrefs (so as to not mess with gc) to instantiated objects and then reassign their class on reload

#

But again, it's not exactly something I'd use in a real project, more of an interesting idea that I'd love to tinker with at some point

verbal escarp
#

you're welcome to try and come up with a prototype, we could put it in justuse as an experimental feature (with a big fat warning)

#

i'm fairly happy with the function-only reloading so far, and i do check if a module contains anything other than functions to give a warning.. so we could put it there

grave jolt
#

!ban 754353181227745440 14d seems like you're only here to drop some random link. Please re-read our #rules.

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied ban to @short marsh until <t:1638634384:f> (13 days and 23 hours).

lusty scroll
#

e.g. a module dict

tame imp
#

I see, thx all!

white nexus
#

!e
ah, logic
print((True + True ) * (False - True) * (False - True) ** (True + True))

fallen slateBOT
#

@white nexus :white_check_mark: Your eval job has completed with return code 0.

-2
raven ridge
#

!e print(~True)

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

-2
feral cedar
#

luckily there's no hashing happening ?

white nexus
feral cedar
#

it's bitwise not, basically inverts all the bits. for numbers, it's equivalent to -x - 1

raven ridge
#

In Python, it does -x-1

white nexus
#

oh neat

fallen slateBOT
#

@unkempt rock :white_check_mark: Your eval job has completed with return code 0.

001 | -5: -5
002 | -4: -4
003 | -3: -3
004 | -2: -2
005 | -1: -2
006 | 0: 0
007 | 1: 1
008 | 2: 2
009 | 3: 3
010 | 4: 4
011 | 5: 5
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/yerowujeyi.txt?noredirect

feral cedar
#

it's because -1 is used a "flag" by the hash function for something

raven ridge
#

it's... weird. Normally it only makes sense to do bitwise operations on unsigned numbers, so it's weird to see that Python picks an implementation of it that's entirely based on signed integers.

#

well, depends on what you mean by "correct"

#

!e print(bin(5))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

0b101
raven ridge
#

if Python says that the bits of 5 are "101", then the bits of ~5 should be 010

#

but they're not:

#

!e print(bin(~5))

fallen slateBOT
#

@raven ridge :white_check_mark: Your eval job has completed with return code 0.

-0b110
raven ridge
#

yes, there is some definition for what the bitwise operation does by which what Python does is reasonable.

#

but it's not what people coming from other programming languages are likely to expect.

#

it's made more complicated by the fact that Python doesn't have both signed and unsigned integers, of course.

white nexus
#

that is weird

#

!e print(bin(abs(~5)))

fallen slateBOT
#

@white nexus :white_check_mark: Your eval job has completed with return code 0.

0b110
elder blade
raven ridge
#

for ints, at least - yeah

spark magnet
#

C and Python show the same thing for ~6: -7

raven ridge
#

yeah, fair enough - I more meant that it's weird to do bitwise operations on signed ints in the first place, and ~6U and ~6 are different in C.

lusty scroll
#

Is it just me or is Counter a bit hard to use

#

every time I use it I end up doing it wrong or else I have to look it up, despite using it occasionally for years now

peak spoke
#

I did find it a bit awkward to use the last time I needed it for something

spark magnet
#

how so?

lusty scroll
#

why is there no method to add/insert an item ?

spark magnet
lusty scroll
#

so it's like a dict that that stores counts instead of values?

spark magnet
#

yes

feral cedar
#

it's just a bag

gleaming rover
#

is there any other language that does if expressions like Python (true expression first)

spark magnet
feral cedar
#

i think they mean the ternary thing, true if cond else false

lusty scroll
#

C?

#

you mean before the condition

elder blade
#

Yeah, JavaScript does cond ? true : false

lusty scroll
#

lua seems to do if <condition> then <true expr> else <false expr>
I assume they mean <true expr> ... <condition> ... <false expr> because the former is pretty common? @gleaming rover

lusty scroll
#

a ha

#

just looking at haskell gave me an actual headache just now

lusty scroll
#

ah, ok

gleaming rover
#

most functional languages that I know do if <expression> then <true expression> else <false expression>

lusty scroll
#

it seems to be called Luau?

#

it has gradual typing.. thinking if you'd say Python has gradual typing

#

ah, cool

#

the type annotations remind me of python's but they have type? for optionals and I assume other differences as well (() instead of NoReturn)

paper echo
#

Luau is like typescript for lua

#

Not the first of its kind but it has a big company behind it so that means clout and adoption

#

Also null/nil in lua is weird

#

It's like javascript but they use nil for both null and undefined

#

Which can be kind of chaotic and it makes handling "missing data" difficult

#

Not to mention that "arrays" are just tables with integer indexes and they are also kinda sorta nil-terminated

#

Great embedded scripting language though

unkempt rock
#

Is there a difference/preference between ```py
class A:
def iter(self):
return iter(self.listofstuff)

vs

class B:
def iter(self):
yield from self.listofstuff

#

I guess A makes an actual iterator so it feels more correct pithink

ruby pilot
#

A is definitely better because py type(iter(B())) # --> generator is pretty weird

#

IMO, idk if there's a standard for this

prime estuary
#

The reason I'd suggest the first is that it's probably a bit more efficient, since you don't have a generator in the way every iteration.

prime estuary
#

Well iter(A()) returns the list-iterator, which is written in C. Then no Python code is executed to loop that after then.

#

For B, each next() has to go into the generator, then go to the iterator.

spice pecan
#

The second creates a new intermediate layer that will slow it down

verbal escarp
#

well, let's see

prime estuary
#

Though there's the lookup of the iter() call, A might be slower for small lists perhaps.

weary flax
#

if i continuosly refresh a website, will the owner notice ?
and will he ban me ? (im doing it with selenium, and running the program pretty much 24/7)

prime estuary
verbal escarp
#

if there's a cache somewhere along the way, you'll just hit that

raven ridge
#

If you're using selenium to automate accessing a website and are afraid that the owner will find out, that sure sounds like a violation of rule 5.

weary flax
verbal escarp
raven ridge
#

!rule 5

fallen slateBOT
#

5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.

raven ridge
#

Whether it results in a DOS is immaterial. The question is whether the website's terms of service allow that access

#

If you're worried about the server owner finding out, it sure sounds like they don't.

weary flax
#

basically i need to access data in the table and get it asap (like every second matters)

#

i wonder if refreshing the website will make it faster

verbal escarp
#

and let me guess, an API key would cost you money

weary flax
raven ridge
#

If the site doesn't want you refreshing it in a loop, we won't help you with that here.

weary flax
verbal escarp
weary flax
#

ye

verbal escarp
#

what scanner?

weary flax
#

its a stock screener, it updates every second, and if my selenium script will find an alert that qualifies certain parameters, it will scrape that

verbal escarp
#

is the scanner part of the website?

white nexus
#

lol this might be too cursed, but how can I dynamically generate a function signature given the signature of a different function?

weary flax
#

so does it take significantly more bandwith if i continously refresh it, than just staying connected ?

weary flax
#

like inside

white nexus
raven ridge
verbal escarp
white nexus
#

i am too tired even, since re-reading that question is not what I meant to ask

#

provided one function definition, how to dynamically generate a new function that has an identical signature?

Eg, if we were to have sys.exit which takes just a code, to replace this with a fake exit method with an identical signature that does other stuff inside

verbal escarp
white nexus
white nexus
verbal escarp
#

using those types is a good idea for replacing callables

#

i used those for aspectizing of descriptors and bound methods, was very simple that way

white nexus
#

neat

spice pecan
#

Use *args, **kwargs to pass params and let wraps handle the rest

white nexus
#

oh neat

verbal escarp
#

iirc wraps didn't solve some problems i had with bound methods, but i can't recall the details

gusty marsh
white nexus
#
>>> assert enhanced_foo(1, 2) == 3  # positional 'b'
hello!
b=2
>>> assert enhanced_foo(b=0, a=1) == 1  # keyword 'b'
hello!
b=0
>>> assert enhanced_foo(1) == 2  # default 'b'
hello!
b=1
#

uhhhh

#

that looks slightly problematic

#

given this code: ```py

our signature-preserving wrapper

@wraps(foo)
def enhanced_foo(*args, **kwargs):
print('hello!')
print('b=%s' % kwargs['b']) # we can reliably access 'b'
return foo(*args, **kwargs)

#

it seems like it would convert the *args to **kwargs

white nexus
#

!d operator.attrgetter

fallen slateBOT
#

operator.attrgetter(attr)``````py

operator.attrgetter(*attrs)```
Return a callable object that fetches *attr* from its operand. If more than one attribute is requested, returns a tuple of attributes. The attribute names can also contain dots. For example:

β€’ After `f = attrgetter('name')`, the call `f(b)` returns `b.name`.

β€’ After `f = attrgetter('name', 'date')`, the call `f(b)` returns `(b.name, b.date)`.

β€’ After `f = attrgetter('name.first', 'name.last')`, the call `f(b)` returns `(b.name.first, b.name.last)`.

Equivalent to:
white nexus
#

so this is interesting.

#

most importantly:

The attribute names can also contain dots

#

so its a recursive getter and setter

peak spoke
#

how is it a setter?

white nexus
#

er

white nexus
# peak spoke how is it a setter?

*just getter, but it can be used for setting too since its recursive. eg get the parent of an object, and then set it last. A little more complicated but not by much.

paper echo
#

I actually didn't know you could use dotted names with attrgetter

#

i still use the 3rd party library glom for this

#

!glom

fallen slateBOT
#

When adding functions or classes to a program, it can be tempting to reference inaccessible variables by declaring them as global. Doing this can result in code that is harder to read, debug and test. Instead of using globals, pass variables or objects as parameters and receive return values.

Instead of writing

def update_score():
    global score, roll
    score = score + roll
update_score()

do this instead

def update_score(score, roll):
    return score + roll
score = update_score(score, roll)

For in-depth explanations on why global variables are bad news in a variety of situations, see this Stack Overflow answer.

elder blade
#

Did you mean this lol

#

nvm isn't in the docs

verbal escarp
paper echo
#

lol oops

#

!pypi glom

fallen slateBOT
#

A declarative object transformer and formatter, for conglomerating nested data.

twilit ridge
#

i have below 2 lists and my expected output. if target [0] and target [1] (adjacent fruits) then simply name
both as FRUITS. keep on doing the same. if adjacent are not FRUITS, just simply leave as it is..

fruits = ['apple','orange','grapes','mango','pineapple']

target = ['orange','mango','CAR','apple','pineapple','BUS','apple']

expected_output: ['FRUITS','CAR','FRUITS','BUS','apple']

i wrote a solution

while (z<len(target)):
      if (target[z] in fruits) and (z+1<len(x)) and (target[z+1] in fruits):
        new_list.append("FRUITS") #replaces with FRUITS 
        z+=1
      else:
        new_list.append(target[z]) #if not fruits, replaces as it is
      z+=1```

i feel this solution is not efficient/more confusing. Is there any better ways to solve this?
As always,thanks for reading πŸ™‚
white nexus
#

hmmmmm

#

is there a difference, if any, between using dict() and {}?

naive saddle
#

{} can't be overridden and is a bit faster

#

!eval

import dis
print(dis.dis("{}"))
fallen slateBOT
#

@naive saddle :white_check_mark: Your eval job has completed with return code 0.

001 |   1           0 BUILD_MAP                0
002 |               2 RETURN_VALUE
003 | None
naive saddle
#

!eval

import dis; print(dis.dis("dict()"))
fallen slateBOT
#

@naive saddle :white_check_mark: Your eval job has completed with return code 0.

001 |   1           0 LOAD_NAME                0 (dict)
002 |               2 CALL_FUNCTION            0
003 |               4 RETURN_VALUE
004 | None
white nexus
#

πŸ‘€ what is dis?

naive saddle
#

it's a standard library module

white nexus
#

ah

white nexus
spice pecan
#

You can unpack an empty tuple

#

But that's pretty cursed

#

!e ```py
set_literal = {*()}
print(type(set_literal))

fallen slateBOT
#

@spice pecan :white_check_mark: Your eval job has completed with return code 0.

<class 'set'>
spice pecan
#

Courtesy of fix error I think

white nexus
#

although

#

that is actual neat to know

#

since builtins can be reassigned, so if you're worried someone would do that to where you don't even want to use builtins...

#

!e ```py
import builtins
builtins.set = list
s = set(('g','h','j','k','l'))
print(s)
print(type(s))

fallen slateBOT
#

@white nexus :white_check_mark: Your eval job has completed with return code 0.

001 | ['g', 'h', 'j', 'k', 'l']
002 | <class 'list'>
deft pagoda
#

reminds me of the improved throwaway variable

#
for {}[()] in "Hail Satan":
    666
spice pecan
#

A bit more verbose than _, but at least it's an actual throwaway

grave jolt
white nexus
grave jolt
#

Why not {}[0] though?

#

1 char shorter

deft pagoda
#

{}[()] is a throwaway variable

deft pagoda
grave jolt
#

it is

#

I tried

deft pagoda
#

i tried

grave jolt
#

well you should've tried harder

deft pagoda
#

i just summoned satin

#

and now i'm comfortable

analog magnet
#

how do i make a lexer

lusty scroll
marsh salmon
#

Hey guys! newbie here, if I want to start learning python, what would be the best website for it?

halcyon galleon
fallen slateBOT
#
Resources

The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.

lusty scroll
#

😹😹

boreal umbra
#

(My above message is a joke, to make that explicitly clear.)

grave jolt
#

that is not esoteric, I push code like that to prod every weekday

#

i have 0 contributions but all of them are extremely useful

#

(set theory joke)

boreal umbra
unkempt rock
#

Yes

white nexus
feral cedar
#

there

white nexus
#

wut i just learned that aiohttp has a webserver

boreal umbra
#
>>> {'a', 'b'} * {'c', 'd'}
{('a', 'c'), ('a', 'd'), ('b', 'c'), ('b', 'd')}

not practical given that itertools.product exists, but what if?

prime estuary
#

Well, what happens if you need 3 products?

#

Does it special case tuples?

#

How does {(1, 2), (3, 4)} * {'a', 'b', 'c'} behave?

#

{1, 3} * {2, 4} * {'a', 'b', 'c'}?

boreal umbra
#

!e

import forbiddenfruit
import itertools

forbiddenfruit.curse(set, '__mul__', lambda a, b: set(itertools.product(a, b)))
print({'a', 'b'} * {'c', 'd'})
fallen slateBOT
#

@boreal umbra :white_check_mark: Your eval job has completed with return code 0.

{('b', 'd'), ('a', 'c'), ('a', 'd'), ('b', 'c')}
boreal umbra
prime estuary
#

Sorry.

boreal umbra
#

!e

import forbiddenfruit, itertools
forbiddenfruit.curse(set, '__mul__', lambda a, b: set(itertools.product(a, b)))
print({1, 3} * {2, 4} * {'a', 'b', 'c'})
fallen slateBOT
#

@boreal umbra :white_check_mark: Your eval job has completed with return code 0.

{((3, 2), 'a'), ((1, 2), 'a'), ((3, 2), 'c'), ((3, 2), 'b'), ((1, 2), 'b'), ((3, 4), 'a'), ((1, 2), 'c'), ((1, 4), 'a'), ((3, 4), 'c'), ((3, 4), 'b'), ((1, 4), 'b'), ((1, 4), 'c')}
boreal umbra
#

Like that, apparently. I don't make the rules here.

prime estuary
#

That's consistent, but kinda an ugly result, who wants to type for (((a, b), c), d), e in ...

boreal umbra
#

(this is not a serious suggestion btw)

#

maybe, but my non-serious suggestion would be counter-intuitive to those familiar with cartesian products and unfamiliar with the limitations of Python's grammar.

pliant tusk
#

!e ```py
import itertools
import fishhook

@fishhook.hook(set)
def mul(self, other):
return itertools.product(self, other)

@fishhook.hook(itertools.product)
def mul(self, other):
cls, args = self.reduce()
return cls(*args, other)

for a, b, c in {'a', 'b'} * {'c', 'd'} * {'e', 'f'}:
print(a, b, c)

boreal umbra
#

(namely that cartesian products are treated as one big chain, whereas Python evaluates it as (((a * b) * c) * d)

fallen slateBOT
#

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

001 | a c e
002 | a c f
003 | a d e
004 | a d f
005 | b c e
006 | b c f
007 | b d e
008 | b d f
prime estuary
#

That's a solution, product the product.

pliant tusk
#

basically set multiplication produces an product object, then product multiplication makes a new copy with the extra arg

boreal umbra
#

(either you'll know what dialogue this is referring to or you won't)

#

what is itertools.product.__reduce__ @pliant tusk?

pliant tusk
#

it is used for pickle

prime estuary
#

That's for pickling.

#

So it dumps the internals of the iterator, which is very much private implementation details but who cares here.

pliant tusk
#

!e py import itertools print(itertools.product('a', 'b').__reduce__())

fallen slateBOT
#

@pliant tusk :white_check_mark: Your eval job has completed with return code 0.

(<class 'itertools.product'>, (('a',), ('b',)))
lavish tartan
#

Print("Hello world")

sly jungle
#

@lavish tartan βœ… Your eval job has completed with return code 1.

Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'Print' is not defined. Did you mean: 'print'?
boreal umbra
#

@sly jungle for a moment I was about to check if you were a self bot

pulsar ridge
#

Any better way to take two inputs

#

It taking two seconds

undone hare
grave jolt
rapid schooner
#

is anybody familiar with the tweepy api?

white nexus
#

lmfao

surreal sun
radiant garden
#

Cartesian products are cool, but why not Minkowski sums?

#

!e

import fishhook
@fishhook.hook(set)
def __add__(A, B):
  return {a+b for a in A for b in B}

print({1, 2} + {1, 4, 5} + {-1, -3})
fallen slateBOT
#

@radiant garden :white_check_mark: Your eval job has completed with return code 0.

{0, 1, 2, 3, 4, 5, 6, -1}
radiant garden
#

The fun thing is, you can recurse this

#

{{1, 2}, {3, 4}} + {{4, 9}, {1, 16}}

surreal sun
feral cedar
#

you can read the definition right there

surreal sun
#

oh lmao

white nexus
#

oh my god

#

type hints are so weird

#

because they can be so easily abused

white nexus
#

actually i just remembered that #type-hinting exists so I'll be there

lusty scroll
#

well, they can run arbitrary code, but I don't know if that counts as abuse, since you can already run arbitrary code in the same python file

lusty scroll
#

has anyone proposed updating python's interactive console so it can parse code like this? I know there "should" be a blank line there, ideally, but it's accepted in a .py file so I thought it might make sense to accept it in the repl

snow bronze
#

I know its made to parse only one block of code....

#

but why? what difficulty it faces?

flat gazelle
#

ipython and bpython tend to make up the difference, though I really see no point not supporting it

snow bronze
#

Yeah..., if they allow it, it just saves us time in copy pasting code to test

#

i mean if you use IDLE instead of some fancy stuff like jupyter notebook!!

verbal escarp
#

how about.. getting rid of IDLE?

spiral lily
#

you can do

def func(): pass
lusty scroll
#

Same result for me:
>>> def func1(): pass ... def func2(): pass File "<console>", line 2 def func2(): pass ^ SyntaxError: invalid syntax

elder blade
#

IDLE definitely has a place in Python

verbal escarp
#

which is?

gusty marsh
#

when you want to run short pieces of code or you are testing/playing out with python

#

you don't want to press the run button or python file.py everytime

gusty marsh
#

jupyter possibly, yeah

lusty scroll
#

jupyter requires pressing a run button?

verbal escarp
#

i meant jupyter is a possible replacement for IDLE in this particular usecase

lusty scroll
#

yeah, that could work

gusty marsh
#

jupyter runs on shift enter also IIRC

verbal escarp
#

not sure how thonny and other editors implement their "interactive consoles" though

snow bronze
fallen slateBOT
#

Source code: Lib/code.py

The code module provides facilities to implement read-eval-print loops in Python. Two classes and convenience functions are included which can be used to build applications which provide an interactive interpreter prompt.

white nexus
#

its publicly exposed, so you could make your own that does allow it IIRC

verbal escarp
#

possibly, not sure

fallen slateBOT
#

Lib/code.py lines 189 to 201

"""Closely emulate the interactive Python console.

The optional banner argument specifies the banner to print
before the first interaction; by default it prints a banner
similar to the one printed by the real Python interpreter,
followed by the current class name in parentheses (so as not
to confuse this with the real interpreter -- since it's so
close!).

The optional exitmsg argument specifies the exit message
printed when exiting. Pass the empty string to suppress
printing an exit message. If exitmsg is not given or None,
a default message is printed.```
lusty scroll
#

ah, I've tried it befire. It's a different implementation, right? in pure python?

white nexus
#

its what python -m asyncio uses

deft pagoda
#

that's what i used to make a kivy interpreter

lusty scroll
deft pagoda
deft pagoda
#

no, that's something different

boreal umbra
#

@exotic bay I deleted your comment, as it was off-topic.

halcyon trail
#

I actually didn't know that anybody used anything but ipython tbh

paper echo
ashen lance
#

Recently in one interview I was asked that how can I restrict the creation of class objects in python. Any idea about this?

boreal umbra
#

@ashen lance I assume by "class object" you really just mean "object". Every object is an instance of a class. A class object refers specifically to something else.

#

That said, you can implement the __new__ method for a given class

#

(on mobile so I'm not going to use markdown) recall that dunder init is not the method that creates the object. It is a method that gets called between when the object is created and when it is returned.

ashen lance
#

Got you, so if I just do something like

    pass

It will just skip creating the object, right?

#

Got it, thanks @boreal umbra and @unkempt rock

olive marsh
ashen lance
#

This is in regards to SQL
Let's say I have the following query -
select id from emp where email_address = 'abc@mail.com'

Now since we have 10 million rows in emp table its taking more than desired time, how can we reduce it?

I know using indexing it can be done, but is there anything done in pure SQL query to make it better?

If its a wrong platform please let me know, I'll delete the question/

sly jungle
#

what's wrong with indices?

#

if those are the queries you perform most often it seems like an index would be a smart choice

#

if you have 10 million rows then there's no way you'd care about the extra space needed for an index

undone hare
#

hey, this question would be better suited in #databases

verbal escarp
elder blade
#

Any ideas on creating subpackages?

#

Similar to NPMs form of @xyz/package

ashen lance
elder blade
ashen lance
# sly jungle what's wrong with indices?

Yea that was my question as well, but this was an interview question and they expected some ans

In same interview they asked me to explain how dictionary works on memory level... πŸ˜…

Long story short I combined my knowledge of linked list and uniqueness of keys and said something, but researching it now

gaunt jay
spice pecan
#

Raymond Hettinger had a talk about how python's dictionaries are structured, it's pretty interesting now they provide iteration over keys in insertion order together with reduced memory

#

IIRC there's an additional array of keys in insertion order with popped ones being replaced by a dummy value

grizzled spire
#

(Moving here from another channel)

  1. Anyone like this syntax? I often find myself wanting this when doing somewhat complex comprehensions
[w(z) for y:=g(i) for x:=f(i) for i in itter]
``` or perhaps
```py
[w(z) for y=g(i) for x=f(i) for i in itter]

basically, its the same as ```py
[w(g(f(i))) for i in itter]

however you can add if conditionals afterwards, as well as not having to define lambdas beforehand

especially if your individual functions are too simple to bother making lambdas but also would cause 
inefficiencies to nest manually [ex having two outputs] this seems like a nice way to do it
** **
** **
** **
2. Also, on an unrelated note can we allow `\` within f-strings? it's really annoying that this isn't valid code ```py
f"""the items are: {'\n'.join([1, 2, 3])}"""



  1. Also also, there should be some built in way to do x if x else default_none_type(x) and x if y else default_none_type(x) (Maybe just none(x) and none(x, y)?), basically eval x/y, if its bool form is true then return x, otherwise get (from the class of x) the "none" representation of it, so like if x is a list it would be [], a string it would be "", a number it would be 0, and a custom object you can define it as like def __none_value__(self) or something, with just None being a default
#

for the macros thing i would imagine it would be community made and be some sort of pre-ran converter

#

because im not sure how fully featured macro integration wouldn't break the "easy-to-read"ness of pythonic code

verbal escarp
grizzled spire
#

i was in another channel when i typed it

verbal escarp
#
  1. pipes would be far more useful
#
  1. yes please
pliant badge
verbal escarp
#
  1. eh.. that sounds weird
pliant badge
# pliant badge

I wanted to ask if there was a way to remove the bold lines from these captcha images.

grizzled spire
pliant badge
grizzled spire
verbal escarp
#

that makes sense, but hiding eval() in there probably isn't as straight forward as it sounds

grizzled spire
#

man that is uh, a pretty big AI project i think

pliant badge
verbal escarp
pliant badge
grizzled spire
#

move channels pls

verbal escarp
#

you might've noticed that the numbers look the same in those images, make a template and match

#

<end>

pliant badge
undone hare
#

!warn 837977275592867881 As said previously, we won't help you bypass captchas. Please stop asking about that here.

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied warning to @pliant badge.

spice pecan
#

I remember 2 being discussed in the mailing list, I think it's in the works? And 3 has significant overlap with None-aware operators (still hoping), and furthermore can easily be implemented as a function

unkempt rock
#

what is a use case for bypassing captchas

#

seems like a waste of time

grizzled spire
#

learning + automating

surreal sun
#

Speaking of PEP 505, any new news on it?

white nexus
#

wow.... tried pypy for the first time yesterday

#

tested it with a simple math question compared to cpython

#
import time
results = []
for _ in range(10):
 start = time.perf_counter()
 for a in range(100_000_000):
    a * a
 end = time.perf_counter()
 results.append(end-start)

for res in results:
  print(res)
import math
print('-------------------')
print(math.fsum(results))
#

pypy:

43.15774749999764

cpython:

118.40171311050653

#

it ends up being much faster than cpython

gleaming rover
white nexus
#

like?

gleaming rover
#

but more seriously...

#

most of the things that would be slow are delegated to a C (or other language) extension anyway

#

so in those aspects, CPython and PyPy are roughly as fast

#

and PyPy is overall less compatible with the libraries out there

white nexus
#

true

#

but that compatibility is probably because people don't test on pypy as much or use cpython implementation details

#

also pypy seems to be 2 minor versions behind

gleaming rover
#

it defo is

#

but ultimately...it doesn't really matter why, right?

#

because people gravitate towards the most convenient choice unless there's some compelling reason not to

white nexus
#

true

elder blade
#

Usually any speedup from PyPy is beaten by the speed of libuv

white nexus
#

libuv?

peak spoke
#

async library

white nexus
#

ah gotcha

feral cedar
#

kinda beating a πŸ’€ 🐴, but pypy speeds up your python code which didn't need to be fast anyway, and gives up easy C ffi, which you would use if you needed fast code

elder blade
#

Yup that's a nice way to sum up the whole discussion

white nexus
#

cffi?

gleaming rover
#

or How To Call C Code from Python

paper echo
white nexus
#

I was trying pypy for fun, I actually have my discord bot on it right now and notice an ample increase in speed

flat gazelle
#

ye, pypy is quite nice for text processing

#

for numerical computation there are better options, but if you need python types to work, it is quite nice

halcyon trail
#

yeah, I don't think I agree with the sentiment either. pypy makes python a generally faster language, at the cost of it being more difficult to speed up the things that are easy to delegate into C, which in practice is often math stuff like matrix multiplication and what have you

#

python being generally slower is seen by a lot of people as being a pretty huge disadvantage, IMHO

verbal escarp
#

well, it's worse

#

if speed was the only practical concern, but now it's also about green computing

halcyon trail
#

what's worse?

white nexus
#

green computing?

halcyon trail
#

he means energy efficiency I guess

verbal escarp
#

yes

halcyon trail
#

okay, I still don't follow the overall thought though

white nexus
#

so which is more energy efficient? lol

flat gazelle
#

there have been arguments emerging to use more optimised languages to save on emissions from running them

halcyon trail
#

yeah, those arguments mostly seem like virtue signalling tbh

verbal escarp
halcyon trail
#

running computers isn't a huge fraction of the energy budget. Even when we're doing profoundly wasteful things like cryptocurrency and NFTs

verbal escarp
halcyon trail
#

hahahaha

white nexus
#

IMO the blockchain is much more important than which languages are more energy efficient

#

the efficiency is more in the code that's being run, not the language

halcyon trail
#

Yeah, exactly

jovial flame
halcyon trail
#

crypto mining is a huge waste of many things

#

electricity among them

jovial flame
#

Ahh we are on the same page πŸ‘

white nexus
#

short example: using generators instead of making huge lists would be more efficient, and faster

flat gazelle
#

sometimes

#

sometimes a list is faster because you need lookback

grizzled spire
halcyon trail
#

can't tell if satire

grizzled spire
#

im 100% seirous

flat gazelle
halcyon trail
#

that's unfortunate then

grizzled spire
#

literally how

flat gazelle
#

but it is true that python is a waste of energy compared to other languages

grizzled spire
#

its adoption is bringing more freedom to the world

halcyon trail
#

....

flat gazelle
#

just well, it is not major enough to be meaningful

halcyon trail
#

positive this isn't satire?

flat gazelle
#

!silence

fallen slateBOT
#

βœ… silenced current channel for 10 minute(s).

flat gazelle
#

!unsilence

fallen slateBOT
#

βœ… unsilenced current channel.

white nexus
#

but there's another aspect to think about for energy consumption: that code had to be made. if it takes less time for someone to code a python program than a c program, lights, computer screen, their pc itself, etc

halcyon trail
#

short example: using generators instead of making huge lists would be more efficient, and faster
Interestinglyt his isn't even always true

#

it depends on the language and runtime

halcyon trail
#

I thought this was true in Kotlin and I was using asSequence() everywhere, and then i saw benchmarks and it turns out that it's not, even for pretty big lists

#

the problem is that the python runtime is very naive and doesn't really do any optimizations on end user code

flat gazelle
#

performance is too complicated for simple statements to be true

halcyon trail
#

so really python should not be taken as an example of anything, IMHO

flat gazelle
#

python performance generally isn't worth thinking about too much beyond "do I need to switch to native code for this"

#

text processing being kind of special here

halcyon trail
#

yes, and the third thought is "I wish I hadn't written this in python" lol

flat gazelle
#

indeed

#

CPU bound code in python is never a good experience

#

it is workable

halcyon trail
#

the thing is that I think people are too black and white, they point at something and they say "this is IO bound" like a web server

#

but the reality is that most things aren't purely one or the other, and in the real world we see that plain old tech companies using python for web server type stuff run into headaches very fast

#

well, not very fast, sorry. faster.

flat gazelle
#

ye, instagram did have to write their own JIT runtime to run their backend

halcyon trail
#

people that use Java or C# or Go are probably going to just be able to write everything in those languages and not ever have a concern, outside one or two niche cases

#

right, with python you have these things coming up again and again

#

dropbox and pyston, which they eventually gave up on, and have now added more Go, or maybe Rust, i forget which.

flat gazelle
#

java programmers do often complain about GC pauses for example

halcyon trail
#

they do, but usually issues can be solved by bringing in one JVM expert, doing some profiling and tuning.

white nexus
flat gazelle
#

but in the end, performance is not the most important quality for a language to write a web server in

halcyon trail
#

or paying for a low latency proprietary GC

#

it's not the most important, I agree.

#

the problem IMHO with python is that you throw away a lot of performance, in exchange for questionable benefit

flat gazelle
#

ye, these days there are actually good alternatives to python

halcyon trail
#

if you are comparing Java/C#/Go/etc to Rust/C++/C, etc, then it's a very clear trade-off

verbal escarp
#

well, there's performance and then there's performance

flat gazelle
#

though the maturity of the ecosystem is still nearly peerless

halcyon trail
#

GC languages are generally less complex, easier to learn, etc, and you take a moderate perf/memory hit.

verbal escarp
#

throwing more cores at a problem doesn't solve the efficiency issue, for instance

white nexus
#

gc?

verbal escarp
#

garbage collector

white nexus
#

what's a gc language then

verbal escarp
#

garbage collected language?

halcyon trail
#

i think it's only really defensible if you absolutely want dynamic typing, which not many people do. Even if you'r eindifferent, the huge performance hit of typical dynamically typed languages is hard to swallow.

flat gazelle
#

you use python for the standard library and ecosystem

halcyon trail
#

python has a good ecosystem but so do JVM and .net

verbal escarp
#

only because a language "feels" faster because it can do things threaded doesn't mean it's more efficient

white nexus
verbal escarp
#

which is another point to consider regarding energy efficiency

halcyon trail
#

what languages are we speaking about?

#

python has trouble with threads, but it's also a super slow single threaded language....

flat gazelle
#

I would take python's ecosystems over javas tbh. .net sure, but until very recently you could really only use it with windows server for hosting

halcyon trail
#

so not sure what example you have in mind

#

just because of how Java code/apis are written?

flat gazelle
#

because you aren't reliant on a half a dozen separate mavens being up at once to do your build successfully

#

also faster builds overall IME

verbal escarp
flat gazelle
#

and well, python stdlib > java stdlib

halcyon trail
#

practically all other languages solve these issues by being faster than python πŸ™‚

verbal escarp
#

any language that doesn't have a GIL?

halcyon trail
#

but that's just misleading, that's what I mean

#

Java doesn't have a GIL but it's also just way faster than python

#

same with C#, Go, Kotlin, etc etc

#

python is slower than almost everything, even single threaded

#

the GIL issues are just icing

flat gazelle
#

yeah, the GIL is IMO a worthwhile tradeoff

spice pecan
#

GIL is a solution, not a problem, and removing it won't magically make CPython significantly more performant

verbal escarp
flat gazelle
#

the GIL may be gone at some point

halcyon trail
#

i wouldn't call the GIL a "solution"....

flat gazelle
#

it solves the problem java has where ArrayList is thread unsafe

halcyon trail
#

it's a disadvantage of python, maybe it's a good trade-off for python, but it's clearly a disadvantage

#

compared to not having it

flat gazelle
#

whereas a python list can be accessed from 2 threads and never corrupt its internal state

halcyon trail
#

err, maybe you don't corrupt the internal state of the low level python data structures

verbal escarp
halcyon trail
#

but in any real program, you are still going to corrupt your own code's invariants

#

if you write multithreaded code without proper synchronization

verbal escarp
#

Ranking Programming Languages by Energy Efficiency

halcyon trail
#

nobody should be using a regular list for multithreaded programming... so I see this is as a non-advantage

#

when I say "use" I mean "concurrent reads and writes"

#

just reading in multiple threads is fine, of course.

flat gazelle
#

it is quite helpful when using threads for IO stuff

spice pecan
flat gazelle
#

only because fo the thread safety enforcement

spice pecan
#

yeah

flat gazelle
#

if python were just not allowed to run in multiple threads at all, removing the GIL would help perf

verbal escarp
#

hm

flat gazelle
#

and well, corrupting your own data is far easier to debug

halcyon trail
#

i just don't really understand the benefit, the GIL enforcement of thread safety is too low level to be some kind of huge benefit

#

it's a very moderate benefit at best

#

and the reality is that the lack of parallelism via threading leads to more complicated solutions, which are far more likely to have problems

flat gazelle
#

ye, parallelism is painful

halcyon trail
verbal escarp
#

there are a couple benchmarks

#

that table is just the normalized result

halcyon trail
#

also I'm confused, python is almost last on this list

verbal escarp
#

yup

halcyon trail
#

okay, i don't understand how this relates to your point at all

verbal escarp
#

except for memory consumption, btw

halcyon trail
#

you were making it sounds like other languages solve their speed problem with more cores, which doesn't help energy efficiency

#

but in fact almost all other languages are more energy efficient than python, as well

verbal escarp
#

but i think you've got a point too about the GIL possibly adding to the overhead for single-threaded code

#

hu?

halcyon trail
#

gotcha

#

yeah, I mean I wasn't really arguing so much that the GIL adds overhead (though it does). basically just saying:

  1. python is slow, period.
  2. the GIL is a downside of python, not a meaningful benefit (though it may be a good trade-off, for python specifically, as things are)
verbal escarp
#

even if there was something like "slow but efficient", if we can't improve python's efficiency, we might lose out on important future usecases like drones and other integrated systems that have to be very conservative

odd fractal
halcyon trail
#

something like a drone is hard realtime

#

generally, professionals (as opposed to hobbyists) aren't writing that kidn of stuff in any GC language

sly jungle
oblique iris
#

Bu ne ya

elder blade
halcyon trail
#

i know the GIL makes python faster in single threaded, i'm just saying that when you're comparing languages, the GIL is a downside. How fast the language is depends on many things, the GIL helps python but obviously not enough to make up for things weighing it down.

#

that's why I said "it may be a good trade-off for python specifically" - python in its current state, the GIL is probably a good thing (at least the GIL versus the ways in which GIL-less implementations were suggested in the past)

elder blade
# halcyon trail you're misunderstanding what I'm saying

Reading it again yeah I think I may have.

The GIL is something I won't recommend to other languages, and it's something I won't hate seeing go. That said it makes Python easier - for everybody - there are much less concurrency problems that people run into.

I guess I have mostly excluded threads in Python because it really doesn't do threads good.

white nexus
#

there is a pep to remove the GIL IIRC

halcyon trail
#

see, the concurrency argument I don't buy

white nexus
#

gonna read what the GIL does brb

halcyon trail
#

this came up recently and the thing with the GIL is that it's no different tha a program in any other language constrained to run on one core (i.e. no parallelism)

white nexus
halcyon trail
#

not accurate πŸ™‚

#

threads in python (w/ GIL) let you run code concurrently, but not in parallel

#

sometimes concurrency is all you need, sometimes you also need parallelism

white nexus
halcyon trail
#

that said, there isn't usually much difference between "concurrency-safety" and "parallel-safety" - it's all just thread safety. You still have to use thread safe data structures and locks in many cases in python.

#

(most cases)

#

i'm actually not too familiar with asyncio. I think asyncio separates the work that needs to be done, from where it gets scheduled

#

it could be scheduled on a thread pool, or in other ways

white nexus
#

in addition asyncio also switches contexts on await statements

halcyon trail
#

threads are not like that, each thread has a specific top level function its running

#

right, there's cooperative multi tasking there as well

white nexus
#

so it's guaranteed that a non coro function will not have any states change

peak spoke
#

with threads there's practically no safety and you have to lock things if they can be scheduled at the same time

halcyon trail
#

Right, yeah, so people say it's easier to program against.

#

right.

#

the point being that the GIL doesn't help with the safety of raw threads in any substantial way

#

and the GIL isn't necessary, fo course, to have a model like asyncio

#

many languages have it and almost none have the GIL

peak spoke
#

the implementation would have to guard against things like data races from parallel writes but I don't think that'd quite get to the language user

halcyon trail
#

so I don't think the GIL is beneficial in a vacuum in any way; it's just something that python seems to need to help address its performance issues. I'm not sure how much of that is a general trade-off between single and multi threaded and how much seems specific to cpython's implementation.

#

i assume the latter though because single threaded performance is mportant and I've never heard any language discussing adding a GIL

peak spoke
#

threads are just a pain to work with when there's shared mutable state compared to something that has explicit yield points so even if the GIL added some safety I wouldn't really say it's worth the other costs (if that was its only point)

halcyon trail
#

i mean sure but you talk about "explicit yield points" like they solve all problems

peak spoke
#

They make things quite a lot easier to wrap around

halcyon trail
#

for doing asynchronous IO, sure

peak spoke
#

concurrency is complex so there's no going around that, but when you know that things can't switch at some point it gets at least a bit easier

halcyon trail
#

if you're doing parallel processing then people don't generally do things that way

#

basically, if you are doing single-threaded, cooperative multitasking as a way to do asynchronous programming, sure, now you created a world where thread safety isn't really an issue because functions dont' get unexpectedly interrupted

#

that's a very very specific kind of programming though, it just ends up dominating a lot of conversations on the internet because it happens to be what a lot of webservers want

#

i guess what I'm saying is a language adding real safety to multithreaded programming would be very valuable, it can't be dismissed just because models like asyncio exists.

#

the GIL doesn't add real safety though

topaz patrol
#

It's think it's worth pointing out that multiprocessing avoids the GIL problem entirely.

halcyon trail
#

"works around" might be a better term, from my perspective πŸ˜› . it does help, and it does make solving many common problems reasonably easy

topaz patrol
#

You're not wrong.

halcyon trail
#

there are real issues with multprocessing that come up even in relatively simple programs

topaz patrol
#

Oh for sure.

halcyon trail
#

just the fact that you can't define a helper closure locally, to pass into a multiprocessing pool

topaz patrol
#

Shared state is just a nightmare

halcyon trail
#

that made me facepalm many times

#

shared mutable state :-). When you use a pool though it's pretty easy to avoid said state, whether it happens to be a process pool or a thread pool.

topaz patrol
#

Right.

halcyon trail
#

it's just that a thread pool also avoids many other headaches for you

#

I mean, actually using processes in most other languages is... pretty rare, i've only done it when I really needed to isolate a process completely from another, e.g. worried about my C++ binary segfaulting, so I launch the binary from a python script, which stays up, and ensures that it hasn't segfaulted. that sort of thing.

topaz patrol
#

I think we're going to see more and more work in this arena though. Especially with the multicore workloads proliferating. Thank/blame the data scientists and crypto nerds?

halcyon trail
#

it started long before those two buzzwords took off though

topaz patrol
#

The Berkeley folks called it a decade? Ago

halcyon trail
#

really as soon as they started not being able to keep cranking up single core speeds, they started adding more cores instead

topaz patrol
#

They said that cores were going to be the new transistor.

halcyon trail
#

seems like a bit of an exaggeration πŸ˜›

topaz patrol
#

In the sense that transistors were once expensive

#

And then became very very cheap

halcyon trail
#

I have seen architectures a bit more like that, there was thing this my company looked at a few years ago, it was like 200 very simple cores, along with a few normal cores, on one socket

#

i think it was called phi or something

topaz patrol
#

I mean that's basically what gpus are

elder blade
halcyon trail
#

well, all of data science, numerics, etc is a pretty big exception to the above

topaz patrol
#

And guess what all the data scientists are learning

halcyon trail
#

i agree that there is a lot software like that, I just meant it's still in the grand scheme of things, pretty specific, just hugely overrepresented in these online forums

paper echo
halcyon trail
#

there's lots of people writting embedded code, for cars, microwaves, device drivers, there's people writing financial software, people writing medical software, GPS, I could go on and on, and that stuff is basically never discussed, compared to how often people are like "okay, so your web server, right, it's fielding requests..."

elder blade
#

That said even asyncio programs sometimes need to do small (but still blocking) CPU-bound work - which is where I would use a threadpool (because that work is not cooperative) - and it would as well benefit from the removal of the GIL.

Here's the post that outlines the scheduling issues that CPython has though https://bugs.python.org/issue7946

halcyon trail
#

it's funny how when I started visiting places like proggit, etc, everyone used frontend dev, and backend dev, and I had no idea what they were talking about

#

even though I had been programming for a while

halcyon trail
#

realistically even if you're using pandas to process some data, you still want a process pool, not a thread pool

paper echo
#

i mean kinda? you can use openmp all wrapped inside the c code and not even touch the gil

halcyon trail
#

enough work happens in python

elder blade
halcyon trail
paper echo
#

i'd guess that web servers make up less than 30% of python code that's deployed in the world

halcyon trail
#

but that's not usually the point

paper echo
#

but why does it have to be C? why not go or nim?

#

those are way way faster than python and still a lot easier to use

#

yes nim compiles to C but that's not the point

white nexus
halcyon trail
#

nim is a meme language. And it's about the ecosystem of packages for data science applications. And the console, as well.

paper echo
#

managed memory languages are so much easier to deal with for a lot of / most use cases

spice pecan
#

that's usually offloaded to gunicorn/uvicorn/etc

halcyon trail
#

but in any case I don't really understand what you're suggesting

#

my point is just that when I wrote some python data scienc-y code that I need to parallelize locally, I still need to use multiprocessing, that's all

paper echo
#

right

#

(but joblib is probably easier to use than multiprocessing and it also memmaps and caches numpy arrays)

halcyon trail
#

the things we do locally are pretty simple

#

for bigger stuff it's being parallelized on a compute grid so we need to use the APIs provided for those

paper echo
#

my suggestion overall is that there should be, and are, intermediate solutions between "use python multiprocessing" and "use C++"

halcyon trail
#

well, the actual solution is just a faster language πŸ™‚

paper echo
#

...like nim

#

or clojure

topaz patrol
#

The functional programmers are pumped about parallelism.

halcyon trail
#

more likely julia, tbh

#

but it's not there yet either

#

clojure as a language for that niche doesn't really make sense

#

nim is statically typed, i think that interactive analysis is probably one of the few niches where dynamic typing seems to be helpful.

verbal escarp
#

i think we're too stuck in cpython land

#

how about cython?

#

prototype stuff with cpython, switch to cython once you got your types all set up

boreal umbra
#

I'm doing a web development thing, and the API has a lot of constructions like this:

    return (
        handler_input.response_builder
            .speak("This is the first response")
            .ask("add a reprompt if you want to keep the session open for the user to respond")
            .response
    )

I don't even know what types the intermediary objects are (those returned by .speak(...) and .ask(...). It might even be that these are each mutator methods that return self. In either case, is this design pattern JS-inspired? It seems like it might be a reflection of keyword arguments not being a language feature.

#

(I write similar-looking statements with Pandas, but that's because I'm just chaining a bunch of operations.)

feral cedar
#

idk where it's from, it shows up a lot in java also

feral cedar
#

it's in Go4 iirc

halcyon trail
#

it's called the builder pattern or sometimes fluent interfaces

#

assuming it is returning self each time

grave jolt
halcyon trail
#

really almost every language

grave jolt
#

yeah

halcyon trail
#

keyword arguments are a really rare feature

#

python and kotlin, and almost no other mainstream language really

#

i think clojure kind of has them

grave jolt
#

To be honest, keyed arguments do create some issues. For example, ParamSpec and friends are pretty much useless because it would be messy to support it with kwargs. And look at how well TypeScript handles signature transformations.

halcyon trail
#

what kind of issues, sorry? Not too familiar with paramespec

grave jolt
#

it's a typing thing

flat gazelle
#

ocaml handles keyword arguments and static typing quite well tbh

halcyon trail
#

so in kotlin, typically, keyword arguments are mostly outside of the type system

#

if you call a function directly, you can pass its keyword arguments, if you pass a callable into a function, you don't

flat gazelle
#

but it has specifically keyword and positional arguments as separate constructrs

halcyon trail
#

which is a simplification but I think is perfectly fine

grave jolt
# grave jolt it's a typing thing

For example, you might want to transform a function with a callback into an async function: (*args, (T) -> None) -> None => (*args) -> Awaitable[T]. You can use ParamSpec for that.

halcyon trail
#

Sure.

white nexus
grave jolt
#

...or you can't. I don't even remember.

halcyon trail
#

I can see how that's an issue, but in practical terms, if you have the type system simply "give up" on that

boreal umbra
halcyon trail
#

you still get like 90% of the benefit of keyword arguments, with almost no effort

grave jolt
#

yeah, they're pretty nice

white nexus
halcyon trail
#

usually when there's a callback it's not a great idea to force users to adhere to a particular set of keyword arguments anyhow, IMHO

#

if you really have enough arguments, or enoughc omplexity in defaulting behavior, etc, that you want to do this

#

I would just have a callback that accepts a dataclass instead

flat gazelle
#

ye, in python it's kind of painful since the default is both keyword and positional, which, while it is quite useful for letting people call functions nicely, makes the types quite annoying

#

for example, a subclass overriding a method and changing argument names is a violation of the LSP

halcyon trail
#

well, typically what you'd do in practice when a function accepts a callble, is to take... Callable πŸ™‚ Which amounts to a promise that you will only call using pisitonal arguments

grave jolt
#

you also need to be careful with __call__able Protocols

#

that is, use / in __call__ if you don't need the keywordness of the arguments

#

or prefix arguments with __ if your version of python doesn't have /

spiral lily
halcyon trail
#

i agree that you're technically correct, I just don't think it's a big issue in practice though

#

which is why you don't typically see the / suggestion in best practices books

grave jolt
#

I think keyword arguments are good for normal function calls

#

I don't know, maybe the lack of type system support is Python's fault

flat gazelle
#

it is a dynamically typed language and a lot of the design decisions are made with that in mind

halcyon trail
#

so was just refreshing. basically, in Kotlin you can only pass functions, into functions, as a positional-argument type

#

so you just can't make any keyword calls in that case. When you override things, only the positional arguments matter

#

if the overriding doesn't follow the positional arguments it's just a compiler error

grave jolt
#

that makes sense

halcyon trail
#

and when you make the call, you can use keyword arguments, but they always follow the "static" function signature

grave jolt
#

language aside, some IDEs/plugins show inlay hints with argument names when you call a functions

#

doesn't work for people reading via notepad/vim/github, but still nice

halcyon trail
#

yeah, jetbrains ide's do that for languages like C++ that don't have it

#

(as well)

lusty scroll
#

which I have had to do in Java at least, and probably C# (iirc c# makes you write all argument names or none of them, and changibg the ordering/defaultedness of functions can break compatibility, whereas with keyword-only arguments you can safely evolve the signature of your function)

halcyon trail
#

yeah, I agree it's worth it. I think the follow up question is just how it's handled in the type system, and then you have a bunch of answers, all of which have significant disadvantages

lusty scroll
#

yeah

swift imp
#

Python devs need to make help more type annotation friendly. When you have a function with a long signature and then add type hints on top the docs produced by help are hard to read. It's almost like they need to run black over it

naive saddle
#

!eval import black

fallen slateBOT
#

@naive saddle :x: Your eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | ModuleNotFoundError: No module named 'black'
naive saddle
#

Alright black is not usable as an eval-able example :p

boreal umbra
#

perhaps we should add this? What do you think, @odd flax?

#

that was supposed to be a ping for Kubernetes Man PeepoShrug

naive saddle
#

Eh, probably not that useful right now as black has no official API

#

maybe this?

#

This is from help(black.main.callback) (click wraps the main function)

peak spoke
#

to be fair you also don't want 30 lines in the help only for the signature

boreal umbra
#

<@&797752289771126784>, we need black to be in the bot's snekbox.

civic otter
#

Hi everyone. I’m trying to solve with the best solution to this problem. I want to check running status of a service in three networked windows 10 machine. This three machine like this. (a,b,c).
Computer a is master and service at first running only this machine and b, c computer are slave that only use when machine a service was down or stopped. I develop a python program that had multiple socket connection to the machine but that wasn’t good and optimal. We want all machines can see and check the service if one of them stopped the service other only one machine starting that service.
Who can help me to solve this? Thanks a lot.

white nexus
#

I actually

#

I meant to do that for some other packages

#

!e import arrow; print(arrow.version)

fallen slateBOT
#

@white nexus :white_check_mark: Your eval job has completed with return code 0.

0.17.0
white nexus
#

!pip arrow

fallen slateBOT
white nexus
#

so old

cobalt nacelle
#

Hi all! I hope this is the correct channel to ask this, if not let me know. Does anyone know why this is not correct in python?:

class TestClass():
    def test_method( test_dict: typing.Dict[str, TestClass]):
        print("test")

I get an error that TestClass is not defined. How can I give a type hint that test_dict is str -> TestClass?

astral gazelle
#

"TestClass" mayhap? Wrap it in quotes

cobalt nacelle
#

Ohh, that seems to work, thanks! What is the reason for having to put it in quotes to get it to pass?

white nexus
cobalt nacelle
#

Ok, i see. Thanks a lot!

amber nexus
#

@still fox Please stop randomly spamming this

still fox
#

hmm

#

no dde

white nexus
#

πŸ‘€ what is the best way to mutate a tuple?
convert it to a list and then change the position I want to change, then back to a tuple?

still fox
#

ok alr

white nexus
#

(its the args passed to an exception which I need to change, so not using a tuple from the start is not an option)

spice pecan
#

Pretty much

#

You could also not convert back to a tuple if there's no need

elder blade
white nexus
#

slicing the tuple around the replaced part, or converting to list and then back

spice pecan
#

i highly doubt slicing and concatenating will be more efficient

gusty marsh
#
In [1]: def via_index(tup, index, val):
   ...:     return tup[:index] + (val,) + tup[index+1:]
   ...: 

In [2]: def via_list(tup, index, val):
   ...:     l = list(tup)
   ...:     l[index] = val
   ...:     return tuple(l)
   ...: 

In [3]: t = tuple(range(1000000))

In [4]: %timeit via_list(t, 500000, 99)
149 ms Β± 20.7 ms per loop (mean Β± std. dev. of 7 runs, 10 loops each)

In [5]: %timeit via_index(t, 500000, 99)
124 ms Β± 23.3 ms per loop (mean Β± std. dev. of 7 runs, 10 loops each)
#

not much difference

spice pecan
#

yeah, they're pretty similar for a single consecutive group

gusty marsh
#
In [8]: %timeit via_list(t, 5, 99)
391 ns Β± 27 ns per loop (mean Β± std. dev. of 7 runs, 1000000 loops each)

In [9]: %timeit via_index(t, 5, 99)
524 ns Β± 17.3 ns per loop (mean Β± std. dev. of 7 runs, 1000000 loops each)
``` For smaller tuples, `index` is taking less time, interesting (range is `10`)
neon troutBOT
white nexus
#

Note
It is possible (but not likely) that the parser stops parsing with a successful outcome before reaching the end of the source; in this case, trailing symbols may be ignored instead of causing an error. For example, a backslash followed by two newlines may be followed by arbitrary garbage. This will be fixed once the API for the parser is better.

#

very nice, python

paper echo
fallen slateBOT
#

lib-python/2.7/unittest/test/test_program.py line 58

assert True```
`testrunner/test/examples/normal/failingsetup.py` line 6
```py
assert True```
`rpython/jit/backend/arm/test/test_regalloc.py` line 688
```py
assert 1```
charred wagon
#

Testing that the assert statement itself works?

#

That comment above the last one implies that

white nexus
#

(yes I know there's a flag for that)

charred wagon
#

I don't think checking if it's in optimised mode is the goal of these tests

white nexus
#

edited

charred wagon
#

Looking at it again, the first test is a test for the unittest library. The asserts seems to behave as they normally would - no special pypy behaviour. They're just there so that the tests have both a passing and a failing result to work with. A bit further down there are some tests that unittest will have results or it will raise SystemExit.

#

Same thing with the second one. It's a test for the test runner and it needs a passing test case. I guess assert True could have just been pass but maybe the former is more clear about its intent?

grave jolt
#

for me assert True would be more puzzling than intention-revealing

charred wagon
#

Well, it's a test case. What do tests do? They assert things. This test needs to ultimately pass. How do tests pass? When assertions are true. What's the most trivial way to get a true assertion? assert True

grave jolt
#

"all assertions are true" is also satisfied if the set of assertions is empty

charred wagon
#

Technically correct, but that is less evident than assert True

grave jolt
#

well I guess that's subjective

terse yacht
#

hello

unkempt rock
#

Is it a bad idea to have types as dict keys? {int: 1, float: 2, MyClass: 3} pithink

#

I've never thought of it before but seems to work fine, just feels weird

grave jolt
unkempt rock
#

just as a little look up table instead of doing bulky if type checks and individual variables per type

spice pecan
#

you do give up inheritance

#

but other than that it's fairly okay imo

paper echo
#

i guess you can use match now too

grave jolt
unkempt rock
#

I know, but luckily that shouldn't be a problem

grave jolt
#

if you don't have inheritance, then yes, it's fine πŸ™‚

paper echo
#

wow, fable compiles to python now

#

πŸ‘€ πŸ‘€

#

that is extremely interesting to me (and RIP coconut)

spice pecan
#

looking it up in a dict won't match

#

whereas you would match with an is instance check

grave jolt
#

which is different from an if-elif chain with isinstance

native flame
#

ah that

white nexus
#

a or b
a: obj.__class__
b: type(obj)

c or d
c: obj.__class__.__name__
d: type(obj).__name__

white nexus
#

i think a and c

quick snow
#

b and d, a and c could theoretically lie

#

!e

class Foo:
    __class__ = 42

f = Foo()
print(f.__class__)
print(type(f))
try:
    print(f.__class__.__name__)
except AttributeError:
    print("error")
print(type(f).__name__)
fallen slateBOT
#

@quick snow :white_check_mark: Your eval job has completed with return code 0.

001 | 42
002 | <class '__main__.Foo'>
003 | error
004 | Foo
white nexus
quick snow
white nexus
#

ohhh, type is implemented in c

paper echo
#

Otherwise i use type()

white nexus
elder blade
peak spoke
#

There's some code you include to make it a namespace package, not having to do that is the reason for the implicit namespace packages

elder blade
#

Ah, right

raven ridge
#

Type 2 and type 3 namespace packages need the __init__.py, type 1 don't.

#

Just the perils of trying to describe all 3 on one page

elder blade
#

Ah lol

#

Why would I want to use those if I still can't import anything?

raven ridge
#

I don't understand the question... You can import things from namespace packages. There's no good reason to use the type 2 or type 3 ones besides backwards compatibility with an existing namespace, though

elder blade
raven ridge
# elder blade As in, importing things inside the `__init__.py`

The entire point of a namespace package is to serve as a namespace under which other packages can be placed. If you want imports in the __init__.py, then what you want isn't a namespace package, because it doesn't make sense to import things from other packages (that may or may not be installed)

#

you need to choose between making a regular package (which exists in only a single directory on disk, comes from a single pypi package, but allows you to fully control what's exposed in it) or a namespace package (which can exist in multiple directories on disk, can come from multiple pypi packages, but whose contents are based on which packages are installed underneath it)

elder blade
#

Yeah that's true, I was thinking of essentially revealing some things under that namespace as an extra shortcut but yeah I'll just do as you're supposed to lol πŸ˜…

unkempt rock
elder blade
#

Yeah?

#

5 * 10⁹ is 5 followed by 9 0s.

unkempt rock
elder blade
#

Yeah that's JavaScript for ya' πŸ˜” πŸ˜” πŸ˜” πŸ˜”

lament sinew
#

that's why you use === instead of ==

flat gazelle
#

!ban @dense rune 14d seems like you are just here to advertise, should you come back, please read our rules.

fallen slateBOT
#

:incoming_envelope: :ok_hand: applied ban to @dense rune until <t:1639138301:f> (13 days and 23 hours).

surreal sun
#

Any news on PEP 679?

#

haven't heard much about it in a while tbh

white nexus
#

!pep 679

fallen slateBOT
#
PEP not found

PEP 679 does not exist.

white nexus
#

@surreal sun ^ what pep?

surreal sun
#

wait what, that's weird

#

maybe it's another number, but it's the late bound arguments PEP

spice pecan
#

!pep 649

fallen slateBOT
#
**PEP 649 - Deferred Evaluation Of Annotations Using Descriptors**
Status

Draft

Created

11-Jan-2021

Type

Standards Track

spice pecan
#

Wait no

#

!pep 671

fallen slateBOT
#
**PEP 671 - Syntax for late-bound function argument defaults**
Status

Draft

Python-Version

3.11

Created

24-Oct-2021

Type

Standards Track

spice pecan
#

That's the one

surreal sun
#

That's it

white nexus
spice pecan
#

Usually, default arguments are evaluated at function-creation time, which is good for optimization purposes, but can have somewhat surprising effects for mutable arguments

#

!e ```py
def f(l=[]):
l.append(1)
print(l)

f()
f()
f()

fallen slateBOT
#

@spice pecan :white_check_mark: Your eval job has completed with return code 0.

001 | [1]
002 | [1, 1]
003 | [1, 1, 1]
white nexus
#

ah yes, where bugbear has saved my ass multiple times

spice pecan
#

In order to bypass this, either None or a custom sentinel is used for the default argument, and then actual initialization happens in the function body

#

The pep proposes new syntax, which would reduce boilerplate and evaluate the arguments at call time

#

So with the PEP implemented, this would work:

def f(l=>[]):
    l.append(1)
    print(l)

f()
>>> [1]
f()
>>> [1]
f()
>>> [1]
white nexus
#

hmm

#

stupid annotations

#

I actually think with the walrus operator, def bisect(a, hi:=len(a)): makes the most sense

#

except for annotations...

#

since that would mean it would look like def bisect(a, hi: int := len(a)):

spice pecan
#

Because [] is evaluated again every time you call the function without providing l. This would also allow making default values dependent on each other, like this:

def f(seq, l=>len(seq)): pass
spice pecan
white nexus
#

although.... def bisect(a, hi: int => len(a)) -> int:

spice pecan
#

I'm not actually sure which option would be optimal for improving readability, reducing ambiguity and keeping visual noise to a minimum

#

But in general, I like the idea

white nexus
#

solution: rewrite all annotations and everything and release python 4.0 /s

spice pecan
#

It would allow for better self-documentation on function signatures and reduce common boilerplate

spice pecan
white nexus
#

I actually like this one
def bisect(a, hi=@len(a)):

#

the @ makes it seem more like a decorator which mutates the function, and that's (kind of) what this does...

#

hmmm

spice pecan
#

noisy tho

white nexus
#

this would be a nice thing too, a good singleton

#
def add_item(item, target=>[]):

# Equivalent pseudocode:
def add_item(item, target=<OPTIONAL>):
    if target was omitted: target = []
#

something like typing.OPTIONAL which is a singleton and could be used for a default value instead of None

spice pecan
#

I think I like ? from a semantic standpoint, but it's visually loud

white nexus
#

that way there's a standard for that

spice pecan
#

I think there's another pep for better sentinels

#

It should be mentioned in this one, maybe in the references section?

#

!pep 661 I think

fallen slateBOT
#
**PEP 661 - Sentinel Values**
Status

Draft

Created

06-Jun-2021

Type

Standards Track

white nexus
#

def bisect(a, hi?=len(a)): ...
def bisect(a, hi=?len(a)): ...

def bisect(a: str, hi: int ?= len(a)) -> str: ...
def bisect(a: str, hi int =? len(a)) -> str: ...

#

huh

#

This is not the original intended use of Ellipsis, though it has become increasingly common to use it to define empty class or function blocks instead of using pass.

#

so what is the intended use?

flat gazelle
#

I think it was meant for numpy etc, just like @

white nexus
#

?

#

wdym

#

what is @ for?

flat gazelle
#

@ is the matmul operator

white nexus
flat gazelle
#

it does make sense to provide features for libraries to take advantage of, even if the stdlib doesn't need them

white nexus
#

true

astral gazelle
#

Why would you not help your premiere 3rd party library be easier to use

white nexus
#

true

fluid lake
#

Is github a good place to just look through projects and study what they do? Or is there a better place for that.

I just wanna see the programs people have made and study them so I can use those techniques.

grave jolt
white nexus
#

90% of github projects aren't going to be very good tho

grave jolt
#

I mean, yeah

white nexus
#

most of them are beginners starting out, or abandoned projects from years past

grave jolt
#

you shouldn't be reading random stuff people wrote