eh, it's not stupidity - it's just a feature that you never needed and didn't need to learn about. It's a feature that's used much more in enterprise Python than open source Python, because enterprise stuff needs to pick names for everything that are guaranteed not to cause problems by conflicting with (current or future) PyPI packages
#internals-and-peps
1 messages Β· Page 149 of 1
if you care about security shouldn't you be using a pypi mirror anyway? or is it about the case where you'd want to use a library that has the same name from pypi?
yep, the latter.
our PyPI mirror overlays our stuff on top of the OSS stuff, but obviously it's a problem if we were to want to depend on a public library that has the same name as one of our private libraries
which is bound to happen relatively often, if you have a few thousand Python developers in the company and let them choose their own names without any guidance.
we have a completely air tight pypi π
also YIL that you can just go to https://pypi.org/simple/ and see all the packages
and the reason why implicit namespace packages are better than the old, explicit ones was the problem of who was responsible for creating the magic __init__.py file for the old, explicit ones. In practice, in order to package them for systems like Debian, you needed to create a package for each namespace package that contained only the __init__.py, so in my example above you'd have 3 explicit namespace packages, one containing each of py somecompany/__init__.py somecompany/team1/__init__.py somecompany/team2/__init__.py and the somecompany.team1 and somecompany.team2 packages would need to declare a dependency on somecompany, and the somecompany.team1.package_a and somecompany.team1.package_b packages would need to declare a dependency on somecompany.team1, and the somecompany.team2.package_c package would need to declare a dependency on the somecompany.team2 package, and the somecompany.team[12].package_[abc] packages would need to explicitly exclude somecompany/__init__.py and somecompany/team[12]/__init__.py from their sdist.
which was an absolute nightmare to explain to people, and to get right - and when someone got it wrong, the most likely result was that it would just break the ability to find the package across multiple directories on disk, and things that are installed would just stop being importable when you installed the new, incorrect package (though importing the new, incorrect package itself might work!)
so: PEP 420 made it much simpler to create and maintain namespace packages. The biggest downside to PEP 420 is that it also made it possible to accidentally create namespace packages - in fact, it's now easier to create namespace packages than the regular packages that almost everyone actually wants
yes, that's what I was generally confused about. I was under the impression that it's just some new way of not having to have __init__.py if you just want to make a structure
nope. Different type of package, for an entirely different purpose. Lots of people's takeaway was "Now I don't need an __init__.py", but that's not right - namespace packages aren't a substitute for regular packages, and some feature that work for regular packages - like, say, importlib.resources - don't work for namespace packages.
spread the word π
the import system in Python is really confusing
it seems to be really customizable for some reason
yeah, it's very complicated
if you want a unique name, can't you just name it acme_string_utils instead of acme.string_utils?
sure, at the cost of not being able to use relative imports, and needing to use import acme_string_utils as string_utils instead of from acme import string_utils, and so on.
You end up with smurf naming where 'acme' is repeated everywhere.
"smurf naming" π Love it.
well... that's just unavoidable, right? if you want to depend on that library
And also acme itself is also a module, so you can import that.
and what if you need a PyPI library called acme? π
it's quite nice, in an enterprise world, that if company.team.app wants to depend on company.team.library, it can do from .library import stuff instead of from company_team_library import stuff
I guess
If two packages exist with similar apis, it's possible you could import one, dynamically switch.
companies can force a takedown on PyPI over trademark violations.
but is the nicity worth the complication?
Oh, that's another reason - you can vendor a library under say yourlib.thirdparty.whatever, and it'll be contained there and not affect the toplevel.
Stuff like pip does that for instance.
I can't imagine the nightmare of needing to deal with a project where every filename had the company and team and project name embedded as part of the filename.
Is there a mechanism like this in other languages?
How do they solve it?
most enterprise software is probably not written in Python
I honestly would be okay with ```py
from acme_string_utils import left_pad
maybe I'm just weird
er - hm? That's what we have today, with namespace packages, right?
as a user, that's not much different, I guess - but it makes a big difference to repo layout. Instead of py acme/ sales/ string_utils.py you wind up with ```py
acme_sales_string_utils.py
But wasn't placing it across different directories, not in a single tree, the whole point of namespace packages?
or maybe I misunderstand again
I'm talking about the experience for the maintainer of the acme.sales.string_utils library
that repo needs to either contain acme/sales/string_utils.py (the namespace package route) or acme_sales_string_utils.py (your fake namespace proposal)
the latter is less nice to work with, in practice.
Why? Just the name being longer?
well, maybe I just lack the 'practice' part
I'll come back when I encounter a situation like this π
yeah, the name being longer, and harder to tab complete, and harder to recognize at a glance. It's just less convenient to deal with a file name that's 50 characters long than one that's 15 characters long, in a lot of tools - like, fuzzy-finding files in an IDE will be harder if every file contains a common prefix that your pattern will wind up matching against
Hol' up.. how does someone accidentally do that?
a package without an __init__.py is a namespace package. Normally you don't want a namespace package - they're a niche feature. But by forgetting one file, you can accidentally make the wrong kind of package.
So discord.ext is a namespace package?
what do you see if you:
import discord.ext
print(discord.ext)
And that's why you can install other stuff and import it using discord.ext.xyz?
Hold on
<module 'discord.ext' (namespace)>
Interesting
but why can't it be a normal one?
it probably could
because multiple different packages want to install stuff under it
right
I think I'll never understand namespace packages
I'm giving up
I'll go write Rust
just never forget __init__.py and you're fine, most of the time
you'll know it when you need it
This causes some issues imo
if you have a directory where you put your projects/repos, and the subdirectories have, that are the same as a namespace package names (an existing one installed elsewhere), you cannot run python from the directory and import a module having the same name as a project directory, without messing with PYTHONPATH ?
it's like the directory version of "Don't name a file io.py"
my workaround is to juat name the directory something else. but it seems like __init__.py were still required, then python wouldn't see those directories. I am aware ut will prefer protobuf.py over a folder named protobuf, though, which is good
well, my workaround for future projects will be to take justuse and use(Path(..)) stuff π
Guys I have a question and its:
how to link python code to GUI
For exempel add URL to a button
hi, this is not a help channel. perhaps ask either in #python-discussion or claim a help channel and ask. see #βο½how-to-get-help
I am doing a node graph where all the node inputs (input slots) are procedurally generated and filled based on the nodes execute() functions parameters and return type; using typing and __annotations__. I want to reload that execute function in a reload function on the node so that the user doesn't have to restart the application every time a node's execute() function has been changed (as in the source code of the node subclass).
The code is going to be used in a pipeline where the developers are actively developing the nodes while using the application, meaning that having to restart the entire application for the execute function to to reload on the node that has is currently being worked on is a MAJOR downtime.
There are two possible interface solutions that could work, ether the user reloads the entire active graph; which i want to avoid if possible. The other option would be to selectively update the execute function on the node(via. the user right click->refresh from source).
But this is where it gets a bit complicated, the logic i ideally would want would be something like this:
newModule = importlib.__import__("userNodesModule")
classToUpdate.execute = newModule[classToUpdateName].execute
But this has one fatal floor, now if i try to re-import this "userNodesModule" again, the base module is changed:
>>> import importlib
>>> thisModule = importlib.__import__("math")
>>> def dummyPrint():
... print("Hello World")
...
>>> thisModule.acos = dummyPrint
>>> thisModule.acos()
Hello World
>>> otherModule = importlib.__import__("math")
>>> otherModule.acos()
Hello World
I want to update just one function in the current runtime without affecting the other parts of that modules, since that would almost certainly break stuff with active references.
Sudo code for the node base class:
class BaseNode(SomeGuiBaseClass):
def reload(self):
# Reload the execute function - this is what i can't figure out how to do.
# Rebuild node input parameters from self.execute.__annotations__.
def executeImplicit(self):
# This is the only place execute() will ever get called from.
# The args variable is pieced from the node inputs and verified against self.execute.__annotations__.
args = [...]
execute(*args)
def execute(self, string: str, integer: int):
# The definition and implementation of execute is overwritten in all inheriting classes.
A working example of the "front-end", user faceing side of the code.
class StringConstantNode(KSNodeItem):
def execute(self) -> str:
return "String123"
class PrintString(KSNodeItem):
def execute(self, string: str, boolean: bool) -> None:
print(string)
nodeGraph = KSNodeGraph(None)
nodeGraph.addNodeType(StringConstantNode)
nodeGraph.addNodeType(PrintString)
sounds cool, it's like an interpreter? you need some scopes to execute in, right?
Could you clarify what you mean when saying "you need some scopes to execute in"?
well, in justuse we can do automatic reloading if a file has changed since last loading, but it only works without problems if it's a function-only module, so my advice would be to only try to reload function-only modules
and don't reference those functions other than via module-attribute lookup
just return a proxy from attribute lookup on your module object and then update all proxies when a module is reloaded /s
Though that wouldn't preserve state, just handle assigned objects
object proxies? hmm
interesting idea, but it's problematic if object instantiation has side-effects
It's incredibly problematic and still leaves a lot of things untreated, such as instances of reloaded classes still referencing the old class
You could maybe remedy the instances with a proxy metaclass, but something tells me it's going way too far
hmm
in justuse we return a ProxyModule by default, so we could keep track of object instantiation.. if we'd keep track of those references via gc, we could replace those objects' class by the reloaded one behind the scene
it's really.. no idea what it is
i'm not even sure if it could work
You could replace types with your own proxy that would store weakrefs (so as to not mess with gc) to instantiated objects and then reassign their class on reload
But again, it's not exactly something I'd use in a real project, more of an interesting idea that I'd love to tinker with at some point
you're welcome to try and come up with a prototype, we could put it in justuse as an experimental feature (with a big fat warning)
i'm fairly happy with the function-only reloading so far, and i do check if a module contains anything other than functions to give a warning.. so we could put it there
!ban 754353181227745440 14d seems like you're only here to drop some random link. Please re-read our #rules.
:incoming_envelope: :ok_hand: applied ban to @short marsh until <t:1638634384:f> (13 days and 23 hours).
what I mean is, you need locals and globals for the exec to work unless it doesn't reference anything except builtins, right?
e.g. a module dict
I see, thx all!
!e
ah, logic
print((True + True ) * (False - True) * (False - True) ** (True + True))
@white nexus :white_check_mark: Your eval job has completed with return code 0.
-2
!e print(~True)
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
-2
luckily there's no hashing happening ?
wait what does ~ do and where is documented along with the other weird things I've seen like |=
it's bitwise not, basically inverts all the bits. for numbers, it's equivalent to -x - 1
In Python, it does -x-1
oh neat
@unkempt rock :white_check_mark: Your eval job has completed with return code 0.
001 | -5: -5
002 | -4: -4
003 | -3: -3
004 | -2: -2
005 | -1: -2
006 | 0: 0
007 | 1: 1
008 | 2: 2
009 | 3: 3
010 | 4: 4
011 | 5: 5
... (truncated - too many lines)
Full output: https://paste.pythondiscord.com/yerowujeyi.txt?noredirect
it's because -1 is used a "flag" by the hash function for something
it's... weird. Normally it only makes sense to do bitwise operations on unsigned numbers, so it's weird to see that Python picks an implementation of it that's entirely based on signed integers.
well, depends on what you mean by "correct"
!e print(bin(5))
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
0b101
if Python says that the bits of 5 are "101", then the bits of ~5 should be 010
but they're not:
!e print(bin(~5))
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
-0b110
yes, there is some definition for what the bitwise operation does by which what Python does is reasonable.
but it's not what people coming from other programming languages are likely to expect.
it's made more complicated by the fact that Python doesn't have both signed and unsigned integers, of course.
@white nexus :white_check_mark: Your eval job has completed with return code 0.
0b110
Lol really?
for ints, at least - yeah
C and Python show the same thing for ~6: -7
yeah, fair enough - I more meant that it's weird to do bitwise operations on signed ints in the first place, and ~6U and ~6 are different in C.
Is it just me or is Counter a bit hard to use
every time I use it I end up doing it wrong or else I have to look it up, despite using it occasionally for years now
I did find it a bit awkward to use the last time I needed it for something
how so?
why is there no method to add/insert an item ?
c = Counter(); c[new_thing] += 1 ?
so it's like a dict that that stores counts instead of values?
yes
it's just a bag
is there any other language that does if expressions like Python (true expression first)
what do you mean by true expression first?
i think they mean the ternary thing, true if cond else false
Yeah, JavaScript does cond ? true : false
lua seems to do if <condition> then <true expr> else <false expr>
I assume they mean <true expr> ... <condition> ... <false expr> because the former is pretty common? @gleaming rover
haskell does that too
ah, ok
yeah, that specific syntactic form
most functional languages that I know do if <expression> then <true expression> else <false expression>
it seems to be called Luau?
it has gradual typing.. thinking if you'd say Python has gradual typing
ah, cool
the type annotations remind me of python's but they have type? for optionals and I assume other differences as well (() instead of NoReturn)
Luau is like typescript for lua
Not the first of its kind but it has a big company behind it so that means clout and adoption
Also null/nil in lua is weird
It's like javascript but they use nil for both null and undefined
Which can be kind of chaotic and it makes handling "missing data" difficult
Not to mention that "arrays" are just tables with integer indexes and they are also kinda sorta nil-terminated
Great embedded scripting language though
Is there a difference/preference between ```py
class A:
def iter(self):
return iter(self.listofstuff)
vs
class B:
def iter(self):
yield from self.listofstuff
I guess A makes an actual iterator so it feels more correct 
A is definitely better because py type(iter(B())) # --> generator is pretty weird
IMO, idk if there's a standard for this
The reason I'd suggest the first is that it's probably a bit more efficient, since you don't have a generator in the way every iteration.
wdym?
Well iter(A()) returns the list-iterator, which is written in C. Then no Python code is executed to loop that after then.
For B, each next() has to go into the generator, then go to the iterator.
The second creates a new intermediate layer that will slow it down
well, let's see
Though there's the lookup of the iter() call, A might be slower for small lists perhaps.
if i continuosly refresh a website, will the owner notice ?
and will he ban me ? (im doing it with selenium, and running the program pretty much 24/7)
Yes, the owner definitely will notice, and no you shouldn't do that.
probably not because of caching
if there's a cache somewhere along the way, you'll just hit that
If you're using selenium to automate accessing a website and are afraid that the owner will find out, that sure sounds like a violation of rule 5.
no, im trying to scrape data real time
reloading a website shouldn't be a problem, as long as it doesn't result in a ddos
!rule 5
5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.
Whether it results in a DOS is immaterial. The question is whether the website's terms of service allow that access
If you're worried about the server owner finding out, it sure sounds like they don't.
basically i need to access data in the table and get it asap (like every second matters)
i wonder if refreshing the website will make it faster
and let me guess, an API key would cost you money
im trying to think if it takes alot of bandwith or stuff like that, otherwise i dont really care, as long as im not harming fellow users
If the site doesn't want you refreshing it in a loop, we won't help you with that here.
there isnt, im alredy paying like 50 a month for the scanner
for the scanner?
ye
what scanner?
its a stock screener, it updates every second, and if my selenium script will find an alert that qualifies certain parameters, it will scrape that
is the scanner part of the website?
lol this might be too cursed, but how can I dynamically generate a function signature given the signature of a different function?
so does it take significantly more bandwith if i continously refresh it, than just staying connected ?
.... call it within the function and use *args **kwargs thanks lol I just answered my own cursed non pythonic idea
Yes, someone is serving those requests, and it's costing them bandwidth. And most sites have terms of service that explicitly forbid scraping and automation.
it might be pretty straightforward actually, just check inspect, get the signature?
i am too tired even, since re-reading that question is not what I meant to ask
provided one function definition, how to dynamically generate a new function that has an identical signature?
Eg, if we were to have sys.exit which takes just a code, to replace this with a fake exit method with an identical signature that does other stuff inside
i see
you mean like a decorator? π
I think the only way to do that, is to define a function that takes *args and **kwargs then inspect what was passed to be within the result of inspect.signature() of the replacing method
maybe? it would be like completely replacing a method but still validating everything does what it should do
using those types is a good idea for replacing callables
i used those for aspectizing of descriptors and bound methods, was very simple that way
neat
I think functools.wraps was made for this exact purpose, to preserve signature, docstrings and so forth
Use *args, **kwargs to pass params and let wraps handle the rest
oh neat
iirc wraps didn't solve some problems i had with bound methods, but i can't recall the details
I think you are looking for @makefun.wraps, its like functool.wraps but relies "properly" on the dynamically generated function
https://smarie.github.io/python-makefun/#signature-preserving-function-wrappers
>>> assert enhanced_foo(1, 2) == 3 # positional 'b'
hello!
b=2
>>> assert enhanced_foo(b=0, a=1) == 1 # keyword 'b'
hello!
b=0
>>> assert enhanced_foo(1) == 2 # default 'b'
hello!
b=1
uhhhh
that looks slightly problematic
given this code: ```py
our signature-preserving wrapper
@wraps(foo)
def enhanced_foo(*args, **kwargs):
print('hello!')
print('b=%s' % kwargs['b']) # we can reliably access 'b'
return foo(*args, **kwargs)
it seems like it would convert the *args to **kwargs
!d operator.attrgetter
operator.attrgetter(attr)``````py
operator.attrgetter(*attrs)```
Return a callable object that fetches *attr* from its operand. If more than one attribute is requested, returns a tuple of attributes. The attribute names can also contain dots. For example:
β’ After `f = attrgetter('name')`, the call `f(b)` returns `b.name`.
β’ After `f = attrgetter('name', 'date')`, the call `f(b)` returns `(b.name, b.date)`.
β’ After `f = attrgetter('name.first', 'name.last')`, the call `f(b)` returns `(b.name.first, b.name.last)`.
Equivalent to:
so this is interesting.
most importantly:
The attribute names can also contain dots
so its a recursive getter and setter
how is it a setter?
er
*just getter, but it can be used for setting too since its recursive. eg get the parent of an object, and then set it last. A little more complicated but not by much.
I actually didn't know you could use dotted names with attrgetter
i still use the 3rd party library glom for this
!glom
When adding functions or classes to a program, it can be tempting to reference inaccessible variables by declaring them as global. Doing this can result in code that is harder to read, debug and test. Instead of using globals, pass variables or objects as parameters and receive return values.
Instead of writing
def update_score():
global score, roll
score = score + roll
update_score()
do this instead
def update_score(score, roll):
return score + roll
score = update_score(score, roll)
For in-depth explanations on why global variables are bad news in a variety of situations, see this Stack Overflow answer.
that's not glom π
i have below 2 lists and my expected output. if target [0] and target [1] (adjacent fruits) then simply name
both as FRUITS. keep on doing the same. if adjacent are not FRUITS, just simply leave as it is..
fruits = ['apple','orange','grapes','mango','pineapple']
target = ['orange','mango','CAR','apple','pineapple','BUS','apple']
expected_output: ['FRUITS','CAR','FRUITS','BUS','apple']
i wrote a solution
while (z<len(target)):
if (target[z] in fruits) and (z+1<len(x)) and (target[z+1] in fruits):
new_list.append("FRUITS") #replaces with FRUITS
z+=1
else:
new_list.append(target[z]) #if not fruits, replaces as it is
z+=1```
i feel this solution is not efficient/more confusing. Is there any better ways to solve this?
As always,thanks for reading π
lol i ended up making a helper method for that since the operator.attrgetter has overhead for my usecase
hmmmmm
is there a difference, if any, between using dict() and {}?
@naive saddle :white_check_mark: Your eval job has completed with return code 0.
001 | 1 0 BUILD_MAP 0
002 | 2 RETURN_VALUE
003 | None
!eval
import dis; print(dis.dis("dict()"))
@naive saddle :white_check_mark: Your eval job has completed with return code 0.
001 | 1 0 LOAD_NAME 0 (dict)
002 | 2 CALL_FUNCTION 0
003 | 4 RETURN_VALUE
004 | None
π what is dis?
it's a standard library module
ah
interesting, is there an equivalent for set?
You can unpack an empty tuple
But that's pretty cursed
!e ```py
set_literal = {*()}
print(type(set_literal))
@spice pecan :white_check_mark: Your eval job has completed with return code 0.
<class 'set'>
Courtesy of fix error I think
lol wow
although
that is actual neat to know
since builtins can be reassigned, so if you're worried someone would do that to where you don't even want to use builtins...
!e ```py
import builtins
builtins.set = list
s = set(('g','h','j','k','l'))
print(s)
print(type(s))
@white nexus :white_check_mark: Your eval job has completed with return code 0.
001 | ['g', 'h', 'j', 'k', 'l']
002 | <class 'list'>
Ok wow
A bit more verbose than _, but at least it's an actual throwaway
I'm surprised this is not malware
what is the throwaway variable?
{}[()] is a throwaway variable
this isn't good enough to summon satan
i tried
well you should've tried harder
neat
how do i make a lexer
this {}[()] is definitely hell-bound
Hey guys! newbie here, if I want to start learning python, what would be the best website for it?
!resource Check out the resources link and you can use #python-discussion with #βο½how-to-get-help for questions.
Resources
The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.
https://pythondiscord.com/resources
πΉπΉ
Sebastiaan quite strongly disagrees with you posting esoteric code in this channel (see #community-meta message)
(My above message is a joke, to make that explicitly clear.)
that is not esoteric, I push code like that to prod every weekday
i have 0 contributions but all of them are extremely useful
(set theory joke)
My university was proud of not having a football team, and you could even get a "[Our] football team: Undefeated since [year uni was founded]", and every time I saw it I'd think "Should be 'there does not exist a [our uni] football team that has been defeated.'"
Yes
there does not exist a [year]*
there
wut i just learned that aiohttp has a webserver
>>> {'a', 'b'} * {'c', 'd'}
{('a', 'c'), ('a', 'd'), ('b', 'c'), ('b', 'd')}
not practical given that itertools.product exists, but what if?
Well, what happens if you need 3 products?
Does it special case tuples?
How does {(1, 2), (3, 4)} * {'a', 'b', 'c'} behave?
{1, 3} * {2, 4} * {'a', 'b', 'c'}?
!e
import forbiddenfruit
import itertools
forbiddenfruit.curse(set, '__mul__', lambda a, b: set(itertools.product(a, b)))
print({'a', 'b'} * {'c', 'd'})
@boreal umbra :white_check_mark: Your eval job has completed with return code 0.
{('b', 'd'), ('a', 'c'), ('a', 'd'), ('b', 'c')}
don't ruin my thing.
Sorry.
!e
import forbiddenfruit, itertools
forbiddenfruit.curse(set, '__mul__', lambda a, b: set(itertools.product(a, b)))
print({1, 3} * {2, 4} * {'a', 'b', 'c'})
@boreal umbra :white_check_mark: Your eval job has completed with return code 0.
{((3, 2), 'a'), ((1, 2), 'a'), ((3, 2), 'c'), ((3, 2), 'b'), ((1, 2), 'b'), ((3, 4), 'a'), ((1, 2), 'c'), ((1, 4), 'a'), ((3, 4), 'c'), ((3, 4), 'b'), ((1, 4), 'b'), ((1, 4), 'c')}
Like that, apparently. I don't make the rules here.
That's consistent, but kinda an ugly result, who wants to type for (((a, b), c), d), e in ...
I do, if that's the only way
(this is not a serious suggestion btw)
maybe, but my non-serious suggestion would be counter-intuitive to those familiar with cartesian products and unfamiliar with the limitations of Python's grammar.
!e ```py
import itertools
import fishhook
@fishhook.hook(set)
def mul(self, other):
return itertools.product(self, other)
@fishhook.hook(itertools.product)
def mul(self, other):
cls, args = self.reduce()
return cls(*args, other)
for a, b, c in {'a', 'b'} * {'c', 'd'} * {'e', 'f'}:
print(a, b, c)
(namely that cartesian products are treated as one big chain, whereas Python evaluates it as (((a * b) * c) * d)
@pliant tusk :white_check_mark: Your eval job has completed with return code 0.
001 | a c e
002 | a c f
003 | a d e
004 | a d f
005 | b c e
006 | b c f
007 | b d e
008 | b d f
wtf have you done
That's a solution, product the product.
basically set multiplication produces an product object, then product multiplication makes a new copy with the extra arg
(either you'll know what dialogue this is referring to or you won't)
what is itertools.product.__reduce__ @pliant tusk?
it is used for pickle
That's for pickling.
So it dumps the internals of the iterator, which is very much private implementation details but who cares here.
!e py import itertools print(itertools.product('a', 'b').__reduce__())
@pliant tusk :white_check_mark: Your eval job has completed with return code 0.
(<class 'itertools.product'>, (('a',), ('b',)))
Print("Hello world")
@lavish tartan β Your eval job has completed with return code 1.
Traceback (most recent call last):
File "<string>", line 1, in <module>
NameError: name 'Print' is not defined. Did you mean: 'print'?
@sly jungle for a moment I was about to check if you were a self bot
Hello @pulsar ridge, please claim an help channel for your question. See #βο½how-to-get-help.
And make it repeat its elements like a list when multiplying by a number!
>>> {1, 2} * 3
{1, 2}
>>>
is anybody familiar with the tweepy api?
lmfao
A bit
Cartesian products are cool, but why not Minkowski sums?
!e
import fishhook
@fishhook.hook(set)
def __add__(A, B):
return {a+b for a in A for b in B}
print({1, 2} + {1, 4, 5} + {-1, -3})
@radiant garden :white_check_mark: Your eval job has completed with return code 0.
{0, 1, 2, 3, 4, 5, 6, -1}
Hmm, would Minkowski sums be different from just using a Union on both sets?
you can read the definition right there
oh lmao
actually i just remembered that #type-hinting exists so I'll be there
well, they can run arbitrary code, but I don't know if that counts as abuse, since you can already run arbitrary code in the same python file
has anyone proposed updating python's interactive console so it can parse code like this? I know there "should" be a blank line there, ideally, but it's accepted in a .py file so I thought it might make sense to accept it in the repl
Exactly!!
I know its made to parse only one block of code....
but why? what difficulty it faces?
ipython and bpython tend to make up the difference, though I really see no point not supporting it
Yeah..., if they allow it, it just saves us time in copy pasting code to test
i mean if you use IDLE instead of some fancy stuff like jupyter notebook!!
how about.. getting rid of IDLE?
you can do
def func(): pass
Same result for me:
>>> def func1(): pass ... def func2(): pass File "<console>", line 2 def func2(): pass ^ SyntaxError: invalid syntax
I disagree
IDLE definitely has a place in Python
which is?
when you want to run short pieces of code or you are testing/playing out with python
you don't want to press the run button or python file.py everytime
jupyter?
jupyter possibly, yeah
jupyter requires pressing a run button?
i meant jupyter is a possible replacement for IDLE in this particular usecase
yeah, that could work
jupyter runs on shift enter also IIRC
not sure how thonny and other editors implement their "interactive consoles" though
Yes!!
!d code
Source code: Lib/code.py
The code module provides facilities to implement read-eval-print loops in Python. Two classes and convenience functions are included which can be used to build applications which provide an interactive interpreter prompt.
its publicly exposed, so you could make your own that does allow it IIRC
^ likely the above
possibly, not sure
darn no snippet
Lib/code.py lines 189 to 201
"""Closely emulate the interactive Python console.
The optional banner argument specifies the banner to print
before the first interaction; by default it prints a banner
similar to the one printed by the real Python interpreter,
followed by the current class name in parentheses (so as not
to confuse this with the real interpreter -- since it's so
close!).
The optional exitmsg argument specifies the exit message
printed when exiting. Pass the empty string to suppress
printing an exit message. If exitmsg is not given or None,
a default message is printed.```
ah, I've tried it befire. It's a different implementation, right? in pure python?
yeah
its what python -m asyncio uses
that's what i used to make a kivy interpreter
oh, nice
really convenient way to customize an interpreter
is this yours? https://kivy.org/doc/stable/api-kivy.modules.console.html
no, that's something different
@exotic bay I deleted your comment, as it was off-topic.
I actually didn't know that anybody used anything but ipython tbh
some people use ptpython or bpython
Recently in one interview I was asked that how can I restrict the creation of class objects in python. Any idea about this?
@ashen lance I assume by "class object" you really just mean "object". Every object is an instance of a class. A class object refers specifically to something else.
That said, you can implement the __new__ method for a given class
(on mobile so I'm not going to use markdown) recall that dunder init is not the method that creates the object. It is a method that gets called between when the object is created and when it is returned.
Got you, so if I just do something like
pass
It will just skip creating the object, right?
Got it, thanks @boreal umbra and @unkempt rock
Cool, can I get the source code?
This is in regards to SQL
Let's say I have the following query -
select id from emp where email_address = 'abc@mail.com'
Now since we have 10 million rows in emp table its taking more than desired time, how can we reduce it?
I know using indexing it can be done, but is there anything done in pure SQL query to make it better?
If its a wrong platform please let me know, I'll delete the question/
what's wrong with indices?
if those are the queries you perform most often it seems like an index would be a smart choice
if you have 10 million rows then there's no way you'd care about the extra space needed for an index
hey, this question would be better suited in #databases
fiddling with __new__ can be quite tricky, plan an additional hour or three if you ever get close to that
Thanks for pointing out, posting it there
Ah here it is, they're called scoped packages
https://docs.npmjs.com/cli/v8/using-npm/scope
Yea that was my question as well, but this was an interview question and they expected some ans
In same interview they asked me to explain how dictionary works on memory level... π
Long story short I combined my knowledge of linked list and uniqueness of keys and said something, but researching it now
Pythonβs dict is just a relatively straight-forward hash table if my memory serves
Raymond Hettinger had a talk about how python's dictionaries are structured, it's pretty interesting now they provide iteration over keys in insertion order together with reduced memory
IIRC there's an additional array of keys in insertion order with popped ones being replaced by a dummy value
(Moving here from another channel)
- Anyone like this syntax? I often find myself wanting this when doing somewhat complex comprehensions
[w(z) for y:=g(i) for x:=f(i) for i in itter]
``` or perhaps
```py
[w(z) for y=g(i) for x=f(i) for i in itter]
basically, its the same as ```py
[w(g(f(i))) for i in itter]
however you can add if conditionals afterwards, as well as not having to define lambdas beforehand
especially if your individual functions are too simple to bother making lambdas but also would cause
inefficiencies to nest manually [ex having two outputs] this seems like a nice way to do it
** **
** **
** **
2. Also, on an unrelated note can we allow `\` within f-strings? it's really annoying that this isn't valid code ```py
f"""the items are: {'\n'.join([1, 2, 3])}"""
- Also also, there should be some built in way to do
x if x else default_none_type(x)andx if y else default_none_type(x)(Maybe justnone(x)andnone(x, y)?), basically eval x/y, if its bool form is true then return x, otherwise get (from the class of x) the "none" representation of it, so like if x is a list it would be [], a string it would be "", a number it would be 0, and a custom object you can define it as likedef __none_value__(self)or something, with justNonebeing a default
for the macros thing i would imagine it would be community made and be some sort of pre-ran converter
because im not sure how fully featured macro integration wouldn't break the "easy-to-read"ness of pythonic code
i'd answer if it wasn't one big blob of text..
i was in another channel when i typed it
- eh.. that sounds weird
I wanted to ask if there was a way to remove the bold lines from these captcha images.

I have tried basic opencv functions but the lines are the same color as the text and almost the same pixels wide as the text.
those are like, less thick than the numbers
that makes sense, but hiding eval() in there probably isn't as straight forward as it sounds
man that is uh, a pretty big AI project i think
That is why I said "almost".
actually, i think it's much easier than you think
Can you tell me how I can do it?
move channels pls
you might've noticed that the numbers look the same in those images, make a template and match
<end>
Do you think tensorflow can do it whithout cleaning the image?

!warn 837977275592867881 As said previously, we won't help you bypass captchas. Please stop asking about that here.
:incoming_envelope: :ok_hand: applied warning to @pliant badge.
I remember 2 being discussed in the mailing list, I think it's in the works? And 3 has significant overlap with None-aware operators (still hoping), and furthermore can easily be implemented as a function
learning + automating
Speaking of PEP 505, any new news on it?
wow.... tried pypy for the first time yesterday
tested it with a simple math question compared to cpython
import time
results = []
for _ in range(10):
start = time.perf_counter()
for a in range(100_000_000):
a * a
end = time.perf_counter()
results.append(end-start)
for res in results:
print(res)
import math
print('-------------------')
print(math.fsum(results))
pypy:
43.15774749999764
cpython:
118.40171311050653
it ends up being much faster than cpython
it's often said that PyPy is faster than CPython at all the things except the ones that matter
like?
okay, that's kind of facetious
but more seriously...
most of the things that would be slow are delegated to a C (or other language) extension anyway
so in those aspects, CPython and PyPy are roughly as fast
and PyPy is overall less compatible with the libraries out there
true
but that compatibility is probably because people don't test on pypy as much or use cpython implementation details
also pypy seems to be 2 minor versions behind
yeah
it defo is
but ultimately...it doesn't really matter why, right?
because people gravitate towards the most convenient choice unless there's some compelling reason not to
true
Supporting uvloop
Usually any speedup from PyPy is beaten by the speed of libuv
libuv?
async library
ah gotcha
kinda beating a π π΄, but pypy speeds up your python code which didn't need to be fast anyway, and gives up easy C ffi, which you would use if you needed fast code
Yup that's a nice way to sum up the whole discussion
cffi?
speeds up your python code which didn't need to be fast anyway
im not sure i agree with this. i have written text processing code that i definitely wished was faster. pypy would have been easier to use than cython
I was trying pypy for fun, I actually have my discord bot on it right now and notice an ample increase in speed
ye, pypy is quite nice for text processing
for numerical computation there are better options, but if you need python types to work, it is quite nice
yeah, I don't think I agree with the sentiment either. pypy makes python a generally faster language, at the cost of it being more difficult to speed up the things that are easy to delegate into C, which in practice is often math stuff like matrix multiplication and what have you
python being generally slower is seen by a lot of people as being a pretty huge disadvantage, IMHO
well, it's worse
if speed was the only practical concern, but now it's also about green computing
what's worse?
green computing?
he means energy efficiency I guess
yes
okay, I still don't follow the overall thought though
so which is more energy efficient? lol
there have been arguments emerging to use more optimised languages to save on emissions from running them
yeah, those arguments mostly seem like virtue signalling tbh
faster languages are more efficient
running computers isn't a huge fraction of the energy budget. Even when we're doing profoundly wasteful things like cryptocurrency and NFTs
yeah, well. now people who have no clue about programming can have opinions which languages are better based on their CO2 footprint π
hahahaha
IMO the blockchain is much more important than which languages are more energy efficient
the efficiency is more in the code that's being run, not the language
Yeah, exactly
? BitCoin mining alone was using the same amount of electricity as the nation of Finland https://www.businessinsider.com/bitcoin-mining-electricity-usage-more-than-google-2021-9
Ahh we are on the same page π
short example: using generators instead of making huge lists would be more efficient, and faster
crypto is literally the most important technology to come out of the last ~decade
can't tell if satire
meh
im 100% seirous
crypto talk should go to #ot2-never-nesterβs-nightmare
that's unfortunate then
literally how
but it is true that python is a waste of energy compared to other languages
its adoption is bringing more freedom to the world
....
just well, it is not major enough to be meaningful
positive this isn't satire?
!silence
β silenced current channel for 10 minute(s).
crypto talk should go to #ot2-never-nesterβs-nightmare or some other off topic channel
!unsilence
β unsilenced current channel.
but there's another aspect to think about for energy consumption: that code had to be made. if it takes less time for someone to code a python program than a c program, lights, computer screen, their pc itself, etc
short example: using generators instead of making huge lists would be more efficient, and faster
Interestinglyt his isn't even always true
it depends on the language and runtime
and the deployment aspect too
I thought this was true in Kotlin and I was using asSequence() everywhere, and then i saw benchmarks and it turns out that it's not, even for pretty big lists
the problem is that the python runtime is very naive and doesn't really do any optimizations on end user code
performance is too complicated for simple statements to be true
so really python should not be taken as an example of anything, IMHO
python performance generally isn't worth thinking about too much beyond "do I need to switch to native code for this"
text processing being kind of special here
yes, and the third thought is "I wish I hadn't written this in python" lol
the thing is that I think people are too black and white, they point at something and they say "this is IO bound" like a web server
but the reality is that most things aren't purely one or the other, and in the real world we see that plain old tech companies using python for web server type stuff run into headaches very fast
well, not very fast, sorry. faster.
ye, instagram did have to write their own JIT runtime to run their backend
people that use Java or C# or Go are probably going to just be able to write everything in those languages and not ever have a concern, outside one or two niche cases
right, with python you have these things coming up again and again
dropbox and pyston, which they eventually gave up on, and have now added more Go, or maybe Rust, i forget which.
java programmers do often complain about GC pauses for example
they do, but usually issues can be solved by bringing in one JVM expert, doing some profiling and tuning.
lol I have a discord bot and I noticed it felt faster on pypy
but in the end, performance is not the most important quality for a language to write a web server in
or paying for a low latency proprietary GC
it's not the most important, I agree.
the problem IMHO with python is that you throw away a lot of performance, in exchange for questionable benefit
ye, these days there are actually good alternatives to python
if you are comparing Java/C#/Go/etc to Rust/C++/C, etc, then it's a very clear trade-off
well, there's performance and then there's performance
though the maturity of the ecosystem is still nearly peerless
GC languages are generally less complex, easier to learn, etc, and you take a moderate perf/memory hit.
throwing more cores at a problem doesn't solve the efficiency issue, for instance
gc?
garbage collector
what's a gc language then
garbage collected language?
yes ^
i think it's only really defensible if you absolutely want dynamic typing, which not many people do. Even if you'r eindifferent, the huge performance hit of typical dynamically typed languages is hard to swallow.
you use python for the standard library and ecosystem
python has a good ecosystem but so do JVM and .net
only because a language "feels" faster because it can do things threaded doesn't mean it's more efficient
looks at malware packages on pypi
which is another point to consider regarding energy efficiency
what languages are we speaking about?
python has trouble with threads, but it's also a super slow single threaded language....
I would take python's ecosystems over javas tbh. .net sure, but until very recently you could really only use it with windows server for hosting
so not sure what example you have in mind
just because of how Java code/apis are written?
because you aren't reliant on a half a dozen separate mavens being up at once to do your build successfully
also faster builds overall IME
that's what i mean - if you compare python with a language that solves its performance issues by throwing more cores at the problem with threading etc. it might not be more energy efficient at all
and well, python stdlib > java stdlib
what is this other hypothetical language
practically all other languages solve these issues by being faster than python π
any language that doesn't have a GIL?
but that's just misleading, that's what I mean
Java doesn't have a GIL but it's also just way faster than python
same with C#, Go, Kotlin, etc etc
python is slower than almost everything, even single threaded
the GIL issues are just icing
yeah, the GIL is IMO a worthwhile tradeoff
GIL is a solution, not a problem, and removing it won't magically make CPython significantly more performant
i'll fetch you the paper i'm refering to, one sec
the GIL may be gone at some point
i wouldn't call the GIL a "solution"....
it solves the problem java has where ArrayList is thread unsafe
it's a disadvantage of python, maybe it's a good trade-off for python, but it's clearly a disadvantage
compared to not having it
whereas a python list can be accessed from 2 threads and never corrupt its internal state
what does the GIL solve?
err, maybe you don't corrupt the internal state of the low level python data structures
but in any real program, you are still going to corrupt your own code's invariants
if you write multithreaded code without proper synchronization
Ranking Programming Languages by Energy Efficiency
nobody should be using a regular list for multithreaded programming... so I see this is as a non-advantage
when I say "use" I mean "concurrent reads and writes"
just reading in multiple threads is fine, of course.
it is quite helpful when using threads for IO stuff
Threadsafety concerns and single-threaded performance (removing it consistently degraded single-thread performance)
only because fo the thread safety enforcement
yeah
if python were just not allowed to run in multiple threads at all, removing the GIL would help perf
hm
and well, corrupting your own data is far easier to debug
i just don't really understand the benefit, the GIL enforcement of thread safety is too low level to be some kind of huge benefit
it's a very moderate benefit at best
and the reality is that the lack of parallelism via threading leads to more complicated solutions, which are far more likely to have problems
ye, parallelism is painful
this is just one benchmark though...
also I'm confused, python is almost last on this list
yup
okay, i don't understand how this relates to your point at all
except for memory consumption, btw
you were making it sounds like other languages solve their speed problem with more cores, which doesn't help energy efficiency
but in fact almost all other languages are more energy efficient than python, as well
sorry, that wasn't the point i was trying to make. my point was that perceived performance and actual performance (in terms of operations per energy) shouldn't be confused and removing the GIL probably wouldn't solve either one, really
but i think you've got a point too about the GIL possibly adding to the overhead for single-threaded code
hu?
gotcha
yeah, I mean I wasn't really arguing so much that the GIL adds overhead (though it does). basically just saying:
- python is slow, period.
- the GIL is a downside of python, not a meaningful benefit (though it may be a good trade-off, for python specifically, as things are)
even if there was something like "slow but efficient", if we can't improve python's efficiency, we might lose out on important future usecases like drones and other integrated systems that have to be very conservative
https://lukasz.langa.pl/5d044f91-49c1-4170-aed1-62b6763e6ad0/ will see with 3.11+
something like a drone is hard realtime
generally, professionals (as opposed to hobbyists) aren't writing that kidn of stuff in any GC language
speaking of 3.11+ I'm checking https://github.com/faster-cpython/ideas a couple times a week now just to see what the brightest minds in python can come up with when performance is the goal
Bu ne ya
- is incorrect. The GIL makes Python faster
you're misunderstanding what I'm saying
i know the GIL makes python faster in single threaded, i'm just saying that when you're comparing languages, the GIL is a downside. How fast the language is depends on many things, the GIL helps python but obviously not enough to make up for things weighing it down.
that's why I said "it may be a good trade-off for python specifically" - python in its current state, the GIL is probably a good thing (at least the GIL versus the ways in which GIL-less implementations were suggested in the past)
Reading it again yeah I think I may have.
The GIL is something I won't recommend to other languages, and it's something I won't hate seeing go. That said it makes Python easier - for everybody - there are much less concurrency problems that people run into.
I guess I have mostly excluded threads in Python because it really doesn't do threads good.
there is a pep to remove the GIL IIRC
see, the concurrency argument I don't buy
gonna read what the GIL does brb
this came up recently and the thing with the GIL is that it's no different tha a program in any other language constrained to run on one core (i.e. no parallelism)
the tldr of my understanding is that the GIL means threads don't do anything
not accurate π
threads in python (w/ GIL) let you run code concurrently, but not in parallel
sometimes concurrency is all you need, sometimes you also need parallelism
so it's like asyncio but switches back and forth constantly?
that said, there isn't usually much difference between "concurrency-safety" and "parallel-safety" - it's all just thread safety. You still have to use thread safe data structures and locks in many cases in python.
(most cases)
i'm actually not too familiar with asyncio. I think asyncio separates the work that needs to be done, from where it gets scheduled
it could be scheduled on a thread pool, or in other ways
in addition asyncio also switches contexts on await statements
threads are not like that, each thread has a specific top level function its running
right, there's cooperative multi tasking there as well
so it's guaranteed that a non coro function will not have any states change
with threads there's practically no safety and you have to lock things if they can be scheduled at the same time
Right, yeah, so people say it's easier to program against.
right.
the point being that the GIL doesn't help with the safety of raw threads in any substantial way
and the GIL isn't necessary, fo course, to have a model like asyncio
many languages have it and almost none have the GIL
the implementation would have to guard against things like data races from parallel writes but I don't think that'd quite get to the language user
so I don't think the GIL is beneficial in a vacuum in any way; it's just something that python seems to need to help address its performance issues. I'm not sure how much of that is a general trade-off between single and multi threaded and how much seems specific to cpython's implementation.
i assume the latter though because single threaded performance is mportant and I've never heard any language discussing adding a GIL
threads are just a pain to work with when there's shared mutable state compared to something that has explicit yield points so even if the GIL added some safety I wouldn't really say it's worth the other costs (if that was its only point)
i mean sure but you talk about "explicit yield points" like they solve all problems
They make things quite a lot easier to wrap around
for doing asynchronous IO, sure
concurrency is complex so there's no going around that, but when you know that things can't switch at some point it gets at least a bit easier
if you're doing parallel processing then people don't generally do things that way
basically, if you are doing single-threaded, cooperative multitasking as a way to do asynchronous programming, sure, now you created a world where thread safety isn't really an issue because functions dont' get unexpectedly interrupted
that's a very very specific kind of programming though, it just ends up dominating a lot of conversations on the internet because it happens to be what a lot of webservers want
i guess what I'm saying is a language adding real safety to multithreaded programming would be very valuable, it can't be dismissed just because models like asyncio exists.
the GIL doesn't add real safety though
It's think it's worth pointing out that multiprocessing avoids the GIL problem entirely.
"works around" might be a better term, from my perspective π . it does help, and it does make solving many common problems reasonably easy
You're not wrong.
there are real issues with multprocessing that come up even in relatively simple programs
Oh for sure.
just the fact that you can't define a helper closure locally, to pass into a multiprocessing pool
Shared state is just a nightmare
that made me facepalm many times
shared mutable state :-). When you use a pool though it's pretty easy to avoid said state, whether it happens to be a process pool or a thread pool.
Right.
it's just that a thread pool also avoids many other headaches for you
I mean, actually using processes in most other languages is... pretty rare, i've only done it when I really needed to isolate a process completely from another, e.g. worried about my C++ binary segfaulting, so I launch the binary from a python script, which stays up, and ensures that it hasn't segfaulted. that sort of thing.
I think we're going to see more and more work in this arena though. Especially with the multicore workloads proliferating. Thank/blame the data scientists and crypto nerds?
it started long before those two buzzwords took off though
The Berkeley folks called it a decade? Ago
really as soon as they started not being able to keep cranking up single core speeds, they started adding more cores instead
They said that cores were going to be the new transistor.
seems like a bit of an exaggeration π
I have seen architectures a bit more like that, there was thing this my company looked at a few years ago, it was like 200 very simple cores, along with a few normal cores, on one socket
i think it was called phi or something
I mean that's basically what gpus are
There's a lot of software that mostly does IO-bound work. If you're doing CPU bound work with Python you're essentially better of using another language or writing C extensions (which can release the GIL) for Python I guess..
I don't want to turn people away from using Python - but I can't honestly recommend Python for such work.
Whenever I see threads used they can almost always use asyncio
well, all of data science, numerics, etc is a pretty big exception to the above
And guess what all the data scientists are learning
i agree that there is a lot software like that, I just meant it's still in the grand scheme of things, pretty specific, just hugely overrepresented in these online forums
well these do what bluenix suggested: use C
there's lots of people writting embedded code, for cars, microwaves, device drivers, there's people writing financial software, people writing medical software, GPS, I could go on and on, and that stuff is basically never discussed, compared to how often people are like "okay, so your web server, right, it's fielding requests..."
That said even asyncio programs sometimes need to do small (but still blocking) CPU-bound work - which is where I would use a threadpool (because that work is not cooperative) - and it would as well benefit from the removal of the GIL.
Here's the post that outlines the scheduling issues that CPython has though https://bugs.python.org/issue7946
it's funny how when I started visiting places like proggit, etc, everyone used frontend dev, and backend dev, and I had no idea what they were talking about
even though I had been programming for a while
of course, we do, but because it's faster, not because it's a pancea for parallelism problems, because it is not
realistically even if you're using pandas to process some data, you still want a process pool, not a thread pool
i mean kinda? you can use openmp all wrapped inside the c code and not even touch the gil
enough work happens in python
I was gonna disagree with this but Python is in space π€
i don't understand
i'd guess that web servers make up less than 30% of python code that's deployed in the world
yeah, if I just rewrite absolutely everything that I do in C or C++, and bind a single function on top to python, the GIL won't matter, that's true π
but that's not usually the point
but why does it have to be C? why not go or nim?
those are way way faster than python and still a lot easier to use
yes nim compiles to C but that's not the point
even though flask, Django are very popular and in some of the top pypi downloads?
nim is a meme language. And it's about the ecosystem of packages for data science applications. And the console, as well.
managed memory languages are so much easier to deal with for a lot of / most use cases
They don't implement the web server part (that runs in prod, they do provide development servers)
that's usually offloaded to gunicorn/uvicorn/etc
but in any case I don't really understand what you're suggesting
my point is just that when I wrote some python data scienc-y code that I need to parallelize locally, I still need to use multiprocessing, that's all
right
(but joblib is probably easier to use than multiprocessing and it also memmaps and caches numpy arrays)
the things we do locally are pretty simple
for bigger stuff it's being parallelized on a compute grid so we need to use the APIs provided for those
my suggestion overall is that there should be, and are, intermediate solutions between "use python multiprocessing" and "use C++"
well, the actual solution is just a faster language π
The functional programmers are pumped about parallelism.
more likely julia, tbh
but it's not there yet either
clojure as a language for that niche doesn't really make sense
nim is statically typed, i think that interactive analysis is probably one of the few niches where dynamic typing seems to be helpful.
oh gotcha
or use python (the language) but use one of the alternative interpreters/compilers that make full use of multiple cpus or even CUDA
i think we're too stuck in cpython land
how about cython?
prototype stuff with cpython, switch to cython once you got your types all set up
I'm doing a web development thing, and the API has a lot of constructions like this:
return (
handler_input.response_builder
.speak("This is the first response")
.ask("add a reprompt if you want to keep the session open for the user to respond")
.response
)
I don't even know what types the intermediary objects are (those returned by .speak(...) and .ask(...). It might even be that these are each mutator methods that return self. In either case, is this design pattern JS-inspired? It seems like it might be a reflection of keyword arguments not being a language feature.
(I write similar-looking statements with Pandas, but that's because I'm just chaining a bunch of operations.)
idk where it's from, it shows up a lot in java also
it's in Go4 iirc
it's called the builder pattern or sometimes fluent interfaces
assuming it is returning self each time
you can also add Rust here π
really almost every language
yeah
keyword arguments are a really rare feature
python and kotlin, and almost no other mainstream language really
i think clojure kind of has them
To be honest, keyed arguments do create some issues. For example, ParamSpec and friends are pretty much useless because it would be messy to support it with kwargs. And look at how well TypeScript handles signature transformations.
what kind of issues, sorry? Not too familiar with paramespec
it's a typing thing
ocaml handles keyword arguments and static typing quite well tbh
so in kotlin, typically, keyword arguments are mostly outside of the type system
if you call a function directly, you can pass its keyword arguments, if you pass a callable into a function, you don't
but it has specifically keyword and positional arguments as separate constructrs
which is a simplification but I think is perfectly fine
For example, you might want to transform a function with a callback into an async function: (*args, (T) -> None) -> None => (*args) -> Awaitable[T]. You can use ParamSpec for that.
Sure.
hey! discord.py has self support for embed objects
...or you can't. I don't even remember.
I can see how that's an issue, but in practical terms, if you have the type system simply "give up" on that
I don't see the connection
you still get like 90% of the benefit of keyword arguments, with almost no effort
yeah, they're pretty nice
'nother library that also has chaining support
usually when there's a callback it's not a great idea to force users to adhere to a particular set of keyword arguments anyhow, IMHO
if you really have enough arguments, or enoughc omplexity in defaulting behavior, etc, that you want to do this
I would just have a callback that accepts a dataclass instead
ye, in python it's kind of painful since the default is both keyword and positional, which, while it is quite useful for letting people call functions nicely, makes the types quite annoying
for example, a subclass overriding a method and changing argument names is a violation of the LSP
well, typically what you'd do in practice when a function accepts a callble, is to take... Callable π Which amounts to a promise that you will only call using pisitonal arguments
you also need to be careful with __call__able Protocols
that is, use / in __call__ if you don't need the keywordness of the arguments
or prefix arguments with __ if your version of python doesn't have /
wth
i agree that you're technically correct, I just don't think it's a big issue in practice though
which is why you don't typically see the / suggestion in best practices books
I think keyword arguments are good for normal function calls
I don't know, maybe the lack of type system support is Python's fault
it is a dynamically typed language and a lot of the design decisions are made with that in mind
so was just refreshing. basically, in Kotlin you can only pass functions, into functions, as a positional-argument type
so you just can't make any keyword calls in that case. When you override things, only the positional arguments matter
if the overriding doesn't follow the positional arguments it's just a compiler error
that makes sense
and when you make the call, you can use keyword arguments, but they always follow the "static" function signature
language aside, some IDEs/plugins show inlay hints with argument names when you call a functions
doesn't work for people reading via notepad/vim/github, but still nice
that's what I like about kwargs--it creates some complexity, but imo it's worth it especially for cases like
Path("mydir").mkdir(True, True)
```where you have default boolean parameters
so it is self-documenting and you don't have to explicitly pass `False` or `None` for a bunch of arguments to get to the parameter you actually want
which I have had to do in Java at least, and probably C# (iirc c# makes you write all argument names or none of them, and changibg the ordering/defaultedness of functions can break compatibility, whereas with keyword-only arguments you can safely evolve the signature of your function)
yeah, I agree it's worth it. I think the follow up question is just how it's handled in the type system, and then you have a bunch of answers, all of which have significant disadvantages
yeah
Python devs need to make help more type annotation friendly. When you have a function with a long signature and then add type hints on top the docs produced by help are hard to read. It's almost like they need to run black over it
!eval import black
@naive saddle :x: Your eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 1, in <module>
003 | ModuleNotFoundError: No module named 'black'
Alright black is not usable as an eval-able example :p
perhaps we should add this? What do you think, @odd flax?
that was supposed to be a ping for Kubernetes Man 
Eh, probably not that useful right now as black has no official API
maybe this?
This is from help(black.main.callback) (click wraps the main function)
to be fair you also don't want 30 lines in the help only for the signature
<@&797752289771126784>, we need black to be in the bot's snekbox.
Hi everyone. Iβm trying to solve with the best solution to this problem. I want to check running status of a service in three networked windows 10 machine. This three machine like this. (a,b,c).
Computer a is master and service at first running only this machine and b, c computer are slave that only use when machine a service was down or stopped. I develop a python program that had multiple socket connection to the machine but that wasnβt good and optimal. We want all machines can see and check the service if one of them stopped the service other only one machine starting that service.
Who can help me to solve this? Thanks a lot.
just make a pr to the kubernetes repo
I actually
I meant to do that for some other packages
!e import arrow; print(arrow.version)
@white nexus :white_check_mark: Your eval job has completed with return code 0.
0.17.0
!pip arrow
so old
Hi all! I hope this is the correct channel to ask this, if not let me know. Does anyone know why this is not correct in python?:
class TestClass():
def test_method( test_dict: typing.Dict[str, TestClass]):
print("test")
I get an error that TestClass is not defined. How can I give a type hint that test_dict is str -> TestClass?
"TestClass" mayhap? Wrap it in quotes
Ohh, that seems to work, thanks! What is the reason for having to put it in quotes to get it to pass?
it's only partially defined at that method, so wrapping it in quotes makes it deferred to be evaluated
Ok, i see. Thanks a lot!
@still fox Please stop randomly spamming this
π what is the best way to mutate a tuple?
convert it to a list and then change the position I want to change, then back to a tuple?
ok alr
(its the args passed to an exception which I need to change, so not using a tuple from the start is not an option)
Use ctypes to hack Python π
hmmm now I wonder which is more efficient
slicing the tuple around the replaced part, or converting to list and then back
i highly doubt slicing and concatenating will be more efficient
In [1]: def via_index(tup, index, val):
...: return tup[:index] + (val,) + tup[index+1:]
...:
In [2]: def via_list(tup, index, val):
...: l = list(tup)
...: l[index] = val
...: return tuple(l)
...:
In [3]: t = tuple(range(1000000))
In [4]: %timeit via_list(t, 500000, 99)
149 ms Β± 20.7 ms per loop (mean Β± std. dev. of 7 runs, 10 loops each)
In [5]: %timeit via_index(t, 500000, 99)
124 ms Β± 23.3 ms per loop (mean Β± std. dev. of 7 runs, 10 loops each)
not much difference
yeah, they're pretty similar for a single consecutive group
In [8]: %timeit via_list(t, 5, 99)
391 ns Β± 27 ns per loop (mean Β± std. dev. of 7 runs, 1000000 loops each)
In [9]: %timeit via_index(t, 5, 99)
524 ns Β± 17.3 ns per loop (mean Β± std. dev. of 7 runs, 1000000 loops each)
``` For smaller tuples, `index` is taking less time, interesting (range is `10`)
bot#1880
Note
It is possible (but not likely) that the parser stops parsing with a successful outcome before reaching the end of the source; in this case, trailing symbols may be ignored instead of causing an error. For example, a backslash followed by two newlines may be followed by arbitrary garbage. This will be fixed once the API for the parser is better.
very nice, python
this isn't the right channel, you should ask in #tools-and-devops
that said, this seems to be the relevant documentation: https://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/JavaScript Notebook Extensions.html#custom.js
lib-python/2.7/unittest/test/test_program.py line 58
assert True```
`testrunner/test/examples/normal/failingsetup.py` line 6
```py
assert True```
`rpython/jit/backend/arm/test/test_regalloc.py` line 688
```py
assert 1```
Testing that the assert statement itself works?
That comment above the last one implies that
huh, you just made me think of the following: a rudimentary way to see if python is in optimized mode would be to try except an assert False statement
(yes I know there's a flag for that)
I don't think checking if it's in optimised mode is the goal of these tests
edited
Looking at it again, the first test is a test for the unittest library. The asserts seems to behave as they normally would - no special pypy behaviour. They're just there so that the tests have both a passing and a failing result to work with. A bit further down there are some tests that unittest will have results or it will raise SystemExit.
Same thing with the second one. It's a test for the test runner and it needs a passing test case. I guess assert True could have just been pass but maybe the former is more clear about its intent?
for me assert True would be more puzzling than intention-revealing
Well, it's a test case. What do tests do? They assert things. This test needs to ultimately pass. How do tests pass? When assertions are true. What's the most trivial way to get a true assertion? assert True
"all assertions are true" is also satisfied if the set of assertions is empty
Technically correct, but that is less evident than assert True
well I guess that's subjective
hello
Is it a bad idea to have types as dict keys? {int: 1, float: 2, MyClass: 3} 
I've never thought of it before but seems to work fine, just feels weird
depends on what you want to do with it
just as a little look up table instead of doing bulky if type checks and individual variables per type
using a dict to replace repetitive if/elif/elif/else chains is pretty typical
i guess you can use match now too
class MyOtherClass(MyClass):
pass
``` this willl break
I know, but luckily that shouldn't be a problem
if you don't have inheritance, then yes, it's fine π
wow, fable compiles to python now
π π
that is extremely interesting to me (and RIP coconut)
break how?
looking it up in a dict won't match
whereas you would match with an is instance check
its type won't be in the dict
which is different from an if-elif chain with isinstance
ah that
a or b
a: obj.__class__
b: type(obj)
c or d
c: obj.__class__.__name__
d: type(obj).__name__
i think a and c
b and d, a and c could theoretically lie
!e
class Foo:
__class__ = 42
f = Foo()
print(f.__class__)
print(type(f))
try:
print(f.__class__.__name__)
except AttributeError:
print("error")
print(type(f).__name__)
@quick snow :white_check_mark: Your eval job has completed with return code 0.
001 | 42
002 | <class '__main__.Foo'>
003 | error
004 | Foo
...how does type() get it right then?
looks directly at the type of the object, without going via attributes. The type of an object is part of the PyObject struct.
ohhh, type is implemented in c
I only use __class__ inside of a class definition
Otherwise i use type()
interesting, i've been using __class__ π
How come in this PyPA documentation a namespace package can have an __init__.py? https://packaging.python.org/guides/packaging-namespace-packages/
There's some code you include to make it a namespace package, not having to do that is the reason for the implicit namespace packages
Ah, right
Type 2 and type 3 namespace packages need the __init__.py, type 1 don't.
Just the perils of trying to describe all 3 on one page
I don't understand the question... You can import things from namespace packages. There's no good reason to use the type 2 or type 3 ones besides backwards compatibility with an existing namespace, though
As in, importing things inside the __init__.py
The entire point of a namespace package is to serve as a namespace under which other packages can be placed. If you want imports in the __init__.py, then what you want isn't a namespace package, because it doesn't make sense to import things from other packages (that may or may not be installed)
you need to choose between making a regular package (which exists in only a single directory on disk, comes from a single pypi package, but allows you to fully control what's exposed in it) or a namespace package (which can exist in multiple directories on disk, can come from multiple pypi packages, but whose contents are based on which packages are installed underneath it)
Yeah that's true, I was thinking of essentially revealing some things under that namespace as an extra shortcut but yeah I'll just do as you're supposed to lol π
Yeah that's JavaScript for ya' π π π π
that's why you use === instead of ==
!ban @dense rune 14d seems like you are just here to advertise, should you come back, please read our rules.
:incoming_envelope: :ok_hand: applied ban to @dense rune until <t:1639138301:f> (13 days and 23 hours).
!pep 679
PEP not found
PEP 679 does not exist.
@surreal sun ^ what pep?
wait what, that's weird
maybe it's another number, but it's the late bound arguments PEP
!pep 649
That's the one
That's it
so what's it mean?
Usually, default arguments are evaluated at function-creation time, which is good for optimization purposes, but can have somewhat surprising effects for mutable arguments
!e ```py
def f(l=[]):
l.append(1)
print(l)
f()
f()
f()
@spice pecan :white_check_mark: Your eval job has completed with return code 0.
001 | [1]
002 | [1, 1]
003 | [1, 1, 1]
ah yes, where bugbear has saved my ass multiple times
In order to bypass this, either None or a custom sentinel is used for the default argument, and then actual initialization happens in the function body
The pep proposes new syntax, which would reduce boilerplate and evaluate the arguments at call time
So with the PEP implemented, this would work:
def f(l=>[]):
l.append(1)
print(l)
f()
>>> [1]
f()
>>> [1]
f()
>>> [1]
hmm
stupid annotations
I actually think with the walrus operator, def bisect(a, hi:=len(a)): makes the most sense
except for annotations...
since that would mean it would look like def bisect(a, hi: int := len(a)):
Because [] is evaluated again every time you call the function without providing l. This would also allow making default values dependent on each other, like this:
def f(seq, l=>len(seq)): pass
Yeeah, that's not very pretty
although.... def bisect(a, hi: int => len(a)) -> int:
I'm not actually sure which option would be optimal for improving readability, reducing ambiguity and keeping visual noise to a minimum
But in general, I like the idea
solution: rewrite all annotations and everything and release python 4.0 /s
It would allow for better self-documentation on function signatures and reduce common boilerplate
Release python 4 as statically typed
I actually like this one
def bisect(a, hi=@len(a)):
the @ makes it seem more like a decorator which mutates the function, and that's (kind of) what this does...
hmmm
noisy tho
this would be a nice thing too, a good singleton
def add_item(item, target=>[]):
# Equivalent pseudocode:
def add_item(item, target=<OPTIONAL>):
if target was omitted: target = []
something like typing.OPTIONAL which is a singleton and could be used for a default value instead of None
I think I like ? from a semantic standpoint, but it's visually loud
that way there's a standard for that
I think there's another pep for better sentinels
It should be mentioned in this one, maybe in the references section?
!pep 661 I think
def bisect(a, hi?=len(a)): ...
def bisect(a, hi=?len(a)): ...
def bisect(a: str, hi: int ?= len(a)) -> str: ...
def bisect(a: str, hi int =? len(a)) -> str: ...
huh
This is not the original intended use of Ellipsis, though it has become increasingly common to use it to define empty class or function blocks instead of using pass.
so what is the intended use?
I think it was meant for numpy etc, just like @
@ is the matmul operator
huh, apparently so, but can't understand why the stdlib would introduce something for a third party
it does make sense to provide features for libraries to take advantage of, even if the stdlib doesn't need them
true
Why would you not help your premiere 3rd party library be easier to use
true
Is github a good place to just look through projects and study what they do? Or is there a better place for that.
I just wanna see the programs people have made and study them so I can use those techniques.
GitHub is a good place to read code
90% of github projects aren't going to be very good tho
I mean, yeah
most of them are beginners starting out, or abandoned projects from years past
you shouldn't be reading random stuff people wrote