But some certainly tend to be more easy to abuse/frequently abused than others.
#internals-and-peps
grizzled vigil
gloomy rain
That might be the case
flat gazelle
in fact, I would say it is much easier to abuse stateful loops and such to an absolute mess and you get much worse code than you can by overusing FP patterns
gloomy rain
Though I think it's often more related to the individual than the language feature
Some people write bad code no matter what tools you give them
@flat gazelle At least FP often has certain safeguards, such that the code won't even compile if the logic isn't sound.
Compile/run*
subtle pike
formal verification is a powerful thing
flat gazelle
computers are much better at consistently not making mistakes than humans.
subtle pike
i have come to realise that autosphinx is rather lacking
it does not like typehints or dataclasses at all
haughty birch
computers are much better at consistently not making mistakes than humans.
@flat gazelle However they are also much better at consistently making mistakes...
heady axle
@subtle pike Jumping into the running train, what type of formal verification are you talking about ?
I studied formal verification at the uni and it's was kinda trash. We would use logical operators (and, or, exists, forall, etc.) to specify methods and a verification engine would check our code pass the specification.
However, syntax was trash (Java Modeling Language), Verification Engine was unusable as it didn't implement most useful features of the language and most of the time, for complex methods, specification was way longer and less maintainable than the actual code it was verifying.
subtle pike
formal verification isn't practical at the moment but it does have useful applications for critical services. The best example of formal verification would have to be Coq, it has a relatively simple syntax
i wouldn't roll full blown formal verification for any old bit of code but runtime analysis can work
Coq is neat in that you roll your own proofs and if your code compiles, it's guaranteed to be functionally correct
heady axle
What would be an actual use case ?
I mean, unit tests are already great but I'm refactoring a 500 loc module right now and it basically has no unit test. I mean this as an example than devs don't always test their code. Because they feel it's not worth it. Proofs seem even more work than unit test.
unkempt rock
@subtle pike The code at least do what the proof says. That may or may not be correct.
There is no guarantee that the proof actually lines up with what the code is expected to do.
subtle pike
@heady axle suppose you were developing a pacemaker or an industrial system, it may prove useful to have a concrete proof of correctness because getting it wrong would have extreme consequences
unit tests don't prove it's correct, they only check certain cases
a proof of correctness verifies it is fully correct, rather than fuzzing to obtain approximation of correctness
@unkempt rock well yes, hardware and the language implementation may not abide by the pure mathematical laws but if they don't, you have found an error in your implementation which is an improvement
gloomy rain
I think they're saying the proof might prove something other than what you intended it to.
Because you're bad at writing proofs.
heady axle
I get the argument, proving is costly but it can be less than the cost of failure. I've never worked on that kind of projects :-s. Could be fun.
subtle pike
to have a useful proof, you provide constraints you want for the solution. If the proof is able to satisfy those constraints, you have solved your problem
unkempt rock
@subtle pike. I'm thinking of the case where the spec is misunderstood. Both developers (if there are two; if not, formal proof is just writing the same code twice with a funny syntax the second time) can easily make the same misunderstanding. That's what I regularly see when doing retroactive QA.
subtle pike
ah, i see
heady axle
I don't like the argument of "the programmer can fail writing the proof too". We can extend that to nothing can ever be proven true. Sure we can fail, but humans have an history of having success with proofs.
subtle pike
if you implement it incorrectly because the semantics of the proof checker don't align to the language? (is that what you mean, awegge)
sacred tinsel
Another example that we studied at university were automated train signaling systems (go / wait) - formal verification is used to prove, at a mathematical level, that the system will not fail and have two trains on the same track at once
unkempt rock
Formal proofs can be useful, but if the are treated like automated tests, where the developer writes the test case for his or her own code, they lose value.
@subtle pike Very primitive: def add(a,b) return a-b
unkempt rock
A bit more than that. This is just stating the general case of code not doing what's expected. In most cases it's a bit more subtle.
gloomy rain
@heady axle I'm not saying that possibility makes formal verification not worthwhile, I think the point is just that it also doesn't entirely eliminate human error from the equation.
subtle pike
true, formal verification only considers itself with computational correctness which does make it slightly hazardous to use if you aren't fully sure of what it's doing
i doubt that'll ever be rectified because it's a subjective human issue and we can't say what is and isn't correct in that regard
gloomy rain
Not rectified with a single silver bullet.
But there are various measures we can take to mitigate the issue.
And probably more to be discovered.
subtle pike
perhaps you need to document your code before it compiles 😆
flat gazelle
@haughty birch a mistake made by a computer is generally much easier to find, as every action is logged and as long as state is replicated, the mistake will happen again. That is actually another merit of FP, it makes it very clear what affects the result and therefore makes it easier to figure out where the error came from.
subtle pike
stateless code is always easier to debug in my experience
way easier to track changes between blocks
heady axle
And here am I in a codebase with mutable objects where everything is stored in T_T.
flat gazelle
ye, that is imo greatest merit of FP. much easier to keep track of state, and if done well no invalid state will ever happen.
subtle pike
managed to crash autosphinx ;-;
paper echo
easier, until you make a tangled mess of functional abstractions 😉
swift imp
I was hoping to discuss PEP 485. https://www.python.org/dev/peps/pep-0485/#relative-tolerance-default
I'm confused on how the simple delta epsilon definition for approximately close fails for any use case.
is_close(), returns a bool indicating if f is within e of L
subtle pike
floating point comparisons have never been fully accurate
flat gazelle
say I can get a range of values anywhere between 1e6 and 1e-6. There is no single e which would be equally tolerant on all of the possible values. There may be one which is good enough, there may be also not be one like that
swift imp
I dont understand what you are saying
Is this like some issue rooted in IEEE 754?
Like if you want make sure f is with some +/- e of L how does the delta epsilon fail?
flat gazelle
floats are inexact, being approximations of the actually correct results. The purpose of isclose is to allow for float comparisons, as equality is not reliable, for example
In [66]: (.1 + .2) + (.1 + .2) - .6
Out[66]: 1.1102230246251565e-16
``` should be 0, but it is not 0
full jay
It's more just the issue of trying to represent fractions of a number using 1's and 0's
Computers just aren't well suited for it
swift imp
so its the floating point standard, gotcha
subtle pike
not so much the standard
full jay
There are other implementations that try to handle it better
subtle pike
try to represent an infinite number space with finite space
it's not an easy problem 👀
full jay
Yarp
swift imp
I think ive read a paper where they suggest an alternative 754, its been a while though
flat gazelle
ye, there is no way to get fast and exact rationals. You can have exact with Fraction, you can have fast with fixpoints (though these days floats are faster), or you can have a bit of both with floats
full jay
Honestly, if it's financial stuff, I generally just try and move the decimal place out of the way and do my work that way
subtle pike
for sure
full jay
Just handle everything as ints
subtle pike
factor your data or modulo it
full jay
Yeppers
full jay
I think Python has a built in lib for handling floats better, but it's slightly more costly in overhead, so it's not the default
I THINK
Don't quote me on that
lofty anvil
Hey guys could somebody help me with my school project? using Raspberry pi Python, Flask and OpenCV
full jay
@lofty anvil This might be better in its own help channel. Probably going to take a decent amount of talking to help out on it. See the #❓｜how-to-get-help channel for details
lofty anvil
ok thank you
magic python
Is there any other way to make use of breakpoint other than breakpoint() ?
i'm wondering about when there are multiple breakpoints in different modules or something, is there anything additional that it can do?
breakpoint(*args, **kws)
Call sys.breakpointhook(*args, **kws). sys.breakpointhook() must accept
whatever arguments are passed.
By default, this drops you into the pdb debugger.
seems that it takes args
but idk what to do with them
breakpointhook(...)
breakpointhook(*args, **kws)
This hook function is called by built-in breakpoint().
doesn't really say much - I've tried breakpoint(' check data') or whatever, and had no joy
visual shadow
The rationale for this is based on the observation that the underlying debuggers may accept additional optional arguments. For example, IPython allows you to specify a string that gets printed when the break point is entered
So, to answer you, no not really. You aren't expected to pass args there
undone hare
Tbh you shouldn't manually write breakpoint()s, but you should rely on your IDE to inject those into your code (like the red dot in PC)
paper echo
if you do manually write them, how should you do it?
in the past i've just done import pdb; pdb.set_trace()
swift imp
.pdbrc, greatest thing ever, and all the more reason to program in a linux environment. Of course you could just an ide instead.
paper echo
TIL
i just use pdbpp and leave it at that
also sometimes i dont have an IDE available
SSH into a server
swift imp
Yeah
Well thats why I like the .pdbrc. I just keep it inmy home area lol. Got a few vim script functions to display them/add them/remove them
magic python
@undone hare hrm ok 🤔
@swift imp yeah - i have it setup to use ipython and ipbd etc instead
swift imp
It works in either ipython or python
magic python
i don't like python
swift imp
it will trigger if you do python -m pdb <script>
or a %debug in ipython or even ipython -m pdb <script>
paper echo
ahh i see
so you set breakpoint then run with any debugger
i see
what if your script is already a module
swift imp
Yeah, that file gets read anytime pdb is firedup
swift imp
What do you mean, can you give an example?
timid orbit
god I hate USB protocols
magic python
i always use breakpoint() in the code
swift imp
manually inserting breakpoints just suck when you have a large project and have to go edit the file
you now if you fire up #debug in python you can type b <path to file>/<file_name>.py:<line> and do breakpoints like that. That's essentially what the .pdbrc is doing, it just saves you from having to type it manually each time.
timid orbit
Does anyone here have experience with microcontrollers
because 0x0F840004 seems to be a register on the AGB bus but I can't figure out what it is
north root
that'd be a better question for #microcontrollers
timid orbit
specifically what this does: value_at_x0F840004 = (value_at_x0F840004 & 0xfff7) | 4
oh yeah
paper echo
@swift imp i have code that i run as python -m mytool.pipeline a b c -- can i debug that with breakpoint()?
swift imp
Why do you use -m for a python script?
python -m pdb mytool.pipeline a b c
Why wouldn't that work?
magic python
@swift imp I'm being thick I guess .. But I use breakpoint () explicitly because I debug from a terminal, is that still wrong?
Or rather, is there a better way to use breakpoints with that workflow
swift imp
With a .pdbrc file
You store where I want them in that
Unless you are literally coding in a terminal
charred wagon
How does that work? You just write a line number and module namein the file?
swift imp
Yup
you now if you fire up
#debugin python you can typeb <path to file>/<file_name>.py:<line>and do breakpoints like that. That's essentially what the.pdbrcis doing, it just saves you from having to type it manually each time.
@swift imp
Just like this
You literally type that into the .pdbrc file
paper echo
@swift imp its not a script, its a command line tool
Even more likely, im running my code as an entry point console script
magic python
@swift imp my pdbrc just has ipbd config stuff in it
None of what you're saying seems familiar, what can I Google?
Typing in the path as you have there seems more hassle to be honest?
<@&267629731250176001> @lunar relic
lunar relic
<@&267629731250176001> @magic python
stable grail
!hush
fallen slateBOT
✅ silenced current channel for 10 minute(s).
north root
...
stable grail
!unhush
fallen slateBOT
✅ unsilenced current channel.
stable grail
this kind of behaviour is unacceptable, thanks for the notice rie
lunar relic
he false reported me sir
stable grail
!ban 723578794027384995 please take your attitude and leave this server. your rules and code of conduct clearly says what kind of behaviour we want here.
fallen slateBOT
:incoming_envelope: :ok_hand: applied ban to @lunar relic permanently.
stable grail
thanks @magic python
magic python
I just want to know these pdb things 😅
https://nedbatchelder.com/blog/200704/my_pdbrc.html
Seems ned uses breakpoint ()
visual kindle
Hello,
for y in sduhhu:
......```
Does any1 knows how I continue the first loop out of the second loop ._.
modern night
@visual kindle i dont think this belongs here, probably best to check out #❓｜how-to-get-help
sacred tinsel
I don't think that's possible, continue and break always only apply to the inner-most loop, but maybe breaking the inner one will do that for you if there is no other code in the outer loop?
Jamie is right that this likely would be suited for a help channel
stable grail
as an advanced topic this could be solved using a generator. It would depend on the details.
true cave
Can someone explain how partial() works?
north root
functools.partial ?
true cave
Yes functools.partial.
Actually I just figured how it works.
I didn't know the first argument needs to be a callable function/class.
What's the point of using partial? The requesting function could have requested one or more parameters.
north root
it allows you to pass in arguments before
true cave
This is my working sample
>>> def status_task():
... return {'something': 'else'}
>>> status_map = {'Capture Start Queued': status_task}
>>> func = status_map.get('Capture Start Queued')
>>> partial(func, {'id': 'someID'})
functools.partial(<function status_task at 0x106dd4ef0>, {'id': 'someID'})
Before the callable function is called?
So in my example, it allows me to pass the callable status_task before I actually call status_task?
north root
functools.partial is roughly equivalent to something like this:
def partial(func, *first_args, **first_kwargs):
def inner(*sec_args, **sec_kwargs):
kwargs = {**first_kwargs, **second_kwargs}
return func(*first_args, *sec_args, **kwargs)
return inner
I guess that makes sense now.
I don't see many uses for it though but I guess that's arguable.
Thanks for the tip.
hollow crane
might help to think of it as like classes for functions
although __call__ is a thing already
boreal umbra
grizzled vigil
@true cave @north root A common use for functools.partial() is implementing callbacks, e.g. passing a function with it's args and kwargs to be called at a later point in time. Callbacks have pretty much an infinite number of use cases, particularly in async (although with asyncio it more so revolves around the task-based model rather than a callback-based one like Twisted). Outside of async, you can use it for things like registering cleanup calls (to be invoked at the end of the program to ensure resources are safely finalized).
To point to a specific example, I recently used it for implementing asyncio.to_thread() in the upcoming Python 3.9.
true cave
@grizzled vigil Thanks, I'd have to check how you used it in asyncio but I was able to use it to pass a whole function too for now. One thing that confuses me is that, if I wanted to get the return of the passed function, I have to call it with .__call__() method, which I think is kind of dumb.
true cave
So this is more the example that I'm working with.
def final_task(x: int, y: partial):
z = y.args[0]
x = x + y.func.__call__()
return x + z
def task1():
return 5
def main():
final(5, partial(task1, 2))
In that example, can you just call y()?
deft pagoda
y.func()
grave jolt
y.__call__ is the method that's called when you do y()
true cave
I figured that just now 🙂
raven ridge
the thing that makes magic methods magic is that the interpreter calls them automatically in particular contexts. The context that makes __call__ get called is when the user tries to call the object like a function, as indicated by a set of parentheses after it.
true cave
So basically when calling any function like x() in the background it's x.__call__()
raven ridge
yep.
true cave
Just like __add__ and __radd__
raven ridge
just like how a+b is type(a).__add__(a, b)
true cave
Makes sense.
pseudo meadow
how, but then wouldn't it be function.__call__.__call__.___call__ and so on infinitely?
deft pagoda
no
bronze garden
>>> def foo():
... return "hi"
...
>>>
>>> foo.__call__
<method-wrapper '__call__' of function object at 0x10aab3c80>
>>> foo.__call__()
'hi'
>>> foo()
'hi'
>>> foo.__call__()
'hi'
>>> foo.__call__.__call__.__call__()
'hi'
>>> foo.__call__.__call__.__call__.__call__()
'hi'
tawdry gulch
Self referencing is a deepcopy’s worst nightmare : /
primal pond
Uh
unkempt rock
LOL
true hollow
As I can see, __call__ is the thing that get executed on a function, right?
unkempt rock
Perhaps.
cloud crypt
I bet call is essentially exec(__code__) but with some addition
unkempt rock
proc = subprocess.Popen(["python", "preferences.py"])
proc.wait()
flat gazelle
Instead of using python as the first element of that list, use sys.executable to support running from a venv, with a different python version and such
terse pivot
what are some of the advance python books ?
cloud crypt
Fluent Python, probably?
subtle pike
tbh advanced books are mostly specialised applications of python
flat gazelle
!resources has intermediate/advanced books too
fallen slateBOT
subtle pike
If you want to learn more about the language itself, read the docs thoroughly. Learn about the built-in modules, the odd behaviours, look at implementation details
terse pivot
That’s a great tip , how to look at implementation ? Can some one throw an example ? How hard is it do dig up python source ?
subtle pike
The CPython repo is on GitHub
if you're comfortable with C then it's really easy to read
Well written and documented
terse pivot
oh cool , I thought cpython was some kind of library of c for python did not knew it’s source code of python itself
subtle pike
it's the reference implementation
terse pivot
C and C++ are my fav languages
subtle pike
good stuff
terse pivot
cool , any books which print python standards ?
subtle pike
nah, the docs are all online
same ones you read to get built-in help
it has a language reference
terse pivot
Cool , thanks a million 🙏
subtle pike
nw
subtle pike
does your response actually have json content?
proud summit
yeah @subtle pike
subtle pike
and you're sure it's valid?
proud summit
yeah
subtle pike
I was trying to walk them through figuring it out themselves :(
hollow crane
can you show what the raw content is
cloud crypt
hollow crane
the json
flat gazelle
I mean, the python json module does generate invalid json by default due to Inf, -Inf and NaN
flat gazelle
Try printing out the actual response bytes
proud summit
alright cheers
heady axle
@proud summit Be careful when sharing API key
proud summit
whoops
ancient ridge
done
heady axle
You better revoke the key
proud summit
its got nothing on it
ancient ridge
oh
heady axle
OK, fine then ^^
ancient ridge
gotem
proud summit
u underestimate me
proud summit
it wasnt just my api key
it was a line of code
for troubleshooting
@ancient ridge thinks he's superior because he knows how to use an API
ancient ridge
uh oh
dudebro
why is dudebro such a bully
@ancient ridge thinks he's superior because he knows how to use an API
@proud summit im sorry if you really feel bad
gloomy rain
@proud summit @ancient ridge Drop the namecalling and the snide remarks. If you can't be respectful to each other, I'll put you both on a timeout.
proud summit
@hollow crane yeah, it worked. Thanks man
proud summit
@ancient ridge I'm only kidding, jokes aside
gloomy rain
Thanks.
ancient ridge
i thought you were joking as well
proud summit
im a csgo player bro
gloomy rain
👍
ancient ridge
whats beaning
cloud crypt
banning
hybrid crag
We like nice
subtle pike
In [34]: def f(f):
...: def g(*args, **kwargs):
...: signature = inspect.signature(f)
...: bound = signature.bind(*args, **kwargs)
...: bound.apply_defaults()
...: print(bound.arguments)
...: print(bound.kwargs)
...: print(bound.args)
...: return f(*args, **kwargs)
...: return g
...:
In [35]: @f
...: def foo(a: int, b: int = None, c: str = "t"):
...: print(a, b , c)
...:
In [36]: foo(1)
OrderedDict([('a', 1), ('b', None), ('c', 't')])
{}
(1, None, 't')
1 None t
why is bound.kwargs empty? The documentation states:
kwargs
A dict of keyword arguments values. Dynamically computed from the arguments attribute.
which doesn't seem to be the case givenbound.argumentshas (key,value) pairs yet no kwargs are produced
spark magnet
@subtle pike you didn't provide any arguments as keyword argumentds.
subtle pike
In [39]: foo(1, b=4)
OrderedDict([('a', 1), ('b', 4), ('c', 't')])
{}
(1, 4, 't')
1 4 t
i thought that was the case earlier but as you can see, the same behaviour is present
spark magnet
hmm, i guess i'd need to read the docs
subtle pike
the same behaviour replicates for CPython
i'll need to look into this later, for now a dict comp works but i'm surprised by the empty kwargs attrib
if anyone has any ideas please ping me, i'll be afk for a while
spark magnet
@lone wraith btw, ipython is just a different shell around CPython anyway
magic python
Can anyone explain how they use pdbrc? I have no idea what @swift imp was on about here #internals-and-peps message , I use a pdbrc for config setting - i don't see how that avoids using breakpoint() in the code though
tacit hawk
I have a strange problem. My test pass when the debugger is running without breakpoints and does not lass when the debugger is not running. I am using VSCode debugger and unittest.
true ridge
In [34]: def f(f): ...: def g(*args, **kwargs): ...: signature = inspect.signature(f) ...: bound = signature.bind(*args, **kwargs) ...: bound.apply_defaults() ...: print(bound.arguments) ...: print(bound.kwargs) ...: print(bound.args) ...: return f(*args, **kwargs) ...: return g ...: In [35]: @f ...: def foo(a: int, b: int = None, c: str = "t"): ...: print(a, b , c) ...: In [36]: foo(1) OrderedDict([('a', 1), ('b', None), ('c', 't')]) {} (1, None, 't') 1 None twhy is
bound.kwargsempty? The documentation states:
which doesn't seem to be the case givenbound.argumentshas (key,value) pairs yet no kwargs are produced
@subtle pike I checked the implementation and looks likekwargsonly contains something when a function signature has either a vararg or some keyword only args.
tacit hawk
It also does not pass when running directly with python -m unittest
So the debugger has some side effect, how can I debug when the test pass while debugging? lol
I am using threads. There is a boolean value that mysteriously becomes True when debbuging and is False when not. That value is private to the test method.
spark magnet
@tacit hawk sounds like a timing-based race condition
tacit hawk
The main thread joins a secondary thread that tries to lock a mutex in non blocking mode
The boolean value tells whether that was locked or not
raven ridge
The change may just be that the program runs slightly slower when the debugger is enabled, and the extra delay makes things work.
tacit hawk
The test still fails even adding sleeps before the assertions
This is the test code
locked = queue.Queue(maxsize=1)
def trylock(locked):
locked.put(self.mutex.lock(blocking=False))
t = threading.Thread(target=trylock, args=(locked,))
t.start()
result = locked.get()
self.assertIsInstance(result, bool)
self.assertTrue(result, 'Deadlock happened')
the last assertion pass when debugging and fails when not
and the whole code does not knowns whether it is being debugged or not
magic python
this is from seaborn, what does * mean here?
tacit hawk
you must pass arguments as keywords
tacit hawk
python allows you pass arguments by position or by name
magic python
oh - so with * as first arg they have to be by name?
tacit hawk
catplot(1, 2, ...) would not work
all the args after * must be passed by name/keyword
catplot(x=1, y=2, ...) is valid
magic python
In [1]: def check(*, x, y):
...: print(f"x = {x}, y = {y}")
...:
In [2]: check(1,2)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-2-09f02a713be1> in <module>
----> 1 check(1,2)
TypeError: check() takes 0 positional arguments but 2 were given
In [3]: check(x = 1, y = 2)
x = 1, y = 2
In [4]:
indeed - i have never seen this before - handy
tacit hawk
generally that is enforced when the function takes many arguments. That improves readability
magic python
tacit hawk
enforced by the developer, not by python
magic python
yeah - i like this, good to know, thanks
any downsides to it? I can't see why i wouldn't want to use it to be honest
tacit hawk
I mean, that is optional
magic python
sure - i just always explicitly state them when i use them anyway, so i don't see why i wouldn't ever enforce it
tacit hawk
For me that is an option when the arg's count is grater than 3
magic python
fair, i've only recently really been writing anything that isn't thrown away pretty shortly after the results are gathered, so am starting to try and consider better software practices etc
tacit hawk
If I am not wrong, Python 3.8 added the option to enforce positional-only arguments with /
undone hare
Yup, you're right :)
raven ridge
It's for cases like def sin(x) where you really don't want people to call it as sin(x=15). It also lets you implement a feature that's otherwise very difficult to implement: perfect forwarding of keyword arguments.
magic python
what does perfect forwarding mean
raven ridge
consider dict.update() - it's implemented in C, but if it were implemented in Python it would have a signature like def update(self, **kwargs): and what it does is:
lost nexus
whats / for in these parameters?
raven ridge
!e
d = {}
d.update(a=1, b=2, c=3)
print(d)
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
{'a': 1, 'b': 2, 'c': 3}
if that were implemented in Python in the way you'd expect, it wouldn't be posible to do d.update(self=42) because self is already being passed positionally, as d
deft pagoda
you can change the name of a positional-only argument as well without messing up someone's code
magic python
this is different between * and / ?
raven ridge
they do different things. * means that an argument after it can only be passed by name, not by position. / means that an argument before it can only be passed by position, not by name.
magic python
oh ok, i don't know i'd ever want the latter
i guess it must be useful for some though
raven ridge
!e
def print_definitions(prefix=" ", **kwargs):
"""Pretty print a list of words and their definitions."""
for word, definition in kwargs.items():
print(word, ":\n", prefix, definition, sep="", end="\n")
print("") # print a blank line after the list
print_definitions(night="A castle's protector", thrown="A castle seat")
print_definitions("-> ", barred="A singer of tales")
print_definitions(baguette="A tiny bag", prefix="A one-price-fits-all meal")
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
001 | night:
002 | A castle's protector
003 | thrown:
004 | A castle seat
005 |
006 | barred:
007 | -> A singer of tales
008 |
009 | baguette:
010 | A one-price-fits-all mealA tiny bag
^ That example illustrates what I'm talking about with forwarding of arguments. The last line doesn't do what the user meant for it to do, and it's because the prefix word that they were trying to pass was consumed by the prefix keyword argument. Making that argument positional only fixes the program.
!e
def print_definitions(prefix=" ", /, **kwargs):
"""Pretty print a list of words and their definitions."""
for word, definition in kwargs.items():
print(word, ":\n", prefix, definition, sep="", end="\n")
print("") # print a blank line after the list
print_definitions(baguette="A tiny bag", prefix="A one-price-fits-all meal")
peak spoke
Most cases would be where you want arbitrary kwargs but also have a normal argument, which cannot shadow a kwarg that can be used
fallen slateBOT
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
001 | baguette:
002 | A tiny bag
003 | prefix:
004 | A one-price-fits-all meal
i think that i never use kwargs probably limits my appreciation here
raven ridge
They're heavily used in decorators, for instance. There are places where you want to wrap another function without knowing anything about what arguments it takes, and you want to be able to forward any arguments you receive to it, but possibly take an argument or two of your own.
magic python
i never use decorators either 🤦♂️ they're definitely on my "todo" list though
raven ridge
functools.partial is another good example: it takes a function, and some positional arguments to call it with later, and some keyword arguments to call it with later. If the function itself is allowed to be passed by keyword, that's one keyword that could never be passed to the wrapped function, because it would always be swallowed by the wrapper.
magic python
oh, i've used that
like s = partial( pd.DataFrame.sample , n ) iirc
something like that, so i could use s(10) to sample 10 rows
raven ridge
so, functools.partial does something akin do:
def partial(func, *bound_args, **bound_kwargs):
def func_with_some_args_already_bound(*args, **kwargs):
return func(*bound_args, *args, **bound_kwargs, **kwargs)
return func_with_some_args_already_bound
and that needs positional-only params to implement properly, because without them it would be impossible to ever bind a keyword argument named func for the wrapped function, because it conflicts with the func parameter for the partial function itself.
cloud crypt
partial is a class that lets you access some of the things
magic python
i should bother using these properly, never really needed them... maybe they make some stuff easier though
cloud crypt
something like keywords exist for a reason imo
partials make stuff easier yeah
fun example, aiohttp uses them for chaining middlewares
and getting original handler requires tracing stuff via partial.keywords
raven ridge
partial is a class that lets you access some of the things
sure, that's true, but not relevant to what I'm illustrating here. The class__init__still needs to use positional-only arguments, for exactly the same reason as my imitationpartialdoes.
raven ridge
there also was a hack available in older versions of the language to work around this problem - you would not give any of your positional-only parameters names, and you would instead manually unpack them from *args. Uglier and harder to read, and gives worse error messages when you call it with invalid arguments, but accomplishes the same goal.
partial now uses positional-only arguments:
https://github.com/python/cpython/blob/master/Lib/functools.py#L282
but it used to use that hack:
https://github.com/python/cpython/blob/3.6/Lib/functools.py#L248-L253
raven ridge
I've never needed to use it. It seems to be mostly useful for writing a REPL of your own, where you don't want to allow arbitrarily long repr's to flood an entire screen.
cloud crypt
and also not get recursion errors with displaying self-containing lists :p
or tuples #esoteric-python moment
raven ridge
the implementation of the repr() function handles that automatically, AFAIK
as long as you don't try to create your own repr for a list or a tuple, there's no problem with implementing your class's repr in terms of list or tuple's repr, even if there's a a recursive list or tuple in it.
tacit hawk
are there alternatives to the pdb debugger?
subtle pike
@true ridge good catch, the docs were ambiguous for me. Thanks.
raven ridge
@cloud crypt ```python
In [4]: class X:
...: def init(self, lst):
...: self.lst = lst
...: def repr(self):
...: return f"X({self.lst!r})"
...:
In [5]: X([1, 2, 3])
Out[5]: X([1, 2, 3])
In [6]: lst = []
In [7]: lst.append(lst)
In [8]: lst
Out[8]: [[...]]
In [9]: X(lst)
Out[9]: X([[...]])
wait, also - lists can be recursive, but tuples can't. 🙂
@tacit hawk there are others, though I've never used any. https://wiki.python.org/moin/PythonDebuggingTools#Debuggers
magic python
@tacit hawk ipdb is what i use
hollow crane
magic python
i have no idea why anyone would choose to use either pdb or python over ipdb or ipython to be honest
raven ridge
is there an ipdb equivalent of import pdb; pdb.set_trace() ?
tacit hawk
I need to check a debugger that does not uses any of the pdb API
magic python
@raven ridge i don't know why one would use that instead of breakpoint() either
tacit hawk
since pdb is bugging the code
raven ridge
@magic python well, same question - is there an idiomatic way to, on one line, break into an ipdb debugger instead of a pdb one?
magic python
breakpoint() puts me in
raven ridge
export PYTHONBREAKPOINT=ipdb.set_trace does it, it seems.
magic python
I have
export PYTHONBREAKPOINT=ipdb.set_trace
in my ~/.zshrc
terse pivot
can we use assembly int 3 ?
magic python
sorry, i didn't fully grasp the question for a bit there - but yeah - i have that line in my zshrc and then just use breakpoint as usual
raven ridge
@tacit hawk if your code is in a state where someone else can run it, I can help you take a look. I'm certain that it's not pdb itself that's causing you problems, it's a latent problem in the code that is exposed by the debugger.
magic python
I also have
import IPython
from traitlets.config import get_config
cfg = get_config()
cfg.InteractiveShellEmbed.colors = "Linux"
cfg.InteractiveShellEmbed.confirm_exit = False
in my pdbrc
raven ridge
I'm an old timer who is slowly coming around on ipython. I'll have to give ipdb a shot.
magic python
its much nicer, similarly with iypthon and python i guess
you get completion etc
sorry I realise i cut my rc file short there @raven ridge
import IPython
from traitlets.config import get_config
cfg = get_config()
cfg.InteractiveShellEmbed.colors = "Linux"
cfg.InteractiveShellEmbed.confirm_exit = False
alias interactive_ipython IPython.embed(config=cfg)
the alias at the bottom is quite important, for me at least
so - from within ipdb you can type interactive_ipython (or whatever you set it to), and then you'll be in a ipython session
tacit hawk
pdb cant break in other thread?
cloud crypt
lists can be recursive, tuples can not
@raven ridge are you sure about that :p
>>> import ctypes
>>> nekit = (13,)
>>> nekit
(13,)
>>> ctypes.c_longlong.from_address(id(nekit) + 24).value = id(nekit)
>>> nekit
((...),)``` well, the GC doesn't track tuples, so what you've made there is a reference cycle that can never be reclaimed.
hollow crane
either ctypes moving or messing around with dunders
raven ridge
don't break the virtual machine 😄
cloud crypt
I love doing it
magic python
what's the point in it
raven ridge
there isn't a point to it; it's an example of something horrible, heh
cloud crypt
and have you heard that PyTuple_SetItem exists?
hollow crane
unfortunately python seems to built sensibly enough that there aren't too many really destructive things you can do with dunders
raven ridge
and have you heard that PyTuple_SetItem exists?
@cloud crypt yes, but it's invalid to call it if the user has ever gotten a reference to that tuple.
hollow crane
i need to find more stuff to do like this
and subclassing Exception into SomeBullshit so you can just catch SomeBullshit and stop it erroring
i was trying to make a brainfuck interpreter in python
like
like this hang on
tacit hawk
@raven ridge I think vscode does not uses pdb, so I was wrong about pdb being the problem. I ran pdb directly and the test did not pass
hollow crane
sadly the brackets are necessary because otherwise python makes it into an a < b < c and messes it up
tacit hawk
when trying to add a breakpoint in a thread I get ValueError: signal only works in main thread
tacit hawk
I think pdb cant debug threads, so I will use gdb with the debug build of python
raven ridge
@tacit hawk again, if you can share the code with the bug, I can probably help you spot the problem.
everything that you've shown looks fine, so presumably the issue is in self.mutex.lock
cloud crypt
@hollow crane ngl if I ever were to write inline brainfuck, I would use # coding: bf trick
tacit hawk
@raven ridge do you know pthread mutexes?
raven ridge
yes.
@tacit hawk though if you want help debugging this, a help channel or #async-and-concurrency would be more appropriate; it's not really on-topic for this channel.
grim quest
anyone here know how to use beautiful soup?
gloomy rain
@grim quest This channel is for indepth discussion about the Python language. If you need help, please check out #❓｜how-to-get-help .
red solar
@raven ridge in cython is there a way to have different code for different platforms? Without checking the platform every function call?
peak spoke
Do it at the start and then initialize your module depending on that check
red solar
oh hmm
decent idea
if i can also conditionally include header files using if/else, then i'm set 🙂
raven ridge
red solar
oh awesome 🙂
raven ridge
I noticed a bug in what I wrote the other day, also: https://github.com/godlygeek/py_atomics/commit/7fc2745cb42186a43f80ff47d621cdd595b651e5
raven ridge
otherwise it lets you pass in a bytes object and mutate it, which isn't what we want. 🙂
nope, it works, and mutates immutable objects, breaking interpreter invariants.
raven ridge
nah - the entire concept of "immutable objects" only exists at the Python level. The C API is allowed to mutate anything, including tuples and byte strings. It's the programmer's responsibility to not mutate them if the user could be holding a reference to them.
red solar
oh what, cython doesn't let you have actual const stuff in C? :/
i mean idk why i'd need it, but still...
raven ridge
well, see the one-line change in that commit - if I tell Cython I need a mutable buffer, it will enforce that for me.
red solar
ok so i'm gonna try and build on yours, but perhaps make it more complete - as an example, on x64 processors, you can have atomic 128bit ints - but you only get them lock free if you compile with -march=native, which ofc you can't in python - so you have to use intrinsics and check if they're available at runtime
it will enforce it being mutable?
it will enforce it in the rest of your python code, or in the rest of your c code too?
raven ridge
but you only get them lock free if you compile with -march=native, which ofc you can't in python
You can with just-extra-compile-args=-march=nativein thesetup.py
it will enforce it being mutable?
In the same way as it enforces that a Pythonintpassed to a Cython function takingint16_tis in the range of anint16_tand otherwise throws an error, it enforces that a Python buffer object passed to a Cython function declared as taking a non-const memoryview must mutable.
red solar
don't think i wanna do that tho (also even with -march=native, clang will make it lock free by using lock cmpxchg16b directly, gcc won't be lock free since it will still delegate to a library call)
ah ok
raven ridge
after fixing that bug, trying to pass in a bytes object gives:
BufferError: Object is not writable.
don't think i wanna do that tho
Hm, what's the harm? All of my extensions compile with-march=native.
although - well, it does get trickier if you plan to ship wheels.
red solar
No harm, just since even March native isn’t enough to get desired behaviour, no point in requiring it
raven ridge
ah, got it.
I have trouble even imagining what an atomic 128-bit number could be useful for, heh
in any event, for taking that past proof of concept stage, it would probably be best to split the three sections - header/per-int-type/footer - into 3 separate .pyx files instead of 3 string literals, so that you can edit them with syntax highlighting and such, heh
that was a bit quick and dirty for showing the simplest way I could think of for generating the Cython code for each type.
red solar
128 bit uuid
raven ridge
Oh, I understand what a 128-bit type is useful for - I don't understand why you'd ever want atomic operations on it.
128 bits is as large as a uuid - that's beyond human comprehension. it's not reasonable to treat them like numbers, in most cases.
red solar
so that you can have it in shared memory without needing to explicitly lock it?
raven ridge
I can't imagine a use case for even atomically storing or loading a uuid...
red solar
i mean you'd mostly just compare exchange it, you wouldn't need to add or sub
raven ridge
if you're generating a uuid for something and someone else needs to know it, it needs to also not be overwritten before they can read it, so you need some sort of synchronization - atomics aren't sufficient.
I guess you could use it to implement a sort of spinlock, initializing to 0 and then atomically reading until the creator of the uuid atomically sets it - but, that's pretty nutty.
red solar
i had a use case for it in c++, trying to think if it would translate to python without needing a lock
wild harness
i made a python package but i need to be able to import functions from the file after installing it
how do i do it?
like it does not recognise it as a python module
nvm got it
north root
we don't allow that here @unkempt rock
unkempt rock
aww
red solar
How is java a platform than python can run on?
brave badger
You take your Python code and you compile it to Java bytecode
That's how Jython works I believe
grizzled vigil
Indeed, see the FAQ: https://www.jython.org/jython-old-sites/archive/22/userfaq.html#what-is-jython
Oh, that's an archived version of the site, it looks like the latest one is hosted on the python.org wiki: https://wiki.python.org/jython/JythonFaq (that part still applies though of course)
red solar
Ok... that makes sense
But how is java a platform for cpython?
Hmm actually I may have just assumed they meant cpython when they meant python
They used both in the docs and it confused me
grizzled vigil
I think that's most likely the case (that it was not specific to cpython).
grizzled vigil
Oh sorry, I must have missed that earlier, but I see it now. I honestly have no idea why we don't currently support atomics. My knowledge of the C-API is general is fairly limited, and even more so about what libraries we choose to support based on compiler compatibility
red solar
It’s ok I found the documentation that says what compilers and architectures are supported
grizzled vigil
Oh yeah that's not an issue to find, I meant the reason as to why we support the ones that we do and what actually went into those decisions.
red solar
Hopefully someone might be able to answer it (since i’m including it at the bottom of my bytearray question)
grizzled vigil
I tried to cc capi-sig@python.org, but it's a subscriber-only list (that I'm not personally subscribed to), so you have to register to start a discussion there.
I'd recommend creating a new email and sending it there.
red solar
Doing it rn 🙂
grizzled vigil
It might take a little bit to get answered, but I know that we have several core devs that are active in there (fully voluntarily, of course) that regularly work w/ the C-API.
north root
@worldly cove we don’t allow recruitment here
north root
okay see #❓｜how-to-get-help
worldly cove
Ah ok
red solar
Convert the least-significant 8*n bits of long
v to a base-256 integer, stored in array bytes.
i'm confused as to what this means
(long means PyLong here)
base 256 just means it's in byte chunks? (since each byte is 0-255)
worldly cove
I made a pokecord bot with all commands but in trade command when a user has x bal and it add his whole x bal into y users bal he can add x again which makes the bal go in minus
solar shell
how tkinter's Scrollbar widget calculates 'fraction' for scrolling?
minor sinew
how does one even go about compiling python to use in other languages? the only other language that i use that isn't statically compiled is matlab, and it's an expensive process since it's a commercial language
flat gazelle
generally, you do not call python from other, but other langs from python. You can use the CPython C api to do so though AFAIK
raven ridge
It's also possible to embed an entire Python interpreter into another process, and feed it input and extract output from it using a C API.
hollow crane
that's generally the main solution i've seen since that means you can use it as a scripting language
raven ridge
From what I've seen that's actually the more difficult route, but both options are possible, depending on which of the two languages you want to drive things from.
magic python
is there a way to see where all processes go? Like - get a tree of all the modules that were called and in what order
raven ridge
If you ask this in #internals-and-peps you'll get an advanced answer: Python has both a line tracing API that you can hook into (like coverage and profile do) and an audit events system that can tell your when certain significant operations have been performed.
magic python
@raven ridge cool - what's it called? can it just be run or does it require config/setup
raven ridge
flat gazelle
is
try:
...
except SystemExit:
raise
except KeyboardInterrupt:
raise
except BaseException:
"error handling code of sorts"
or
try:
...
except BaseException as e:
if isinstance(e, (KeyboardInterrupt, SystemExit)):
raise
else:
"error handling code"
``` bettertry:
...
except (SystemExit, KeyboardInterrupt):
raise
except BaseException:
"error handling code of sorts"
what's the goal of this? To do the error handling on GeneratorExit and Exception, but propagate the exception on SystemExit and KeyboardInterrupt?
final whale
catching baseexception and not exception ?
raven ridge
this should either just be:
try:
...
except (Exception, GeneratorExit):
...
``` or ```python
try:
...
except Exception:
...
some things are not ever supposed to crash the program and may possibly be from an external source, but you also do not want to stop exit() and ctrl+c from terminating. Though you can probably just use except exception instead and be fine in most cases
but it is pretty niche
raven ridge
@flat gazelle the only reason to ever catch BaseException is if you want to capture SystemExit, KeyboardInterrupt, or GeneratorExit, instead of letting them propagate. Since you almost always want them to propagate, you almost certainly want to be always doing except Exception instead, which already won't capture those types.
green musk
if you have important things to do before exiting, you might want to catch them, do that thing, and then raise
(saving a state, closing a file you don't want corrupted, cleanup temporary data, whatever)
raven ridge
that should almost certainly be done with a finally block or a context manager, or possibly an atexit handler, rather than catching and re-raising SystemExit and KeyboardInterrupt, because you want the cleanup to be unconditional, not conditional on the reason that the process is exiting.
flat gazelle
there is the also the option of some other programmer subclassing BaseException rather than exception
but that is bad form in most cases
raven ridge
if they did that, it's because they didn't want you catching it, which probably means that you shouldn't.
and if you want to, you can always catch that type of exception explicitly.
unkempt rock
Hey. I am trying to edit a CSV record. The way I want to do this is by the person selecting the persons first name from a dropdown, which i have working. Then i want it to pull the record from the csv and display it. How can I go about getting the record for the first name selected.
raven ridge
@unkempt rock #❓｜how-to-get-help
unkempt rock
I opened one in #help-cake
raven ridge
then wait patiently for someone to help.
mortal dagger
Hey guys. New here so sorry if this isn't the right place for this but.
Can someone tell me what the select library is what does it do with queue and sockets. Like why does everyone use select when making a multi client server? I've always been confused.
red solar
@mortal dagger people use select directly? they don't use selectors?
tacit hawk
is it possible to hook into a thread exit?
raven ridge
@tacit hawk a thread that you started, or an arbitrary one, or every one?
tacit hawk
an arbitrary
tacit hawk
is it ok to use the threads' local storage to attach objects' lifetime to the thread lifetime?
grizzled vigil
@mortal dagger
Can someone tell me what the select library is what does it do with queue and sockets. Like why does everyone use select when making a multi client server? I've always been confused.
It's quite a bit to break down, but the gist of it is that select/poll/epoll are used to check if a file descriptor (used for open system files, sockets [network or local inter-process], pipes, or anything I/O related) or group of them are ready to write/read to without blocking the program flow (in the current thread). If you were to directly call the read/write operation (assuming the file descriptor isn't set to non-blocking), it would block until the file descriptor is available, which is frequently undesirable when performance is a significant concern.
For the default event loop implementation that most people use for asyncio on unix (SelectorEventLoop), it makes calls to select under the hood.
tacit hawk
where is the source of the _thread C module?
seems to be this https://github.com/python/cpython/blob/252346acd937ddba4845331994b8ff4f90349625/Modules/_threadmodule.c
forest flicker
Can pyinstaller be used to convert any kind of script into an executable? Are there certain scripts that cannot be converted into executable?
prime estuary
@forest flicker Ideally yes. The problem is when either you have C extension modules (that it can't locate or copy over properly), or regular modules imported in unusual ways that it cannot detect.
forest flicker
@forest flicker Ideally yes. The problem is when either you have C extension modules (that it can't locate or copy over here properly), or regular modules imported in unusual ways that it cannot detect.
@prime estuary
What's meant by importing modules in unusual way?
prime estuary
For example, if you used exec("imp" + "ort " + "some_" + "module"). Which works fine in Python, but PyInstaller's not going to be able to recognise there's an import there.
Or more simply if you called importlib.import_module("modulename").
forest flicker
For example, if you used
exec("imp" + "ort " + "some_" + "module"). Which works fine in Python, but PyInstaller's not going to be able to recognise there's an import there.
@prime estuary not a problem for me. I always useimport <module>I don't even know modules can be imported in the way you described.
seems like pyinstaller is a good way to hide source code for those who're sensitive about it
prime estuary
Not really, it basically just includes the code in the EXE.
forest flicker
Not really, it basically just includes the code in the EXE.
@prime estuary is the code obfuscated or minified like minified js code?
prime estuary
It's bytecode format, so a little yes. But it's fairly easy to understand still, all the important info is there.
forest flicker
It's bytecode format, so a little yes. But it's fairly easy to understand still, all the important info is there.
@prime estuary talking about hiding code. Dropbox was written in python. Yet, I don't see any source code. Is it because they used stuff like pyinstaller? I wonder how these big companies protect their source code when software is written in python
prime estuary
I'm not sure.
undone hare
I'm pretty sure that you can extract the source code or at least the bytecode from a pyinstaller executable pretty easily
forest flicker
I'm pretty sure that you can extract the source code or at least the bytecode from a pyinstaller executable pretty easily
@undone hare the question is whether effort to reverse engineer is worth it compared to developing on your own
undone hare
Oh, the python bytecode is really clear, you don't have any reverse engineering to do, you can get a pretty similar source file with variable names and stuff by decompiling it
So I'd say that in most cases it is worth it
glass robin
dropbox only use python for its backend IIRC
forest flicker
Oh, the python bytecode is really clear, you don't have any reverse engineering to do, you can get a pretty similar source file with variable names and stuff by decompiling it
@undone hare oh... didn't know that. I thought it will at least be obfuscated or minified. Is the bytecode indented and easy to read almost like original source code? Variable names remain the same?
undone hare
No, it would be still hard to read, but the variable names has to stay yes
raven ridge
If the source code is obfuscated, the byte code will be more difficult to read. The byte code closely matches the source code.
glass robin
but if the source code is too obfuscated, pyexe is more likely not to understand it either
undone hare
Basically, when you access the attribute foo of an object bar, python will look into the dictionary-like representation of the object bar and search for the string 'foo', that's why you have to keep the variable names
Well, it can be hard to follow sometimes with all the stack operations godly
pastel crest
isn't foo and bar metasynthetic variables used to store info
cedar glen
of course not
undone hare
Yup they are
red solar
Tf does metasynthetic variable mean?
undone hare
It is an equivalent for placeholder variables
sturdy timber
I think they meant metasyntactic
red solar
oh
timid orbit
Oh, the python bytecode is really clear, you don't have any reverse engineering to do, you can get a pretty similar source file with variable names and stuff by decompiling it
Clearly you haven't seen my bytecode optimizer library
undone hare
Well sure haha
forest flicker
I assume running from a executable compiled by pyinstaller will be much faster than running from a script. IS that correct? faster means noticeably faster
flat gazelle
no, pyinstaller afaik just packages the python exe with your code
nuitka may be faster, but intel python or pypy or numba are generally the main ways to speed up python depending on the specifics
forest flicker
no, pyinstaller afaik just packages the python exe with your code
@flat gazelle i'm surprised. Do you mean no performance gain?
flat gazelle
I think pyinstaller essentially just creates an exe that does some_embedded_python_exe your_code.py
forest flicker
I see. I guess pyinstaller is a solution for deployment problems, not performance problems. Thanks for your answer
glass robin
Depending on how specific your problem is and how much performance do you need it might be easier to switch from language
hollow crane
@forest flicker are you thinking of pypy perhaps?
polar pecan
is there an algorithm/function that returns all possible solutions to write a word in different upper and lower letters? Like: de De dE DE. Maybe in a list or sth
open trout
you could just literally do that with mathematical permutations
though that sounds like mumbo jumbo
flat gazelle
I don't think permutations are useful here. You could do from 0b00 to 0b11 and use each bit to determine whether a char should be upper or lower
visual shadow
Seems easy enough to write one, I'd personally probably go for itertools product approach. Have thr lower and upper cased versions of the string zipped and just run it through the Cartesian product
sacred tinsel
yeah, I'm just writing that out lol
>>> string = "abc"
>>>
>>> import itertools
>>>
>>> for mask in itertools.product([0, 1], repeat=len(string)):
... chars = []
... for n, char in zip(mask, string):
... chars.append(char.upper() if n else char.lower())
... print("".join(chars))
...
abc
abC
aBc
aBC
Abc
AbC
ABc
ABC
Haha nice
sacred tinsel
maybe there's a nicer way to apply the mask
polar pecan
Oh nice, didnt know that 😮 ❤️
sacred tinsel
wanted to oneline it but couldn't find a pretty way
polar pecan
thank you two :))))
sacred tinsel
sorry if I just threw a solution at you, let me know if you have any questions about it
it may not be the best approach but it was the most instinctive to me
polar pecan
I didnt work with zip before
sacred tinsel
zip helps you join multiple iterables, which is super nice
>>> a = [1, 2, 3]
>>> b = [4, 5, 6]
>>>
>>> for x in zip(a, b):
... print(x)
...
(1, 4)
(2, 5)
(3, 6)
the alternative if we didnt have zip would likely be grabbing the other value by the index of the first one
flat gazelle
could also do
for mask_i in range(2**len(string)):
mask = map(int, bin(mask_i)[2:])
or using bitwise operators
sacred tinsel
I think I almost understand that
polar pecan
ah nice thank you 😄
visual shadow
Yeah i personally don't find masks very intuitive, but zip and itertools I can always use intuitively with ease
royal sundial
could you also do
itertools.product(*zip(string.lower(), string.upper()))
```?There's the one I was thinking of
flat gazelle
ye, that is probably neatest
sacred tinsel
I like it too
flat gazelle
it essentially skips the mask
sacred tinsel
really cool solution
paper echo
using * on really long iterables always feels weird to me
north root
lol my Python just took over 10 gigs of my ram running that
paper echo
...did you list it?
north root
yes
polar pecan
I want to check every solution with an dictionary if this solution is in it or not
visual shadow
Oh
That does not seem like the right move
You want to check if a string belongs in this set? Then you can do a much simpler check instead
Just do a lowercase check against the string. If the lowercase of a string matches, then it is guaranteed to be in this set
Basically, checking against this set is equivalent to a case insensitive check
polar pecan
yeah i already check upper and lower but if there is no result i check with every possible solution with upper and lower
visual shadow
Naj
Don't do that
Lowercase both sides of the comparison
This solution is extremely wasteful for that check
Basically, checking against this set is equivalent to a case insensitive check
@visual shadow this is key.
polar pecan
I dont have influence on the dictionary
visual shadow
You can read this dictionary though yes?
Lowercase the strings in memory at runtime. You have control of the actual condition I assume.
polar pecan
okay I give it a try 🙂
visual shadow
👍 Put it this way, making this dictionary is a 2**n operation where all you need is an order of n conversion and then check
(in terms of time complexity. In simple terms, the former is really really slow for large string lengths)
worldly pebble
I have a python class > https://paste.pythondiscord.com/zaxopehome.py
I need to port it to c++ compile the shared object, and use the shared object within Python.
Can someone help me how to do it
The reason being to understand Python Implementation and Python/C++ Hybrid Implementation
polar pecan
Nice, solved the problem, thanks 🙂
tacit hawk
Does python tries to optimize the imports in a sense that only what is used is imported?
flat gazelle
no, the entire module gets evaluated
worldly pebble
Does python tries to optimize the imports in a sense that only what is used is imported?
@tacit hawk You can do that byfrom module import function
minor sinew
i think he was asking if python implicitly did that for you
not if there was a way for you to do it yourself
paper echo
yes it runs the entire module no matter what
pearl river
You can also make your IDE do this, if that's what you mean. But yeah, it certainly isn't done as an automatic optimization.
paper echo
and then caches the module so it's only run exactly once
import json
import attr
import requests
@attr.s
class MyClient:
http_session = attr.ib(factory=requests.Session)
endpoint = 'https://api.example.net/hello'
def query_streaming(query):
response = self.http_session.post(self.endpoint, data=query, streaming=True)
with response:
for line in response.iter_lines():
yield json.loads(line)
my_client = MyClient()
data = list(my_client.query_streaming('hello'))
how does python handle this case w/ the context manager? does it stay open automatically until the iterator is exhausted? or will it be closed immediately and i'll get an error for trying to iterate over a closed response?
presumably the "right way" to do this would be to turn the query_streaming method itself into a context manager and return it
is that reasonable?
onyx garden
!e
fallen slateBOT
You are not allowed to use that command here. Please use the #bot-commands channel instead.
flat gazelle
In [232]: class A:
...: def __enter__(self):
...: self.open = True
...: return self
...: def __exit__(self, *_):
...: self.open = False
...: def op(self):
...: if not self.open:
...: raise ValueError
...: return 5
In [233]: def gen():
...: with A() as a:
...: yield a.op()
...: yield a.op()
In [234]: g = gen()
In [235]: next(g)
Out[235]: 5
In [236]: next(g)
Out[236]: 5
In [237]: next(g)
StopIteration Traceback (most recent call last)
----> 1 next(g)
StopIteration:
```seems to work as expectedinteresting
!e ```python
class A:
def enter(self):
self.open = True
print('Opened')
return self
def exit(self, *_):
self.open = False
print('Closed')
def op(self):
if not self.open:
raise ValueError()
print('Op!')
def gen():
with A() as a:
yield a.op()
yield a.op()
list(gen())
@paper echo :white_check_mark: Your eval job has completed with return code 0.
001 | Opened
002 | Op!
003 | Op!
004 | Closed
neat
that makes life easier
hm thats also an interesting opportunity for resource leaks
def gen():
with A() as a:
yield a.op()
yield a.op()
def gen2():
a_vals = gen()
return 1, a_vals
val, _ = gen2()
this could end up leaving a resource open forever
at least until _ is garbage collected
flat gazelle
where does gen2 get a from?
flat gazelle
well, the context manager never opens like this
paper echo
it doesnt? gen is executed
i guess you'd have to run next on it once
making it a lot more contrived
def gen2():
a_vals = gen()
next(a_vals)
return 1, a_vals
something like that, right?
flat gazelle
ye, that would keep it the generator gets GC'd/python exits, except for os._exit()
paper echo
right. seems unlikely that you'd ever do that
makes sense
and in which case that's the fault of whoever wrote gen2 not the end user
languid dagger
Any function that contains a yield returns as a generator object on call which does not progress until __next__ is called and then pauses immediately after the yield is run
paper echo
right
so as long as you dont start the generator, then stop it midway, then leave it dangling forever
it's fine
raven ridge
Note that in your example, the generator is GC'd as soon as gen2 returns.
paper echo
is it?
raven ridge
Yeah. The reference counting GC gets it, because it's not trapped in a reference cycle.
paper echo
!e ```python
class A:
def enter(self):
self.open = True
print('Opened')
return self
def exit(self, *_):
self.open = False
print('Closed')
def op(self):
if not self.open:
raise ValueError()
print('Op!')
def gen():
with A() as a:
yield a.op()
yield a.op()
def gen2():
a_vals = gen()
return 1, a_vals
val, data = gen2()
list(data)
@paper echo :white_check_mark: Your eval job has completed with return code 0.
001 | Opened
002 | Op!
003 | Op!
004 | Closed
youre saying its GCed because nothing uses _?
presumably it would hang open for at least some period of time before the GC decides to cull it
flat gazelle
it will not afaik. You would need to do del _ after the call
raven ridge
Nevermind, I misread
flat gazelle
if you did not return it, it would indeed get GC'd instantly in CPython though
paper echo
yes that much i know
paper echo
ive deled things before when i have large temporary objects in long running functions
usually big pandas dataframes where im doing aggregations then discarding the original data
flat gazelle
ye, CPython GC being predictable-ish does mean you can optimize things sometimes
paper echo
results = []
for filename in filenames:
df = pd.read_csv(filename)
results.append(df.agg(my_agg_fn))
del df
yeah its nice
flat gazelle
though here it is not useful, because the reference is dropped next loop
paper echo
i know just a silly example
hollow crane
ahh right
raven ridge
What if there isn't a next iteration.
languid dagger
__exit__ seems to be run when the generator is deleted
paper echo
i might have a bunch more stuff down further in the loop
and yes you still have to del after the loop ends
hollow crane
i thought i'd been making horrible mistakes with that lol
languid dagger
!e
class A:
def __enter__(self):
print('Enter')
self._open = True
return self
def __exit__(self, *args):
self._open = False
print('Exit')
# def __del__(self):
# if self._open:
# self.__exit__()
def gen():
with A():
yield 1
yield 2
g = gen()
next(g)
del g```rip
fallen slateBOT
@languid dagger :white_check_mark: Your eval job has completed with return code 0.
001 | Enter
002 | Exit
ye, you cannot bypass exit that easily
hollow crane
doesn't del explicitly call it
languid dagger
I can change it to
def main():
g = gen()
next(g)
main()
And it still calls it
hollow crane
does sys.exit bypass that
paper echo
yeah im not talking about resource leaks outside the application
im just talking about things like leaving HTTP responses or file handles open too long
languid dagger
It does not
peak spoke
os._exit doesn't do cleanups
hollow crane
does an uncaught error still try to wind down context managers
languid dagger
If you do raise ValueError then it doesn't exit the context manager
Since uncaught exceptions by nature interrupt the program and halt it
flat gazelle
uncaught exceptions call __exit__
flat gazelle
you can even use context managers for exception handling by returning True from it
hollow crane
but then like
flat gazelle
In [256]: with A():
...: raise ValueError
...:
Opened
Closed
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-256-68aafbb48064> in <module>
1 with A():
----> 2 raise ValueError
3
ValueError:
what if your init messes up so you don't have a property you expected to have
in some horribly convoluted setup
languid dagger
I was referring more to this example
def main():
g = gen()
next(g)
raise ValueError
If you are still inside the context manager then yea it calls __exit__
tacit hawk
so its better to do from foo import bar than import foo; foo.bar ?
hollow crane
yes, if you only use that submodule
paper echo
in some horribly convoluted setup
like "not using attrs"? 😛
tacit hawk
I like to keep namespaces, I almost never use from
flat gazelle
wait what. Why would that not close it. Does the gen not get gc'd
hollow crane
i misread your first one as 'import foo.bar'
languid dagger
The parameters __exit__ gets are actually related to any exception that happens, it is how the suppress method works (forgot which module it's from)
flat gazelle
contextlib afaik
hollow crane
yes, i would avoid from
paper echo
i use from in some specific contexts
hollow crane
i much prefer keeping namespaces, probably to a greater extent than others seem to
tacit hawk
but that makes python load the entire module
paper echo
from itertools import product
...
languid dagger
Ah yep, contextlib.suppress thanks
hollow crane
you can do import discord.something
paper echo
i used to @hollow crane i got lazy though
do you really want to write discord.ext.commands.Command?
tacit hawk
in that case you could namespace with commands, the module's __doc__ identifies it
paper echo
yes that's the convention they recommend
but i could just as easily want to write my own commands module
tacit hawk
in this case I would make an alias for the dependency
hollow crane
my opinion is it's all stupid so you should just give up and learn to basket weave
fossil jetty
Is there a way to make this feel less ugly:
class Finance:
...
def get_timeseries_data(
self,
securities=None,
features=None,
chunksize=None,
dropna=True,
start=None,
end=None,
frq=None,
):
if securities is None:
securities = self.securities
if features is None:
features = self.timeseries_features
if chunksize is None:
chunksize = self.timeseries_chunksize
if start is None:
start = self.start
if end is None:
end = self.end
if frq is None:
frq = self.frq
i.e. avoid the if kwarg is None, kwarg = self.kwarg
maybe in the same vein of how dataclasses have the
var : type = default
notation?
radiant fulcrum
Its not type hinted anyway so it doesnt make much diffrence
fossil jetty
but then would the var show up in the function sig? My understanding of your answer is this:
class Finance:
...
def get_timeseries_data(
self,
**kwargs
):
securities = kwargs.get('securities', self.securities)
features = kwargs.get('features', self.features)
...
but that makes python load the entire module
@tacit hawk Python can't ever load part of a module. If a module has something like:
x = 10
# 1000 more lines of code
x = 20
``` then `from module import x` needs to execute all 1002 lines of code to know that the `x` that it imports should have a value of `20`.@fossil jetty then rather than doing =None just have the self.xyz there
fossil jetty
@radiant fulcrum You can't do this:
class Finance:
...
def get_timeseries_data(
self, securities=self.securities
):
pass
😦
cloud crypt
you can write some default decorator if you want to achieve this behavior
dark coyote
Is there a good way to detect whether a player is touching a square?
fossil jetty
@cloud crypt How would you write the deco?
cloud crypt
I need to think about it for a bit
fossil jetty
change locals of the function?
cloud crypt
eh no
radiant fulcrum
do those defaults actually need to be in that class tho
or are they just in there for the sake of being in there
fossil jetty
I think so... I have a config.yml that feeds the defaults to the class. Then different methods use those defaults to pull data
but you can also go away from the defaults if yo uwant
unkempt rock
@dark coyote This channel isn't for questions like that, it's for discussions of Python as a language. Check out #❓｜how-to-get-help or ask in #python-discussion
dark coyote
Ah
modern night
@fossil jetty it wouldn't support type hinting as well but you could use kwargs and specify the var name and type in the methods doc string, although editors may not highlight that the type is wrong, they can see in the doc string what types the function expects, then you just iter over kwargs and use setattr to set the attributes from the input, idk if this would work as you want but its a suggestion
fossil jetty
Is there a way to inherit the long code snippet above? Example below:
class Parent:
def method(self, start=None, end=None, frq=None):
if start is None:
start = self.start
if end is None:
end = self.end
if frq is None:
frq = self.frq
do_stuff()
class Child(Parent):
def method(self, start=None, end=None, frq=None):
# if start is None: <-- Get this for free?
# start = self.start
# if end is None:
# end = self.end
# if frq is None:
# frq = self.frq
worldly venture
@spiral willow I'm sure it is somewhere
spiral willow
I just searched
spiral willow
so they put it's not in standard library which could be annoying
worldly venture
It's not at 1.0.0
spiral willow
The pytoml TOML parser is ~300 lines of pure Python code, so being outside the standard library didn't count heavily against it.
worldly venture
so people were arguing it should hit 1.0.0 before it was added
spiral willow
I think it's HUGE against it
worldly venture
However
"As of version 0.5.0, TOML should be considered extremely stable. The
goal is for version 1.0.0 to be backwards compatible (as much as humanly
possible) with version 0.5.0. All implementations are strongly
encouraged to become 0.5.0 compatible so that the transition to 1.0.0
will be simple when that happens."
peak spoke
I remember not liking the implementation of the now recomended toml package
worldly venture
H Bastian,
IMHO we should wait until the format reach version 1.0, since the
stdlib has a slow release cycle (one release every 18 months). Too
slow for a "fast moving" standard.In the meanwhile, I'm sure setuptools and pip will manage to install a
toml parser/generator for their needs, as they already do :-)Victor
there was a discussion on python-dev about this
spiral willow
Sure but cart before horse
spiral willow
TOML seems fine, it seems like bikeshed discussion on file format
just not stdlib could end up with weirdness
worldly venture
yeah
spiral willow
like Step 1) push for TOML in stdlib, Step 2) Push pyproject
but it's also my SRE tick also firing off
does PIP require anything not in standard lib?
worldly venture
that is vendored https://github.com/pypa/pip/tree/master/src/pip/_vendor
spiral willow
got it, so pip would need to vendor TOML code
rich wharf
didn't know there was another conversation going on
worldly venture
lol it's alright
rich wharf
alright
worldly venture
we're not going too fast and I think we are kind of done
rich wharf
for x in [1, 2], y in [3, 4]:
print(x, y)
essentially sugar for multiple loops
you can already do this with loop nesting or itertools.product
worldly venture
so ```py
for x in [1, 2]:
for y in [3, 4]:
print(x, y)
for x in [1, 2]:
for y in [3, 4]:
print(x, y)
hah
rich wharf
yeah
worldly venture
yeah, interesting idea
peak spoke
That just looks confusing to me
narrow kettle
ya that raises questions
peak spoke
I'd interpret it like zip behaviour
deft pagoda
there is some precedence for that idea though, with with blocks
narrow kettle
what if you wanted to interlace them
rich wharf
you'd use zip
narrow kettle
can you have an inner one be based on an ourside loop value
worldly venture
I mean
rich wharf
iterating both of them at once
worldly venture
it's valid syntax
worldly venture
I wonder if I could play with AST to make it work
rich wharf
that can't be
deft pagoda
though i like using product, it's hard to be cleaner than that
worldly venture
>>> for x in [1, 2], y in [3, 4]:
... print(x, y)
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'y' is not defined
how
wait wait
for x in ([1, 2], y in [3, 4])
that's how it's interpreting it
as a tuple
rich wharf
I think it's current usage is bad
You should have to surround it with a tuple for that usage
To make it more clear
Anyways, this would be good for a few reasons
Mainly it would mean you won't have as much indenting
It's a lot more beginner friendly
And it's just convenient
deft pagoda
i wouldn't say it's beginner friendly, i think it's better for beginners to see the explicit nesting of the loops
rich wharf
That's true
worldly venture
I want to make it work lol
rich wharf
Another thing
How about for if loops
for i in range(5) if i > 2:
print(i)
It also removes a layer of nesting, and I'd argue it's also beginner friendly
It also follows a similar pattern to list comprehensions
Also, one more thing, how about return VALUE if CONDITION
deft pagoda
you can use dropwhile
rich wharf
you can also use generators
but I prefer sugar, @deft pagoda
also it's more like filter
not dropwhile
worldly venture
also holy shit ast in 3.9 is fucking nice
rich wharf
anyways
peak spoke
I see both of those as possibly understood in different ways by users
rich wharf
#Proposed
for x in [1, 2, 3], y in [4, 5, 6]: pass
#Current
for x, y in itertools.product([1, 2, 3], [4, 5, 6]): pass
#Proposed
for i in range(5) if i > 2: pass
#Current
for i in filter(lambda i: i > 2, range(5)): pass
for i in (i for i in range(5) if i > 2): pass
#Proposed
return 5 + 5 if 4 > 2
#Current
if 4 > 2:
return 5 + 5
How would that conditional expression work? Or is that only intended for returns
rich wharf
@peak spoke It's a statement
Just like [EXPR for NAME in SEQ [if COND]], it's return EXPR [if COND] and for NAME in SEQ [if COND]
deft pagoda
are we ready to do generator-like lambdas now
rich wharf
@deft pagoda none of these features apply to lambdas
unkempt rock
i need
rich wharf
both of them are statements
unkempt rock
help
deft pagoda
this isn't a help channel
rich wharf
@unkempt rock #❓｜how-to-get-help
rich wharf
@unkempt rock Can I DM you a link to a server with an active JS channel?
rich wharf
This is the Python server, next time don't ask for help like this there
We have off-topic channels
peak spoke
The return is probably the only thing I could see myself using from those but not sure if I like it
For any non trivial condition it'd be better to just have a normal if
rich wharf
I prefer it a lot over if COND: return VALUE
deft pagoda
it's not too much different than if something: return something_ekse
worldly venture
@rich wharf
worldly venture
hacky solution only for two elements though, I wonder if I could make it go further
rich wharf
for x in ([1, 2, 3], y in [3, 2, 1])
Make this work like it did before
for x in [1, 2, 3], y in [3, 2, 1]
Make this convert to the nested loops
worldly venture
hmmm
worldly venture
I wonder how to handle that, it's done before it reaches the visit_For function
rich wharf
and a hint is to use itertools.product
worldly venture
yeah, I want to do that now
rich wharf
x in [1, 2, 3], y in [4, 5, 6] -> x, y in itertools.product([1, 2, 3], [4, 5, 6])
remember to import itertools
deft pagoda
if you really want to still with syntax:
for [1,2,3] as x, [3,2,1] as y:
well this is ast transformation, so imports are irrelevant I guess
rich wharf
I like for x in [1, 2, 3], y in [3, 2, 1]: ...
worldly venture
I'll think of something
digital briar
is this some new python syntax?
worldly venture
no we're just messing around with the python parser to add some of our own syntax