can anyone tell where is the error
#internals-and-peps
proper seal
worn flicker
I have a question, how would I be able to get the last one or two characters of this string: 3 x 2 = 6. I need to be able to get the last two if the last number is a two digit number. How would I use .split() to achieve this
half wolf
@slim island Check out APL, and fortran
slim island
I probably should have clarified modern language. Except for some quirks, Julia looks very nice to work with - mostly because it doesn't require the shift in thinking that numpy does for the smaller stuff. Although I'd also guess just making that shift in thinking would probably be far easier
visual shadow
Numpy is consistent. once it clicks it stays that way.
I've come to appreciate it more and more over time, though it was not intuitive for me at first. Call it an acquired taste 😉
unkempt rock
Numpy is consistent.
@visual shadow
Source?
visual shadow
Me.
gleaming rover
is it not consistent?
wary trout
hi
slim island
I just don't care enough to learn numpy. I'm pretty sure I'll be forced into it, but it's so different from Python - that I start just wanting to go use a different language
but I would really appreciate the Matrix stuff as like a stepping stone I think
slim island
C isn't used in scientific computing at all and provides absolutely no utility for scientific computing
unkempt rock
wrong
slim island
it was hyperbole, and I get that C is where the underlying implementations lie
but it's still not used at the high level in the way R or Numpy is
unkempt rock
I have a PhD in theoretical chemistry. Lots and lots of high-performant code is either written in C or Fortran
slim island
Sure, but I'm not gonna do EDA in C, i'm not gonna do basic linear models in C. Even if I knew C, I'd still want a higher level language to do simple stuff with
paper echo
this sounds like a better debate for #data-science-and-ml 🙂
maybe talking about how numpy is or is not pythonic would be on topic here? idk
slim island
#data-science-and-ml doesn't really support discussion. too well IME But I take your point that it's not really for this channel. This did stem from me asking about an idea for a matrices module in #mailing-lists
paper echo
ok, so on topic here: what would a more pythonic matrix API be, in your opinion?
imo numpy is very pythonic
its object oriented, mostly no-copy data structures, supports iteration, et alia
slim island
I don't really see it. It takes something like indexing, and forces you to learn a new way. It doesn't allow for the use of some of the nice idiomatic syntax like list comprehensions or looping
paper echo
its indexing is a strict superset of python indexing
and it absolutely does allow for looping
its just much slower than vectorized operations
slim island
it allows it - but not whilst remaining idiomatic/fast
Numpy comes with its whole own set of idioms and standards
paper echo
but so does every big complicated library
so i ask again, what would a pythonic matrix library look like?
would you extend array to have n-d arrays & matrix ops backed by blas and lapack?
slim island
There's the mailing list suggestion for it - and I don't necessarily think it's a good idea, but it is interesting to me
paper echo
python looping also can only get so fast, there's a lot of overhead
slim island
I think it's worth reading
The main appeal to me is simpler indexing. Being able to index with two square brackets appeals to me a lot - as that's the intuitive way to me coming from other programming languages and treating matrices as 2d arrays
slim island
Can I do my_numpy_matrix[x:x+10][y:y+3] and get a 10x3 slice?
that's the intuitive way to me
paper echo
but is that more pythonic? or is it just your particular expectation?
im not saying your expectation is bad or wrong, but it doesnt even really make sense from a math perspective and it requires a lot more complicated implementation on the API designer's part
trying to infer whether you're on your 1st or 2nd indexing operation
full jay
@slim island Just to remind you, numpy is written in C
flat gazelle
I prefer the tuple indexing tbh. using multiple brackets seems quite annoying and it would begave differently from regular python sequences, whereas numpy is consistent and extends atop it
paper echo
^^
full jay
So while the majority of the data scientists may not be writing in it, it's still a vital language to use for them
Their tools need that power and speed
paper echo
ok granted, for looping over a numpy object is actually quite a bit slower than a list or array.array
which i actually dont quite understand
it must be that the machinery required to implement __iter__ adds a lot of overhead, but im a bit surprised
slim island
I dunno - I originally just asked for opinions and explained what I would actually prefer. Given that the proposal is for a module that makes matrix stuff more accessible to say high-school students, I think the 2 brackets way is more intuitive personally.
paper echo
In [22]: import numpy as np
In [23]: from array import array
In [24]: data_list = list(range(1000))
In [25]: data_numpy = np.array(data_list, copy=True)
In [26]: data_array = array(data_list)
In [29]: data_array = array('i', data_list)
In [30]: %timeit out_list = [i/10 for i in data_list]
56.3 µs ± 794 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
In [31]: %timeit out_numpy = [i/10 for i in data_numpy]
508 µs ± 40.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
In [32]: %timeit out_array = [i/10 for i in data_array]
71.9 µs ± 1.54 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
@full jay I'm not arguing to ditch or change numpy, just that I'd like the proposed matrix module as a sort of stepping stone towards using numpy
paper echo
is the indexing your main hangup here?
full jay
In fairness, I was still scrolled fairly high up
C isn't used in scientific computing at all and provides absolutely no utility for scientific computing
I was remarking on this specifically
slim island
the indexing, and the sheer amount of different things availabile in numpy. A leaner module with a few core things I feel would help me get my head around using matrices in Python. Rather than right now where I just want to use a different language
@full jay yeah, that was pretty much wrong. I more meant that C and numpy are solving different problems
full jay
Ah, gotcha
slim island
if you read the mailing list, I think I come down slightly on the pro-matrix-module side of things - but I can also see a world where it's just worthless
to be clear, my complaints about numpy aren't a reason as to why numpy is bad. Just a reason why I'd like something else to get used to it. I'm not at all from a mathsy background, I've never touched matlab - so I'm not really the target audience for numpy
paper echo
i wouldnt be opposed to extending array.array to include array.matrix, and implementing __matmul__, __add__, etc for both of the above
no blas/lapack though, just in slow python
swift imp
Honestly @ is the weirdest thing ever
Matrix multiplication is such a rare thing, even in numpy or matlab that I don't even think its necessary
But I suppose its better than nothing
paper echo
i agree i dont understand its inclusion
its nice but it was a surprising addition to the language spec
i see it as "generic bin op that you can implement for whatever you want"
they should have called it __extraop__ instead of __matmul__
swift imp
Plus its not intuitive at all, but I guess they were forced to use it because of the way they used *, **, and ^
paper echo
its arguably better than %*% in R
teal yacht
i'd argue it is more often used than many other operators we currently have rn, not that it justifies adding it in the core language
paper echo
i wish we could use @ for function composition
swift imp
@teal yacht but isnt that because of how numpy has overloaded the operators?
paper echo
which has the added nicety that matrix multiplication is composition of linear transformations 🙂
teal yacht
no, .matmul was already used a lot, even before @ was added
paper echo
def f(x):
return x+3
def g(x):
return x/10
(f @ g)(5)
i'd love this
yes but @ is pretty much only implemented by numpy and pandas, no?
swift imp
like iirc, if you wanted to do a bitwise/elementwise xor in numpy, its .bitwise_xor
paper echo
yep
teal yacht
it's just very domain specific, which is why i don't understand why they'd add it
paper echo
can i write a pep to use @ for function composition
swift imp
why they cant do ^ is bc they have a bunch of extra optional arguments
paper echo
will that get laughed out of the mailing list
swift imp
Honestly its a shame tbh
paper echo
@swift imp isnt ^ already used in numpy
swift imp
^ isn't xor, in numpy?
teal yacht
it is
paper echo
process_name = (
methodcaller('replace', '\t', ' ') @
methodcaller('strip') @
attrgetter('name')
)
I personally think Python fucked up by not adding dot operators to indicate element wise like in Matlab, since they wanted to be inclusive with Linear Algebra
paper echo
i disagree there
plus arent dot operators going to be used for the none aware ops
hopefully
this is the fundamental limitation of a language like python
versus julia or r
swift imp
I think they did, matlab syntax is very intuitive, and that is their main competition
paper echo
its not purpose made for the job and its limited in how much you can hack apart the syntax
but python isnt matlab
numpy is an imitation of matlab
i strongly disagree that matlab is python's main competition
thats not even true in data science
swift imp
Yeah but the entire reason for adding @ was bc they wanted Python to start catering towards data-science/linear-algebra
paper echo
maybe in engineering or astronomy
swift imp
maybe
Yeah I guess, Python is trying to fit the generalist of needs, Matlab is specifically a linear-algebra program
Of course im contradicting myself here. I'm suggesting they make the most commonly used operations, their own (element-wise) but prefixing a . to them while simulatenously suggesting they least used operations the current operators.
While complaining that @ is never used
¯_(ツ)_/¯
pseudo cradle
Am in engineering, it's definitely numpy vs matlab
lean garnet
You'd love julia then @swift imp , this is exactly how it works
and nobody uses matlab in 2020 @covert aspen , unless you're in a big company that has a huge matlab codebase, it's gone. In academics, everybody switched to julia, and in good companies it's julia/np or R
imagine paying for a matlab license when there are better free options out there
broken halo
Hi. @lean garnet Would you let me know main difference between tensorflow and pytorch. When I want to make AI infra on any kind of business, which one is better? As far as I know, tensorflow is using more than pytorch in production. But I am not sure which one is better. Is anyone have recomendation ?Hope answers.
lean garnet
tensorflow is bigger, and scales better. Pytorch is easier (even than keras), but has less features and doesn't scale as well. Now it's about if you prefer Torchscript or Gradient tapes for autodiff (both suck)
paper echo
@lean garnet what fields have adopted julia so readily?
teal yacht
bold claim for the "in academics, everbody switched to julia"
broken halo
Thank you, @lean garnet
lean garnet
it is, but I maintain it (of course not everybody switched to julia, but switched from matlab at least)
@paper echo if you're talking about academics only, Operational research, math researchers (especially the ones often facing differential equations) and physics departments
in business environment it's way slower of course, but I know a few places where they allow Julia in dsc now (most teams don't care about the language used by their data scientists, since they have a fixed deployment pipeline and it doesn't matter how models were trained)
paper echo
yeah unfortunately im not in such an org
finance people seem to have more leeway to do whatever they want with whatever software they prefer
lean garnet
yeah I'm sure people in finance use it too, I just can't say as I don't know
paper echo
since they have a fixed deployment pipeline and it doesn't matter how models were trained
a man can only dream... but thats off topic here
paper echo
i love the design of julia, fixes a lot of the above complaints that people have with numpy
lean garnet
this is a nice little talk if it interests you
paper echo
will watch, ty
broken halo
@lean garnet Have you ever experienced in whatsapp bot?
broken halo
@lean garnet Sure. that's is really hard technical area.
Because whatsapp is not allowed much permission even business account.
lean garnet
also a nice blog article for people whose only arguments in the last 5 years were "But.... Julia is so young! Not enough packages!"
paper echo
lets maybe move this out of #internals-and-peps btw
@ me in offtopic for more julia content though
swift imp
Julia is kinda a joke tbh
well from what conferences and demos Ive seen
For large scale sims and prototyping I've never seen anything come close to the Matlab/Simulink/CodeGen MathWorks has going @lean garnet
swift imp
The whole Model Based Design workflow that The Mathworks has going with Simulink and their codegen is fucking insane, and the amount of device specific drivers they have going to equally as nuts
paper echo
we are already discussing julia in #ot2-never-nester’s-nightmare 🙃
(just trying to keep this channel clear of crosstalk)
@teal yacht i was 100% convinced that python did implicitly compile regexes and cached them all
am i 100% wrong?
EDIT: After a quick glance at the actual Python 2.5 library code, I see that Python internally compiles AND CACHES regexes whenever you use them anyway (including calls to re.match()), so you're really only changing WHEN the regex gets compiled, and shouldn't be saving much time at all - only the time it takes to check the cache (a key lookup on an internal dict type).
teal yacht
Interesting, that was not what I noticed with simple testing, I'll try again in a few minutes
paper echo
yeah i have not tested it myself, ive taken this on faith for years
radiant fulcrum
ig the only thing extra compile provides is making it easier to use the same regex in diffrent places
then again ig you could do that without compile aswell
paper echo
its really nice though
whitespace_re = re.compile(r'\s+')
then you can use whitespace_re in 100 different places in your code
whitespace_re.sub(' ', s), whitespace_re.split(s)
peak spoke
When testing it, compiling did bring a huge speed improvement for me in a loop
unkempt rock
same
peak spoke
Small test I made now, bit above 2 times as fast on a simple regex like this ran against a few strings
In [9]: %%timeit
...: for string in ["aa", "bc", "ef", "GG"]:
...: re.match(r"(.)\1", string)
...:
...:
3.04 µs ± 18.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [10]: %%timeit
...: RERE = re.compile("r(.)\1")
...: for string in ["aa", "bc", "ef", "GG"]:
...: RERE.match(string)
...:
...:
1.31 µs ± 9.89 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
outer root
I have some doubt in plotly dash....
paper echo
@outer root this channel is for discussion of the python language itself, as per the channel description.
woven flame
When I try to install kivy I get an error. I tried it both in terminal and through the interpreter but nothing worked out. Error: ERROR: Command errored out with exit status 3221225477: 'c:\users\karlo\appdata\local\programs\python\python38-32\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\Karlo\AppData\Local\Temp\pip-insta
ll-9giw4uh1\kivy\setup.py'"'"'; file='"'"'C:\Users\Karlo\AppData\Local\Temp\pip-install-9giw4uh1\kivy\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f
.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\Karlo\AppData\Local\Temp\pip-record-0_m56zfk\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\karlo\appdata
\local\programs\python\python38-32\Include\kivy' Check the logs for full command output.
paper echo
@woven flame same applies to your query as well
woven flame
Can someone help me with this error pls?
paper echo
please ask in help channels: #❓｜how-to-get-help
woven flame
okk sry
paper echo
for kivy you can ask in #user-interfaces . but not both, just pick one
woven flame
I already tooh help.
half wolf
The regex cache is a fifo queue with a pretty small size afaik
outer root
@outer root this channel is for discussion of the python language itself, as per the channel description.
@paper echo plotly dash is based on python only
half wolf
You can fiddle with it and that can have dramatic performance implications.
paper echo
@outer root it's about the design of the python language. not just anything related to the python language. again, this is clearly stated in the channel description.
unkempt rock
Hello in this code can anyone suggest how to change the “S” key press to middle mouse button click(pynput)?
delay = 0.001
button = Button.left
start_stop_key = KeyCode(char='s')
exit_key = KeyCode(char='e')
half wolf
@unkempt rock this is for advanced discussion. It's in the name.
unkempt rock
when im creating a module, is it good practice to use a virtual environment?
or not
teal yacht
you should pretty much always use a virtual environment
it's two commands to run anyway
unkempt rock
yeah i guess
half wolf
Python tooling isn't awesome. It should be integrated so you don't have to do anything. Like leiningen was several years ago in Clojure.
boreal umbra
is it true that Python's regex engine doesn't support named groups?
flat gazelle
it does last I checked, django uses it for regex paths
flat gazelle
In [47]: re.match(r'(?P<ABC>\d+)', '124').group('ABC')
Out[47]: '124'
https://www.regular-expressions.info/refext.html there is a list, just set one of the dropdowns to python
boreal umbra
!e
import re
match = re.match(r'(?P<ABC>\d+)', '124')
print(match['ABC'])
fallen slateBOT
@boreal umbra :white_check_mark: Your eval job has completed with return code 0.
124
great, works with __getitem__ too
!e
import re
match = type(re.match(r'(?P<ABC>\d+)', '124'))
print(match.group is match.__getitem__)
@boreal umbra :white_check_mark: Your eval job has completed with return code 0.
False
😮
cloud crypt
hah
cloud crypt
either of them uses another internally I guess
lone wagon
Man, wouldn't it be great if python had pointers and references?
Also, how does the new parser affect us
raven ridge
It only has references.
teal yacht
everything is a reference, you just can't dereference it arbitrarily
raven ridge
!e ```python
a = []
b = a
b.append(5)
print(a)
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
[5]
this works because a and b refer to the same object. That's references.
marble kite
Trying to understand Linked Lists in python made me understand how references work much better
feral cedar
storing the next element is a waste of space, just store where it is
pseudo cradle
bogosort
raven ridge
Also, how does the new parser affect us
@lone wagon For 3.9, it doesn't - 3.9 will ship both parsers and the user can choose which to use, defaulting to the new one. For 3.10, there's talk about introducing some new features that wouldn't have been possible - or at least, would have been much harder - with the old parser. Like https://www.python.org/dev/peps/pep-0622/
brave badger
I'm really hoping that PEP gets approved
raven ridge
I'm pretty meh on it...
swift imp
Why?
Seems like it accomplishes a lot
Like it coulda kind of act as multiple dispatching no?
raven ridge
I worry that it hasn't been given enough thought... The current version has:
For the most commonly-matched built-in types (bool, bytearray, bytes, dict, float, frozenset, int, list, set, str, and tuple), a single positional sub-pattern is allowed to be passed to the call. Rather than being matched against any particular attribute on the subject, it is instead matched against the subject itself.
But the original specification suggested doing that in general when there is no__match_args__defined, which would have meant that adding a__match_args__to a class that didn't previously have one would have been a backwards incompatible change.
gleaming rover
I love pattern matching
but
honestly I don't know.
- it not being an expression irks me
raven ridge
I also feel like this is one more mini-language inside the language that would make it harder to learn. We've already got two different string-formatting mini-languages, plus regex, all of which are... a lot to teach someone.
boreal umbra
I looked through the pep and I feel like it's not immediately obvious what the syntax is meant to do
and we already have perl for when we want non-obvious syntax.
raven ridge
the use of something that looks like a constructor but isn't a constructor irks me. Maybe if you're a newbie it might actually be less confusing, but for me seeing case Foo(5) do something other than call the Foo constructor with 5 as an argument is just... weird.
If it happens, I'm sure I'll find some use for it, but I'm not enthusiastic about it.
teal yacht
there is also a case, where idk at all how we will do, case T(): does not match the pattern T with no argument, but any instance of type T, which is extremely inconsistent, afaik, there's no way to match the empty set at all with it for example
pattern matching just does not fit the data model of python, it's just being hacked into the language forcibly and it shows
raven ridge
It's not the worst - it's nothing I can't live with - but it does leave a bad taste in my mouth. There are few enough times when it would have been really useful to me, and I dread the questions we'll get about it in the help channels when it lands, heh.
unkempt rock
A new parser... Sounds like a big update.
Its definitely not immediately obvious what that code does. I dont have time to read the PEP tonight but I'm interersted. I still forget what the walrus syntax does.
pseudo cradle
Something something assignment
boreal umbra
@unkempt rock the walrus operator just lets you do variable assignment within an expression. There was some reason why that functionality had to be a new operator rather than expanding the use case of the regular assignment operator
so you can do
unkempt rock
You saying that is bringing it back to me now. 😛 I just forget every time I see it because I never use it
boreal umbra
import re
if (m := re.match(r'[0-9]+', '123'):
print(m[0])
narrow kettle
was there an actual reason to make a new operator, or was it just to avoid the pitfalls of c based langs where you mix up = and ==
boreal umbra
I wonder if that would work without the parens, in this case
!e
import re
if m := re.match(r'[0-9]+', '123'):
print(m[0])
@boreal umbra :white_check_mark: Your eval job has completed with return code 0.
123
you summed it up well @narrow kettle
they specifically wanted to avoid the issues that arise in C-like languages with assignment as an expression
really that's the gist of all the drama about the pep
boreal umbra
I thought the drama was that some people thought it was non-obvious and would bring about the downfall of Python's self-explanatory syntax.
pseudo cradle
I think it's kind of ridiculous
narrow kettle
i mean theres alot of python thats non obvious lol
feral cedar
it is a little unintuitive
half wolf
:= was silly because it's not broadly useful.
feral cedar
but i mean cmon, it looks like :=
unkempt rock
My main thing is that I dont want an excess of syntactic sugar. If it does a thing I could not do before, cool.
But I also won't get overly angry about it if they decide to add some.
narrow kettle
tbf := is a standard assignment operator in programming
its hardly NEW syntax
its very old
half wolf
If we could get short form for kwargs it would be like 1000x more useful than := imo
boreal umbra
short form for kwargs?
narrow kettle
wdym short form kwargs
half wolf
@narrow kettle that's part of the problem though. Because it's different in python.
foo(=bar) as a short form for foo(foo=Foo)
teal yacht
uh, how often do you write foo(foo=...) ?
narrow kettle
uhh
half wolf
Ocaml has it (but it's foo ~bar)
teal yacht
i don't think i've ever written similar code
feral cedar
i mean kwargs is pretty much as short as it can be without losing information
narrow kettle
im not seeing how thats the same thing?
teal yacht
ocaml doesn't have that
narrow kettle
i dont recall ocaml having that tbh, but ive heardly written alot so i could be wrong
half wolf
Like a million times. Every time you pass a dict to a template in django for example.
narrow kettle
foo(=bar) as a short form for foo(foo=Foo)
how are these equivalent tho
im not following
half wolf
The syntax is just a suggestion obviously.
narrow kettle
how does that work with multiple named args?
half wolf
Meh, my auto complete messed up my example. Yes sorry.
charred wagon
So, to use the variable's name as the kwarg name implicitly?
teal yacht
i don't think it'd be incompatible
narrow kettle
the whole point of named args is that you have have the name
half wolf
Jay: foo(=a, =b, c=3)
narrow kettle
so you just want a way to override the , *, operator?
half wolf
@charred wagon yep
charred wagon
Okay, that's an interesting proposal
half wolf
More kwargs == better. Imo
charred wagon
Maybe for what you do that'd be quite helpful, but I don't encounter that too often, so I could say about that what you said about :=
merry pollen
oh that would be nice
charred wagon
It's all subjective
narrow kettle
i mean ya i can def think of a few places in my code that would be interesting
half wolf
I wrote a tool to do statistics on that actually, so I can prove it's common ;)
half wolf
It's super common for us web devs. Like suuuuuuper common
Ah, here we go: https://gist.github.com/boxed/610b2ba73066c96e9781aed7c0c0b25c
Man. 2 years ago. :(
Scroll down for some statistics!
narrow kettle
ya i feel like i could get used to that
half wolf
I also almost always use dict(...) instead of {...} because it's nicer, but with this feature hoooo boy it'd be amazing
half wolf
You guys remember the debug f string feature? It's not needed at all with this feature of mine:
debug(=1*5+x)
(A bit more controversial now ;))
narrow kettle
how would that work here
pseudo cradle
What would we call it though, the smileyface operator?
half wolf
Kwargs can be any string :)
narrow kettle
thats not a string tho no?
teal yacht
debug(**{"1*5+x":1*5+x})```debug(=1*5+x)
debug(**{'1*5+x': 1*5+x})
Phew. Ah. Yea you got it. 👏👏👏
Typing that on my phone sucked ;)
narrow kettle
im really not seeing how those are equivalent
what am i missing
unless im slightly misunderstanding ur idea
half wolf
The keyword argument string is created by taking the thing on the right hand side of the = instead of being manually written by the programmer.
It's a trivial compile time transform
narrow kettle
ya i get that part, but 1*5+x cant be compile time
half wolf
The problem here is that getting a feature into python is all about the nepotism.
I was literally told that if I don't know who the core devs are then they won't tell me. And you need a core dev sponsor for a feature.
half wolf
Submit yes. Get it in no.
I even implemented this feature.
I can't even get this thing in without knowing a core dev: https://github.com/python/cpython/pull/9655
2 years for a trivial change that would make cProfile not worthless by default. While stuff is merged without code review that breaks things constantly.
Steering of cpython is a mess :(
And funding is almost non-existant.
narrow kettle
ik very little about the inner workings of cpython tbh, my experiences with such things have been over on roslyn
which has a very active and clear direction
have you drafted a pep for this?
half wolf
Kind of. I drafted a pep for a different syntax: foo(*, a, b) because my original suggestion of foo(=a, =b) was shot down. But then that proposal was shot down an people even suggested my old syntax to me 😂
Python-ideas is quite brutal.
narrow kettle
sugar suggestions like this are always hard, people focus on syntax anyways. but often times can be reminded that syntax is the last part of a langauge feature to be considered
but with features that ARE all sugar then the syntactic nature of them makes them controversial
half wolf
Python-ideas shut down the suggestion to fix so time(0) isn't falsy because of backwards compatibility. Imo this shows the group is just a bunch of angry dudes. This change would have fixed more latant broken code than what it would broke.
narrow kettle
python is looser about backwards compat then most tbh
narrow kettle
in a lang like c# breaking changes are a HUGE nono
fun fact
backwards compat is so important to c#
that when they wrote the new compiler roslyn about 6 years ago
they had to emulate the bugs of the old native aot compiler
as in hard code the bugs
because fixing them would break things in legacy code
half wolf
Typical Microsoft yea. Not a fan imo :)
narrow kettle
their clients are tho lol
half wolf
Maybe. But it's a trade off and the customers aren't happy windows is shite ;)
Apple has the right approach imo. It's frustrating but it moves forward so much faster.
half wolf
Iphone 3g was the best! ;)
narrow kettle
Windows already has enough difficulty getting people to update, if their updates broke stuff like apples do then companies would still be on 95 or xp
half wolf
They are on xp.
narrow kettle
It’s not as common
narrow kettle
7 has the lions share from what I’ve seen
half wolf
Microsoft has just got the worst customers
narrow kettle
My last company was on 7, that because the software we used was 7 only
teal yacht
companies are moving away from 7 too, due to them stopping security updates
narrow kettle
So we were locked in
teal yacht
we got f-ing chromebooks when they started replacing it ...
tired epoch
lol
half wolf
@teal yacht waaaay too slow
narrow kettle
I’d start job hunting tbqh lol
teal yacht
i am, but i can't ditch the job because i need it to graduate :[
half wolf
I work in banking and hooo boy it's slow
tired epoch
its prob with low specs lol
narrow kettle
Y’all are still on cobol i hear
half wolf
At least we're rid of ie6 finally
tired epoch
chromebook can be pretty fast too
half wolf
Well, the consumer banks are. I am in fintech and we're the good guys and modern but still have to deal with their browsers and stuff
Broken CSV parsers aren't just for COBOL :(
narrow kettle
Tbf i feel like all csv parsers are broken LOL
tired epoch
ah fair enough
half wolf
Pythons is good. Very good in fact.
tired epoch
def but for me its only lacking in concurrency bit
half wolf
Thank God toys r us went bust. That company has special code to fix broken CSVs
Who put quotation marks in their name?!?!
Anyway. Maybe I should submit short form keyword arguments as a pep.
inland oxide
can i have help with html code??
formal meteor
Like <script> in html?
feral cedar
this isn't the channel for that
raven ridge
@half wolf
cProfilecommand line output now defaults to cumulative time, and gives you one level of the path if the name of the file starts with__, like__init__.py
May I suggest breaking that up into two PRs? It's very tough to discuss the merits of one PR that makes two unrelated changes. Though admittedly the silence on that PR is disappointing.
half wolf
@raven ridge not gonna start the process from zero for such a trivial change after 2 years. The problem isn't the change. The problem is the under staffed group going through them.
raven ridge
Perhaps, but devil's advocate: the PR introduces two independent changes, and does not explain why either is an improvement over the current behavior. Not to say that it isn't, but it certainly isn't explained well.
Or, to put that a different way: one could that the fact that the proposal didn't get interest from even a single core dev is at least in some part due to the way in which the proposal is presented.
icy summit
I'd like to try to make something like a 4chan clone, would you guys recommend Flask or something else as framework?
flat gazelle
#web-development is probably the more suitable channel, I would say pretty much any framework will do fine, 4chan is pretty basic after all.
icy summit
Thanks for the redirect, I'll try there as well
torpid plinth
hey guys, I'm trying to learn Linux, not just the distro,kernel or os but i want to learn how it works and how it can be developed for. so i thought the best way to learn would be to use Linux. ik about ubuntu but i wanted to know if there was a os which had a more original Linux kernel/core that i can learn to develop for. thanks
half wolf
@raven ridge hmm? The output listed is pretty clear no? The old output is 100% worthless and the new output is useful. How could I improve the presentation of such an obvious and large improvement?
@torpid plinth this is for python discussion.
spice cape
void lake
CAn anyone suggest me a good resource to get brief understanding about Probabilistic Programming in Machine Learning?
Please ping me.
somber python
question about naming convention (wasn't sure which topical chat/help channel it's most relevant for)
nonpublic methods start with a prefix, right?
what if I have the same method name across different classes?
slim island
if you want to avoid duplicating names, you can use a trailing underscore. e.g max_ = max(things)
paper echo
@somber python same method name on different classes is fine
As long as it's not confusing
somber python
yeah I have _get_name, _get_price etc. for a few different classes, as they are retrieved in different ways
thanks as always salt rock lamp 😄
paper echo
Note that names starting with _ are usually considered "private" and are not meant to be used from outside the class
unkempt rock
Realizing I could do self.id = _id changed my life.
Or at least, it changed the amount of time I was trying to think of an alternate name for id when id is the best name
I guess, should it be id_ since the one at the beginning already has another meaning?
spice pecan
trailing underscore to avoid overshadowing, leading underscore to imply that it's for internal use and not part of the public API
unkempt rock
Right
raven ridge
I feel no remorse about shadowing the name "id" with a function argument. It's often a good name, and I never need to use the built-in id for anything. I'd avoid doing it at global scope, but inside a function I vote it's fine
hexed maple
Same goes for type
raven ridge
Agreed.
willow obsidian
is @fallen slate open source?
slim island
!source yes it is, this isn't really the right channel to discuss it though (I think). #dev-contrib I think is the right place if you have implementation questions, or maybe #community-meta I'm not entirely sure
fallen slateBOT
slim island
pseudo cradle
and obviously as long as you're not importing them into the same namespace
oops, I'm responding to sometihng old, disregard
thorny root
how would i make an api wrapper? someone just point me in the right direction
visual shadow
Try in #python-discussion, this is off topic for this room. (an api wrapper sounds like the api already exists? So it depends on what your goal is. Wrapper for who?)
thorny root
ai my bad
pastel zinc
How do I make a Loading Screen Animation for my program
grave jolt
@pastel zinc This is offtopic for this channel. Please read the channel description. If you want help, check out #❓｜how-to-get-help or ask in #python-discussion
also, provide more information on your quesiton: the code you have, the library you're using etc.
burnt wind
does OOP could as advanced python?
class Students():
def __init__(self, name, age, gender, grades):
self.name = name
self.age = age
self.gender = gender
self.grades = []
def __repr__(self):
print(f"{self.name} is {self.age}, and is {self.gender}")
def AvgGrade(self):
for grades in self.grades:
average = (grades + grades)/len(self.grades)
print(f"The average grade of {self.name} is {average}")
S1 = Students("Jia Hong", 17, "Male", [10, 10, 7, 9, 8, 8])
print(S1.grades)
So the output here is just []
grave jolt
@burnt wind Please read the channel description. This channel is for discussing certain topics, not asking/providing help. Check out #❓｜how-to-get-help or ask in #python-discussion if there isn't too much traffic in there.
Your issue is that you don't call the AvgGrade method.
spark magnet
you also don't save the grades you passed to your constructor
grave jolt
And the algorithm for finding the average is quite strange.
<complaint> I don't understand why so many courses/curriculums are obsessed with classes. They teach about classes before properly explaining other parts of the language. Students can be forced to use them no matter if it's appropriate or not. And there are so many skills to master and problems to solve before jumping to classes. </complaint>
spark magnet
@grave jolt i think it's java-itis
pseudo cradle
I get it. They want them to adopt an OOP paradigm early
feral cedar
since you need a class to write hello world
pseudo cradle
I support using classes when it's... relevant
feral cedar
that's pretty reasonable
pseudo cradle
not kludging it in to something like Hello World
grave jolt
Many applications of classes that are taught in the beginning can be easily accomplished with dictionaries and other familiar data structures. I don't know we that idea isn't used often.
!e
def make_student(name, age, gender, grades=None):
if grades is None:
grades = []
return {"name": name, "age": age, "gender": gender, "grades": []}
def print_student(student):
print(f"make_student({student['name']!r}, {student['age']!r}, {student['gender']!r}, {student['grades']!r})")
alice = make_student(name="Alice", age=42, gender="xenomorph")
print_student(alice)
@grave jolt :white_check_mark: Your eval job has completed with return code 0.
make_student('Alice', 42, 'xenomorph', [])
If you're brave enough, you can even do inheritance.
grave jolt
alice["name"] = "Bob" 👀
dense lagoon
I think learning dataclasses/POD classes early is good (think structs in C) because it avoids Javascript-itis where everything is just a dict being thrown around (and it lets the IDE help you by autocompleting)
tired epoch
oh ye haha forget u r returning dict
dense lagoon
learning full-blown OOP right at the beginning is not great though; too much information overload particularly about bits of OOP people don't need to spend a lot of time on to get their confidence up
grave jolt
Well, it's kind of weird to teach dataclasses before classes...
tired epoch
oop is more like of a concept
like in js u can just define a constructor and add the desire function in that after using prototype
dense lagoon
I haven't taught python to beginners, so I don't have experience in that. But I can safely say that there is nothing wrong with teaching things out of order
dense lagoon
think they're called chainmaps 🙂
(back to pedagogy) e.g. in c you don't teach people anything about what hello world actually does, you just show them a program and say "trust me" to pretty much everything
feral cedar
"gender='xenomorph'" 🤔
feral cedar
yeah my "first line of code experience" was just the class copying down exactly what the teacher said
tired epoch
r u trying to someting like python hey = { "object": def someting(): print(someting) } #is it for inheritance ??
can u do this?
grave jolt
Well, you could make a lambda inside...
tired epoch
ye that too haha
grave jolt
Yeah, the very first examples can be like "trust me". But I don't think it's a good idea to keep too many things "magical".
that's why I don't like some resources telling people to use list with map without explaining what map does
tired epoch
@dense lagoon out of curriosity., do u guys start teaching programming with java for beginners?
grave jolt
it's OK to use a suboptimal solution and later show a better way
feral cedar
i started with java
dense lagoon
[long story - abridged] I didn't do CS in my university, but I ended up lecturing chemistry students an introductory fortran course (computational chemistry still heavily uses fortran for Reasons). That's my teaching experience.
The CS degree in my uni started with intro to programming through Java in 1st year, then they just expected you to pick up other programming languages as needed by other lecture courses (e.g. C/C++/Haskell/etc.)
tired epoch
ye mine was c++ after java i guess its common pathway for uni programming course
dense lagoon
(I always wanted to go back to that chem course and teach them a mix of python and fortran; both would be super useful to them 😦 )
slim island
@grave jolt what is the advantage of not learning classes? Once you wrap your head around the basics, they're very intuitive. Most people learning OOP wrap their head around them, then don't use them all that much for a while
Using dictionaries doesn't seem to have any advantages - I'm not sure using dictionaries like bad dataclassess is a good idea
grave jolt
Well, the problem is that many courses teach students classes before they wrap their head around the other basics.
slim island
In your student example, it should very clearly be a class or dataclass. I don't see any reason to teach the wrong way first
I think classes are pretty basic
grave jolt
Yeah, maybe it's not the best idea.
slim island
One problem i see somewhat often is courses jumping straight from basic OOP to inheritance
dense lagoon
Lots of stuff about classes are basic, understanding the MRO and what qualifies as "good" or "bad" cohesion is not.
slim island
I feel like there should be a step inbetween, get the basic syntax/idea of classes down before teaching about inheritance
dense lagoon
The problem is most courses spend about 80% of the time on that 20% of the messy stuff
which is unhelpful to someone seeing them for the first time
tired epoch
it was struggling with MRo at start lul
but i heard of that term only when i was learning python
slim island
I don't think the solution is to not teach classes - I think it's to just get better at teaching classes. If you teach bad practices like the dict example above, it will become much harder for students to pick up the right way down the line; if you teach bad practices, that almost always sticks with students long term
feral cedar
"just teach better"
slim island
Well. I gave a concrete example of what that means a message above
so
there's no reason to repeat myself
if you're incapable of reading back more than a couple of messages - then I don't really know what to suggest
tired epoch
lul
feral cedar
in the classes that i took, inheritance was almost completely separated from classes by about a year of other stuff
tired epoch
for me they cchucked in everything in the same class
slim island
Alright - sounds like they're doing it correctly then. That's not my experience
Yeah, I've seen OOP taught all in one chunk for 2 completely different courses (A-Level and Uni). To the extent where you go from no idea what a class is to multiple inheritance over the course of 2 hours
that obviously doesn't work too well
tired epoch
totally agree with ya
slim island
I've also seen some courses/books that do it
grave jolt
👀
feral cedar
at least for books you can slow down and reread
hexed maple
Yea
slim island
Yeah, but you're putting faith in the author that they're sensibly organising the content. Giving some breathing room where the author just uses basic classes in other applications before coming back to inheritance would make a lot of sense
tired epoch
someone might pop up and say its off topic _-_
hexed maple
Guys this is getting off topic
hexed maple
LOL
outer osprey
Have you used linq libraries in python, if yes which one is more common?
For simple queries, we do not need linq libraries like where, find, any, all, etc but what about for the complex one?
half wolf
@outer osprey do you have an example of such a lib?
last pollen
a lot of linq stuff can be done using the standard library in Python, afaik
grave jolt
Why are functions and classes hashable?
That feels wrong, since they are mutable. But I guess they're not intended to be mutated
And their identity is their value
visual shadow
dang, til. didnt even know they were hashable
pearl river
oh, isn't that wonderful. How are they hashed, by source code?
grave jolt
seems like id/16
in CPython
!e
class A:
pass
print(hex(id(A)), hex(hash(A)))
class B:
pass
print(hex(id(B)), hex(hash(B)))
@grave jolt :white_check_mark: Your eval job has completed with return code 0.
001 | 0x555d9f4a85e0 0x555d9f4a85e
002 | 0x555d9f4a8990 0x555d9f4a899
oh, no, something else
no, yes, I was right
uhhh, wait
Yes, it's just that the alignment weird. hash = id/16
deft pawn
Is there a way in PyAutoGUI to make it run as a background process without physically controlling the input from mouse and keyboard?
grave jolt
@deft pawn This is not the right channel, please read the channel description. Check out #❓｜how-to-get-help
Keep in mind that if you want to make something malicious like a keylogger, we won't help you
deft pawn
No im trying to automate tasks in the background
outer osprey
@last pollen OK.
@half wolf Many libs in python packages (py-linq,Linq · PyPI,etc.)
heady mauve
I am wanting to make an application that will bundle with pyinstaller and will detect if another instance of it is already running when launched. If there is another instance running, it will contact the other instance and request that it do something. I am thinking I'll do this with a lock file. Is there a better way?
I will be checking if there is a running instance using win32ui.
gleaming rover
Why are functions and classes hashable?
@grave jolt mutable thingies can be hashable as long as the hash doesn't change when the object is mutated
grave jolt
hm, "The only required property is that objects which compare equal have the same hash value"
So I guess it is totally valid for identity-comparing but mutable objects to be hashable.
I.e. the objects whose identity is their value.
raven ridge
Class instances are hashable by default, too, unless you add a __eq__, in which case they're not hashable unless you also add a __hash__
teal yacht
you can have that with if you compile with debug symbols
raven ridge
That is a thing. https://www.nongnu.org/libunwind/
raw agate
Its possible create a custom for ?? To increase the velocity
Or increase velocity of my current for
radiant fulcrum
what
raw agate
Create a custom for ou increase velocity of my current
teal yacht
I don't think whatever you're asking fits this channel
raw agate
Or some alternative
teal yacht
I'm detecting some physics-related keywords
raw agate
Its possible?
feral cedar
uh, check out #❓｜how-to-get-help
teal yacht
it's not possible no, not for arbitrary iterables
raw agate
Is there any alternative?
radiant fulcrum
if your tasks can be moved into threads (or better processes) without directly depending on each other, then you would get a performance increase
otherwise no
and idk why you even want it to be that fast, its already way faster than what most people get affected by
raw agate
I didn't like the speed that mine is going, its fast but I need more speed
radiant fulcrum
Its almost 100% your code bottle knecking it rather than the for loop
raw agate
There's not much code in the body, it's just for and a function
radiant fulcrum
making the for loop faster isnt going to speedup your sync based code
flat gazelle
for loops across lists are fast enough that ram is the bottleneck even with ints IIRC
raw agate
radiant fulcrum
well thats fucking why
you have 4 nested for loops which is so freaking inefficent
raw agate
Could be my silabas
flat gazelle
string + is slow, that probably creates 3 intermediate strings that immediately get deallocated
radiant fulcrum
whats the O(n) factor on that lol
flat gazelle
use ''.join
flat gazelle
that should not apply in the case of a + b, just in the case of a += b
radiant fulcrum
wrong graph is in my head
spice pecan
oh that's the article I couldn't find last time!
radiant fulcrum
also that dude is just checking each combo of letters for a single word no?
flat gazelle
anyway, I would slap a numba.njit on it and see
radiant fulcrum
wouldnt if 'pumpkin' in silabas work
raw agate
also that dude is just checking each combo of letters for a single word no?
@radiant fulcrum Yes
radiant fulcrum
or am i being randomly dumn
flat gazelle
it should infer everything correctly
well, the equivalent operation here is itertools.product
raw agate
He checks in syllables forming words until he finds the pumpkin
flat gazelle
not permutations
radiant fulcrum
Why bother going through each letter when you know the characters used in the word pumpkin lol
raw agate
Porque quem usa o programa pode usá-lo para qualquer palavra
radiant fulcrum
the way his system works, if each one of those letters exist it will say "found pumpkin"
yeah
im agreeing with you
i wasnt saying you're wrong lol
raw agate
the way his system works, if each one of those letters exist it will say "found pumpkin"
@radiant fulcrum
I will test this
radiant fulcrum
i probably worded it wrong
raw agate
I solved my problem, thanks
feral cedar
how'd you solve it?
raw agate
I found a topic that can help me, and I tested and improved the for
feral cedar
your code though
raw agate
It's not ready yet, but I managed to test it, now I'm just improving some things
raven ridge
that should not apply in the case of
a + b, just in the case ofa += b
@flat gazelle Ah, true, it doesn't.
pseudo cradle
I can't remember the last time I implemented anything above n^2
or n*m
Professionally or recreationally
teal yacht
well some problem do not (currently?) have polynomial time solutions, so it really depends on what you write
pseudo cradle
Right, of course
Huh, I am actually working on a problem that is somewhere between n^3 and n^4 depending on how I need to implement it
I just haven't implemented it yet
I'm generally too busy to justify actually coding an implementation for it, so I just think about it sometimes when I'm free
It's an advanced version of the knapsack problem which is N*W or N^2, there's an extra parameter which complicates things and there's a fractional component added which complicates things further
Google says it's np-complete
Either way, if someone ever has insights into this problem I'm always open to hear them 🙂
swift imp
Man, Mark Shannon rips up Pep 622. However he does bring up some very valid points.
boreal umbra
!pep 622
fallen slateBOT
boreal umbra
Ah yeah. Idk if I'm on board with this one.
slim island
https://github.com/markshannon/pep622-critique
I don't understand the area well enough to have an opinion. But reading the pep and the critique was interesting
radiant scroll
If [] is a way to make an instace of list
could I make something like ^^ which would reference to my own class?
class Foo:
def __init__(self, *vals):
self.vals = vals
a = ^1, 5, 8^
print(type(a))
<class '__main__.Foo'>
pliant tusk
Not natively
signal tide
"hows a few of cases where PEP 622 shows readability advantages over much simpler alternatives. That's about one line of code per 100 thousand." I feel like it should be more of a personal thing (' vs ")
like if it's only improving readability slightly/isn't improving it, why have it in the first place?
pseudo cradle
Well, it wouldn't be a very interesting problem if it wasn't a challenging one
charred barn
i'm a big fan of pattern matching in languages like rust and elixir but i'm not a fan of pep 622 personally
gleaming rover
me neither
charred barn
did some messages in here just get deleted or am i going crazy? O_o
north root
just keeping the channel clean
charred barn
oh okay sorry for re-messing it then 😄
just wanted to make sure it wasn't a discord problem :X
north root
hahah, all good
feral cedar
gary at the next team meeting "guys, discord keeps randomly deleting messages, and i don't know why"
charred barn
peak spoke
I've read through the PEP a while ago but imo in its current form it's just too complex. The syntax looks cryptic in some cases to a person that hasn't used it a lot and I don't think it brings many advantages over what you can do with the current syntax
teal yacht
they simplified it a lot, initially they went with an atrocious design using some __match__ protocol, but thankfully they ditched it
pseudo cradle
Sounds like it would help a small handful of people and confuse everybody else?
potent shell
pickle is cool for storing trained data as byte streams with sklearn, but is that the appropriate approach?
charred barn
my impression when i read it was it was a half measure that added all the complexity with little benefit. i'm happy to see more people pushing back against it. when it first came out people were very hype
peak spoke
It has some big names attached to it, and I image a bit of the community simply looked at it as a simple switch statement (two of which were rejected before).
narrow kettle
my 10cents: i think a dynamic language like python would benefit tremendously from pattern matching, however 622 does seem to miss the mark in a lot of cases tbh, lots of nuance and edge cases
i think its a great idea, but perhaps the implementation leaves somethign to be desired
teal yacht
@potent shell no, pickle is almost never the appropriate approach, it's extremely unsafe, and there are other file formats for saving models
potent shell
Could you please give me more information on the alternatives? (I've done my bit of googling)
teal yacht
ugh, apparenly sklearn doesn't have any alternative to pickle, one more reason to never use that library
signal tide
tensorflow is too tense
@potent shell I'm gonna frame that ty
potent shell
lol
have you seen their documentation
hands down, Django has the best documentation in the world
boreal umbra
I have a module where I make a lot of classes and all of them need a __lt__ and sometimes a __eq__ as well
so they typically follow this pattern
with suppress(AttributeError):
return <some code that will inadvertantly raise AttributeError if the other object is the wrong type>
return NotImplemented
seems like there should be a decorator that handles this, returning NotImplemented automatically
I tried searching for one but didn't find one, so I'm wondering if anyone is aware of such a thing before I try implementing it
teal yacht
should be easy to write in a couple lines
something like
In [1]: import functools
...:
...: def noexcept(*excs):
...: def decorator(f):
...: @functools.wraps(f)
...: def wrapped(*args, **kwargs):
...: try:
...: return f(*args, **kwargs)
...: except excs:
...: return NotImplemented
...: return wrapped
...: return decorator ```guess I'll have to credit you in the readme
teal yacht
boreal umbra
you know, maybe a decorator already exists that returns a given value if a given exception is raised
@exceptreturn(AttributeError, NotImplemented)
def __eq__(self, other):
....
I don't think that exists in the stdlib, but there's definitely at least 90 implementation of that same code i posted above in github repos
boreal umbra
the last time I made a suggestion for the stdlib, it wasn't well received, so I think I'll stick to not doing that
unkempt rock
I think pickle is only problematic if you're putting/getting it on the web. I'm pretty sure data science people use it all the time locally.
raven ridge
It's also problematic if you're pickling something using one Python version and unpickling with a different one. Or if you're unpickling data from an untrusted source.
teal yacht
The main issue is indeed the security, but it's also not portable across versions or other languages, nor is it particularly efficient
I'd say it's all around pretty bad, but it's not like I have a solution that could fit any type of model like in sklearn
undone hare
It's also problematic if you're pickling something using one Python version and unpickling with a different one. Or if you're unpickling data from an untrusted source.
@raven ridge actually, the pickle implementation guaranty that you can pickle and unpickle data across version, even across major releases
raven ridge
If you use the lib carefully, perhaps. But I'd bet most people just use pickle.dumps(obj) without thinking about the protocol.
teal yacht
Is that something that was always here, I swore I saw ppl talk about versions of pickle themselves, and/or python's causing issues
undone hare
I'm pretty sure that there's a magic number stating the protocol version
teal yacht
It says in the doc it's not guaranteed to work
undone hare
Hmm, hold on then
raven ridge
And it's still possible to fail to deserialize something if it was encoded with a newer protocol version than your interpreter supports.
teal yacht
Wait no that's marshal
wanton smelt
there's a protocol version for pickle
but even then it relies on library versions as well
i didn't bother looking too much into it
but our dev team had a custom pickle object that wouldn't deserialize with different pandas versions
undone hare
raven ridge
The pickle serialization format is guaranteed to be backwards compatible across Python releases provided a compatible pickle protocol is chosen and pickling and unpickling code deals with Python 2 to Python 3 type differences if your data is crossing that unique breaking change language boundary.
It can work, but the user has to know how to drive it to make that guarantee
undone hare
Well, that make sense
The protocol version of the pickle is detected automatically, so no protocol argument is needed.
raven ridge
Objects pickled by 3.8 can't be unpickled by 2.7 by default, for instance.
undone hare
It is trying to match the protocol version by default
raven ridge
Because the default protocol version in 3.8 is one that didn't exist in 2.7
raven ridge
Couldn't say if there's any backport, but the standard library doesn't have it
undone hare
That's a fair point then
raven ridge
I believe every version of Python 3 has used a protocol version that does not exist in 2 by default.
lost rain
How can I parse .inv file I aren't able to understand pybot docs
Code
How it parses
Any simple example can help
Plz ping when someone answer
paper echo
The problem with pickle isnt the protocol version
Its that if any library changes anything internal at all, your pickle load breaks
0.43.2 bumped to 0.43.3 with one internal name change? Sorry you are SOL whole thing breaks
The fact that sklearn has not seriously attempted to switch to a json-based model serialization format is depressing
But their internal architecture is a little wacky
As user friendly as it is
I'd probably donate $100 towards a better serialization format for ML models that arent "networks"
Something simple but more general than ONNX
Or just json or xml
narrow kettle
stuff like this is so tempting to do ```py
def foo(test):
print(test)
foo(a := 9)
print(a)
brazen jacinth
i had to do a double take, to get that
flat gazelle
use the help channels for these kinds of questions, see #❓｜how-to-get-help
balmy mountain
kk
feral cedar
python 2 in 2020?
ornate violet
welcome to debian
paper echo
debian has py 3 though
flat gazelle
IIRC, the version I downloaded had 3.4, at which point py2 is a much more valid choice
paper echo
does anyone have an example of an actual API written using .send() and generators? im having a hell of a time figuring out how to actually use this in real code.
use case: implementing a sans-io client, the idea being is you .send() the result from the last i/o action and that returns a list of i/o actions to take next. but i dont see why youd actually want to do this with generators, compared to just a class with .send and .recv methods
wide shuttle
Hmm, you could look at how coroutines were implemented before the async def/await syntax was introduced. The send was what powered those old-style coroutines.
There's a nice article on that floating around somewhere, which includes examples of how it's used
Maybe that could get you started
Uhm, this is one such article: https://dev.to/codemouse92/dead-simple-python-generators-and-coroutines-21ll
I haven't fully read that one, though
I can't seem to find the other one I had in mind
peak spoke
Don't think I've ever seen it used anywhere apart from when I was playing around with coroutines and sending values directly without awaits
paper echo
im more wondering if there is an actual nontrivial example of this in production code, that actually uses the send/receive thing for real work
flat gazelle
the @contextmanager decorator does use it, but I am not aware of anything larger in scope
paper echo
@flat gazelle does it use .send for the exception info?
deft pagoda
i used .send when writing a generator for real ranges -- a use that involved no coroutines at all
paper echo
got an example salt?
deft pagoda
yeah, sec
paper echo
thanks
wide shuttle
I don't think I've seen a lot of actual examples outside of coroutines. I think send was introduced to have support for coroutines in Python.
deft pagoda
https://github.com/salt-die/real_ranges/blob/master/real_ranges/range_set.py
def replace_least_upper(self_set, other_set):
"""A helper iterator for the __and__, __or__, and __xor__ methods of RangeSet, this will call next
on the correct RangeSet iterator (the one that last yielded the range with the least `upper` bound).
This incurs some overhead, as we repeat comparisons, but it hopefully makes RangeSet dunders easier to
follow.
"""
self_ranges = iter(self_set)
other_ranges = iter(other_set)
self_range = next(self_ranges, None)
other_range = next(other_ranges, None)
carry_over = yield self_range, other_range
while self_range and other_range:
if self_range.upper == other_range.upper:
self_range = next(self_ranges, None)
other_range = next(other_ranges, None)
elif self_range.upper < other_range.upper:
self_range = next(self_ranges, None)
if carry_over:
other_range = carry_over
yield
else:
other_range = next(other_ranges, None)
if carry_over:
self_range = carry_over
yield
carry_over = yield self_range, other_range
if self_range:
yield self_range
yield from self_ranges
elif other_range:
yield other_range
yield from other_ranges
this was the generator
flat gazelle
it uses .throw() actually, so probably nevermind
wide shuttle
The PEP that introduced it is even named "coroutines via Enhanced Generators"
paper echo
@wide shuttle kind of by definition, using .send makes it a coroutine, i think
deft pagoda
and one of the methods that used the generator:
@ensure_type
def __xor__(self, other):
iter_ranges = replace_least_upper(self, other)
self_range, other_range = next(iter_ranges)
xored_ranges = []
while self_range and other_range:
if self_range.will_join(other_range):
dif = self_range ^ other_range
if isinstance(dif, RangeSet):
r, dif = dif
xored_ranges.append(r)
if self_range.upper == other_range.upper:
if dif:
xored_ranges.append(dif)
else:
iter_ranges.send(dif)
else:
xored_ranges.append(min(self_range, other_range))
self_range, other_range = next(iter_ranges)
xored_ranges.extend(iter_ranges)
s = RangeSet()
s._ranges = xored_ranges
return s
Yes, but I see no real reason of not actually using more modern variants now
paper echo
what's a more modern variant? i dont need or want asyncio here
this is purely cpu bound computation
wide shuttle
That's the modern variant that was meant to more or less replace these type of coroutines
paper echo
but a coroutine is more general than asyncio
flat gazelle
has anyone tried to use asyncio with a custom eventloop for different kinds of coroutines?
wide shuttle
Yes, sure.
paper echo
@flat gazelle what would that even look like?
obviously we do have custom event loop policies like the uvloop based one
but they still use the same asyncio machinery
@deft pagoda what is this doing lol
also it looks like you will have errors if your next(self_ranges, None) returns None -- self_range.upper access will fail with NoneType has no attribute 'upper'
deft pagoda
real ranges are these:
In [1]: from real_ranges import *
...: from datetime import time
...:
...: bob_meeting_times = RangeSet(Range[time(8, 30): time(9)],
...: Range[time(11): time(12)],
...: Range[time(14): time(16)])
...:
...: sue_meeting_times = RangeSet(Range[time(8, 30): time(9, 30)],
...: Range[time(10): time(10, 30)],
...: Range[time(11): time(14, 30)])
...:
...: work_day = Range[time(8): time(17)]
...:
...: bob_free_time = bob_meeting_times ^ work_day
...: sue_free_time = sue_meeting_times ^ work_day
...:
...: print(bob_free_time, sue_free_time, sep='\n')
...:
...: bob_free_time & sue_free_time
{[08:00:00, 08:30:00), [09:00:00, 11:00:00), [12:00:00, 14:00:00), [16:00:00, 17:00:00)}
{[08:00:00, 08:30:00), [09:30:00, 10:00:00), [10:30:00, 11:00:00), [14:30:00, 17:00:00)}
Out[1]: {[08:00:00, 08:30:00), [09:30:00, 10:00:00), [16:00:00, 17:00:00)}
the __xor__ method is efficiently xoring two sets of ordered ranges (almost all previous implementations i've found have been O(n**2))
peak spoke
You can still use some of the generator API on coros
paper echo
nice
so how does this work, from a coroutine perspective?
you fetch part of the range with next, then send back some information for the next step in the "loop"?
deft pagoda
yeah
i send back left over bits of the xored ranges, that could be yielded back in the next iteration
paper echo
interesting, so that is basically what i had in mind
deft pagoda
the construction is a bit complicated, but the generator reduced a lot of repeated code in all these range set methods
paper echo
sequence_gen = client.start_sequence(input_data)
next_requests = next(sequence_gen)
while sequence_gen.state != 'COMPLETED':
responses = [http_send_request(req) for req in next_requests]
next_requests = sequence_gen.send(responses)
is this offensively weird
grave jolt
I used send to implement this atrocity:
!e
def null(generator_function):
def new_generator_function(*args, **kwargs):
generator = generator_function(*args, **kwargs)
state = generator.send(None)
while state is not None:
try:
state = generator.send(state)
except StopIteration as e:
return e.value
return None
return new_generator_function
def sqrt(x):
if x < 0: return None
return x**0.5
def inverse(x):
if x == 0: return None
return 1/x
@null
def inverse_sqrt(x):
inv = yield inverse(x)
sq = yield sqrt(inv)
return sq + 5
print(inverse_sqrt(2))
print(inverse_sqrt(-5))
print(inverse_sqrt(0))
@grave jolt :white_check_mark: Your eval job has completed with return code 0.
001 | 5.707106781186548
002 | None
003 | None
If the yielded result is not None, it's sent back as is. If it is None, then the execution stops and None gets returned.
paper echo
nice
grave jolt
but I don't know any practical usage of send
paper echo
that business with StopIteration is my big concern
grave jolt
(apart from what was mentioned before)
grave jolt
Huh?
paper echo
maybe in my sequence_gen thing, i would just return an empty list forever instead of raising StopIteration
because otherwise you risk throwing StopIteration's at the user everywhere
in your case it's all wrapped up in your library internals
so it doesn't matter
but if you're using generators as part of a user-facing API you need to be careful otherwise you're forcing your user into a clunky annoying pattern
deft pagoda
there's an example message server using .send from a beazley talk
though i don't remember which
!e
from collections import deque
class MessageServer:
def __init__(self):
self._post_queue = deque()
self.users = {}
def deliver(self, n=1):
for _ in range(n):
if not self._post_queue: break
name, post = self._post_queue.popleft()
self.users[name].send(post)
def post(self, user, msg):
for friend in user.friends:
self._post_queue.append((friend, msg))
server = MessageServer()
class User:
def __init__(self, name, friends=None, server=server):
self.name = name
self.friends = [] if friends is None else friends
self.server = server
service = iter(self)
next(service)
server.users[self.name] = service
def post(self, msg):
self.server.post(self, msg)
def __iter__(self):
while True:
friend_post = yield
print(f'{self.name} got:', friend_post)
salt = User('salt', ['grandma'])
grandma = User('grandma', ['salt', 'ves'])
ves = User('ves', ['joe'])
joe = User('joe', ['grandma', 'ves'])
salt.post('hi grandma')
grandma.post('hi')
ves.post('yo joe')
joe.post('k')
while server._post_queue:
server.deliver()
@deft pagoda :white_check_mark: Your eval job has completed with return code 0.
001 | grandma got: hi grandma
002 | salt got: hi
003 | ves got: hi
004 | joe got: yo joe
005 | grandma got: k
006 | ves got: k
hm
i see
so the user wouldnt typically use this directly, but you'd give them a class that wraps it
deft pagoda
probably, i think it was just academic mostly
deft pagoda
i think if you run into a generator that yields one of multiple items that you might modify and that modification affects which items are yielded --- send will be pretty handy
paper echo
wait, can you even use .send with a class-based iterator?
or do you have to implement all that stuff manually
deft pagoda
yeah, you can use .send and __iter__ it's the same thing really
paper echo
hmmmmm.... so i'd want to subclass collections.abc.Generator, then manually implement __next__ ,send, close, and throw
as well as return self from __iter__
deft pagoda
something something maybe
paper echo
heh
deft pagoda
i didn't find out about send until ves showed me during aoc last year
but now i'm a few dozen beazley talks deep
beazley does build up curio from generators, but i don't think he uses send at all --- it's a really nice looking library if you get curious
paper echo
yeah i saw that one
deft pagoda
the code is easy to grok
paper echo
really cool talk
i love his presentation style
great job of building on principles, pacing is perfect
class StatefulCoroutine(collections.abc.Generator):
def __init__(self):
self.state = "INITIALIZED"
def send(self, prev_action_result):
self.state = # compute a new state
next_actions = # compute the next actions to be sent back to the caller
return next_actions
def close(self):
self.state = "CLOSED"
def throw(self):
# ???
def __next__(self):
return self.send([])
idk what im doing
paper echo
Another possibility might be to send and change the current state mid-loop
I still have no idea how throw would work - maybe just immediately raise whatever was thrown?
!e ```python
def chaotic_counter(i=0):
while True:
inc = yield i
if inc is None:
inc = 1
i += inc
cc = chaotic_counter()
for k, val in enumerate(cc):
print(val)
if k == 5:
cc.send(500)
if k >= 10:
break
@paper echo :white_check_mark: Your eval job has completed with return code 0.
001 | 0
002 | 1
003 | 2
004 | 3
005 | 4
006 | 5
007 | 506
008 | 507
009 | 508
010 | 509
011 | 510
😬
flat gazelle
throw essentially makes the current yield it is stuck on raise an exception
paper echo
right, im wondering how an implemention of a subclass of collections.abc.Generator should look
ive never seen one in the wild
frozen hamlet
i need help to implement a path directory from MTP connection, can i ask in this tag??
unkempt rock
e!
import time
for i in range(5):
print('shsh')
time.sleep(1)
radiant fulcrum
#bot-commands
magic python
do python requirements support https://semver.org/ , is there any guideline / suggestion on when to use it? I think i've heard ~= for packages that have had a major release and == if they haven't, i can't seem to find anything on it though
unkempt rock
does anyone know how to do a command in a bot dm instead of doing it in a channel? so like do a !help command by dming the bot and it shows u the message in the dm
so like executing a command through dming the bot
peak spoke
@magic python something like this? https://www.python.org/dev/peps/pep-0440/
unkempt rock
I liked that MessageServer code.
narrow karma
is there a standard for how people structure their files and directories in python apps?
for example, if I see a folder called core in the root directory of a python app are there generalizations that can be made about the fucntionality of the code inside that folder?
wide shuttle
I don't think it's standardized to that degree, although files like cli.py are common when an application also has a command line entry point
There are some guides out there on how to structure a project in general, but it's obviously subjected and those articles are typically opinated (most are honest about that as well). This is a commonly linked one: https://docs.python-guide.org/writing/structure/.
narrow karma
thank you
paper echo
let it be known that i am strongly in favor of using src/ http://bit.ly/pypackaging
@narrow karma ^
magic python
@paper echo i thought src was old school for some reason
wintry gazelle
I'm trying to package a library i created, but... the PyPI Guide isn't clear. What extension should my LICENSE file be in?
wintry gazelle
packaging_tutorial
├── LICENSE
├── README.md
├── example_pkg
│ └── __init__.py
├── setup.py
└── tests```it only says this
I believe it doesn't need one...?
Then how would i open it?
wintry gazelle
... dont tell me this is one of those awkward moments
where theres something i shouldve learned back then
and i didnt, and stood with it for years...
and yes i am
gleaming rover
okay
it's a Windows thing.
but on Mac/Linux derivatives, you don't need an extension to open stuff
(kind of glossing over the details but BASICALLY)
peak spoke
I like keeping the code in a folder, but believe a project name or something more descriptive than src should be used
wintry gazelle
I always used windows 😅
gleaming rover
e.g. if I want to see what's in my Makefile, I can just run more Makefile
edit? vi Makefile
(btw not really sure if this kinda stuff is on topic for this channel)
wintry gazelle
Oh, i thought it was... xd...
gleaming rover
I like keeping the code in a folder, but believe a project name or something more descriptive than src should be used
@peak spoke conceptually I can see the value ofsrc
as opposed to test, res, etc.
which should be on the same level...?
wintry gazelle
e.g. if I want to see what's in my
Makefile, I can just runmore Makefile
How would i write into it, in windows?
just need to put a license there ngl
gleaming rover
if you're using an IDE, create a file without an extension directly
if you're not, use like Notepad++ or Notepad or something (remove the extension after if you want)
wintry gazelle
Can i just make a .txt file, write and then take the extension out?
wont it get corrupted or smth?
gleaming rover
nope
what's in the file is data. the filename is part of metadata (data about the file)
a rose by any other name would smell just as sweet
wintry gazelle
Dunno man, i installed Windows on my PC, and it'd be a headache to put it to linux, would have to format it and all
I'll just stay in win, but ty 😄
zealous yoke
In Windows there is WSL. Dare I say it, Windows is the most versatile.
mossy pumice
WSL is amazing. I probably couldn’t use Windows without it
topaz horizon
first of all i am not advanced
feral cedar
ok
haughty fiber
hello I'm am new to python and would like to get some advice on it
visual shadow
try #python-discussion this room is a bit more specific to advanced discussions around the language itself
fervent pulsar
could you unpack a list with a new line in an f string?
I'm thinking somehow using the walrus operator
radiant fulcrum
- not really topic for this channel
- Why would you do that when
.join()method exists
radiant fulcrum
yh, so just use standard string formatting/concatenation
f"some {thing}" + "\n".join(["a", "b"]) is still valid
fervent pulsar
i set it as a variable but you can't have a back slash in {} in an f string
placid pivot
Why i would want to use a tuple instead of a list?
radiant fulcrum
immutability ish like behaviour
placid pivot
So if i dont care to be changed its ok?
radiant fulcrum
yh
placid pivot
Oh ok then thanks!
radiant fulcrum
if you only need to make it once and not modify it you generally want to use a tuple
placid pivot
Ok i was wondering why i would want to use since o never met a user case before but i guess i will in the future
fervent pulsar
I was trying to find a good use for the walrus operator that I haven't seen to get a better grasp on it
radiant fulcrum
i mean in your case its pretty pointless
placid pivot
Also python crash course skip tuples :/
fervent pulsar
the book?
placid pivot
Yeah
dusty haven
@placid pivot tuples are also smaller and faster to create
placid pivot
Oh ok thanks for the input!
@fervent pulsar im reading intro to python for computer science now and it goes way more in depth
visual shadow
make sure to move further discussion away from this channel, perhaps #python-discussion is a good fit for this.
placid pivot
Ok
fervent pulsar
@placid pivot the one by John Zelle?
swift grove
anyone used Panda3d?
anyway I want to rant! lol
why the f to deform your model you need to loop in python through each vertex instead of giving user a function to write all vertices positions at once? such an overhead unbelievable. it could give 100x speed improvement on such a basic operation
i understand developers of the engine have their reasoning but damn
so much frustration
and i thought i've finally found a worthy way to 3d program in python
now need to dig through cython again or some sht
flat gazelle
try throwing a numba.njit on it
tulip spear
Does anyone know the International Baccalaureate?
Basically I have to write a 12 page research paper using high level 12th grade mathematics and beyond
Is there any raw investigative mathematics in image processing?
Like I just use openCV/scikit functions which are already implemented
So I don’t do any of the maths
Are there any areas where I actually have to do maths instead of just plugging in values or even worse, just using already implemented functions?
(Not sure if this is the right channel for this but it doesn’t seem general and it doesn’t fit the topical channels either)
teal yacht
Your question could be translated to "Is there anything that hasn't been done in image processing?" and the answer is yes, however, it's not like it's easy to be to find what exactly hasn't been done while being useful in some way
Also, the vast majority of image processing research these days is in deep learning, so you will pretty much have to use some libraries to do the grunt work for you if you don't want to waste time
grave jolt
CPython question. Why do functions store None as a constant even when they don't need it?
!e
import dis
def f():
return 1
print(f.__code__.co_consts)
dis.dis(f)
Maybe your question would fit #data-science-and-ml better too
fallen slateBOT
@grave jolt :white_check_mark: Your eval job has completed with return code 0.
001 | (None, 1)
002 | 4 0 LOAD_CONST 1 (1)
003 | 2 RETURN_VALUE
i'm pretty sure it's due to the return value being None by default
grave jolt
but the None constant isn't used anywhere in the bytecode
teal yacht
https://stackoverflow.com/questions/27667747/what-is-none-doing-in-the-code-objects-co-consts-attribute this guy says it's because it's cheap enough to have it so why not
grave jolt
Well, maybe
The answer actually doesn't answer the question. It answers a totally different question.
teal yacht
In your simple case, it is obvious to us humans that there is just the one RETURN_VALUE opcode, but to extend this to the general case for a computer to detect is not worth the effort. Better just store the None reference and be done with it, that one reference is extremely cheap.
grave jolt
Well, it's entirely possible to check whether the None constant is used anywhere in the bytecode. But maybe it's not worth the effort.
peak spoke
You would have to check the paths for it in most cases, I guess just not worth doing when you can store the None that exists anyway
grave jolt
I didn't say that you should check whether the return None is needed at all
In the cases where there are no branches <#internals-and-peps message>
it doesn't put a return None at the end
so the None constant is unused in the bytecode
anyway, that's not a big deal, I was just curious
peak spoke
The cost of it is very negligible, and would still need some special handling so the trade-off of lower maintenance may just be better than the optimization
flat gazelle
that would be an interesting esoteric way to get None, ```py
(lambda:0).code.co_consts[0]
+1