#internals-and-peps
1 messages · Page 69 of 1
[a,b][bool]
it's fun to use, but has much more limitations than a ternary
if i theoretically wanted to patch in @ to be function composition... i assume i'd have to write that in C right? could i do it as a C extension or would i have to patch cpython itself?
i found out the hard way that you can't just assign function.__matmul__ = ...
you would have to patch CPython directly I think, though in theory memsetting the function type may work, but it is quite hacky and probably error prone. You cannot add arbitrary attributes to the function type.
sad
i wonder how that would work. quite beyond my normal usage of python and i'm not exactly a good C dev
I thought all built-in types had a value for each dunder method specified in their C implementation, which would often be null
correct
I made a C extension a few months ago and I remember there being something like that.
you can use forbiddenfruit library to mess with dunders btw
I tried forbidden fruit exactly because I wanted to implement function composition
I don't like writing extensions in C because they become unnecessarily boilerplate
part of the problem is that you don't get a built-in reference to the function type, that I know of
@paper echo yeah
forbiddenfruit doesn't seem to work with FunctionType
okay then, not this time, haha
another problem is that i want @ to work with all callables ultimately
what was the use case of doing that though?
meaning that it would probably have to be baked into the parser right?
well, isn't less boilerplate part of writing in Python, and you write C extensions so that the performance critical parts of your beautiful python code get C-like performance?
i just dont want it to work for "functions" and break for "anything else"
callables are callables and should all act the same
@paper echo then what would you do if a class implements __call__ and __matmul__?
heh 😄 probably defer to matmul
@boreal umbra I was saying boilerplate not compared to python, but compared to Rust and PyO3, which did a really nice job in connection between Rust and Python
actually thats a good point @boreal umbra
however there should be a way to enable __matmul__-as-composition by default, maybe with a mixin class you can inherit from
@cloud crypt I haven't worked with Rust but if I ever write another extension and Cython isn't enough, I'd probably look into Rust.
here's a small module I wrote some time ago, which implements XOR function that applies to each byte in given bytes input. You can see how nicely PyO3 allows writing extensions without boilerplate here
that's a python extension written in rust?
xor function that is exported to python can already take native Rust types, which removes boilerplate of doing it by hand, and module initialization is just beautiful.
@paper echo if this were a PEP, I think it would be hard to sell the core devs on the idea that __matmul__ should now automatically have a default implementation for any class that implements __call__, if that's what you're suggesting
that's a python extension written in rust?
@paper echo yep
because that locks the specification down even further
@boreal umbra yeah i suppose that is what im suggesting. and i agree its an unappealing edge case
its also ugly because you lose function composition when you define __matmul__
I really want func comp for functions and lambdas though.
but that's really just a matter of using @
which is a shame that they stole it for matmul
because @ is already used for decorators, meaning that it's already kind of a function composition mnemonic
it's still funny how @ as matmul is not even used in standard types
as a very frequent numpy user i think @ was an arbitrary and bad decision
I don't know that it's stolen for matmul per se
it benefits me at the expense of like... the rest of the language
how did you do matmul before?
x.dot(y) or np.dot(x, y)
interesting
which is arguably done more like that even now
yep that too because a lot of scientific code needs to run on relatively old python versions
Is there a way to overwrite default str class so I could add something to it?
I know there is a way to just make another class which inherits from it
but I want to make this in a way so that it would use this new class when I make a new string
class str(str):
# Some extension to str
a = "some string"
a should use the new class instead of the default str
e.g. a HPC or Spark cluster that's still on 3.5
yeah
@radiant scroll I think you have to state that you're using your version of str each time
@radiant scroll you can't change how "" works but you can try to use the Forbiddenfruit library linked above to patch str itself
probably better to just subclass str though...
yeah
alright I'll look into that library, thanks
note that there are some caveats to subclassing builtins https://stackoverflow.com/q/7255655/2954547
you know what? im just gonna use Hy more:
(comp f (comp g h))

I also just used np.dot()
I love typing less, don't get me wrong, but np.dot() is a lot easier for most people to understand than @ when sharing code
thats just a matter of familiarity though. if you start using @ and tell people "it's matrix multiplication" people will get it
5. Do not provide or request help on projects that may break laws, breach terms of services, be considered malicious/inappropriate or be for graded coursework/exams.
Is sum(range(n)) a zero-cost abstraction? From my benchmarks it seems no, but I can't imagine why they wouldn't write the compiler to handle that
sum always iterates over an iterable. adding a special case for range objects would be silly, and a custom __sum__ dunder would be seldom useful. (n * (n - 1)) // 2 is not that much worse
Yeah, implementing those builtins for classes that potentially can't use them, would in my mind, be bad.
I can't see it being useful unless Python starts allowing subclass specific dunders. Like you subclass Iterable somehow and want to define how sum, filter, etc. work for that subclass, that may be cool, but still fairly limited.
some languages can do that, for example R (at least afaik). Nim as well.
but python does not have that because it would complicate things quite a bit
I can't imagine why they wouldn't write the compiler to handle that
If you mean a compile-time optimization for the specific case of summing arange- the Python compiler just does very, very few of those.
like, literal lists not bound to a name are converted to tuples, a,b=b,a gets optimized a bit... but generally speaking, it doesn't really optimize things for you.
It's effectively O(n^2) -- I consider that a lot worse. It'd be really nice if it were O(n) since the math-based alternatives can be fairly complex (e.g. yeah sum(range(n)) is easy enough to memorize and write the formula for, but what about sum(range(j, n, 3)). Mmm...
It's effectively O(n^2)
Huh? What?
it is O(n)
Oh sorry, derp
I was thinking about n(n+1)/2 vs O(n) -- it's O(n) vs O(1)
still, worse by a factor of n, right?
well, yes, using the analytic formula for sum(range(n+1)) would be O(1) compared to the O(n) of actually summing it.
I mean really you just have to define your class around __iter__ to dictate how you want sum and the like to function
compiled languages have these optimizations, to fit one in python would be ..tough
you could just do an if check in the sum impl
@swift imp nope, there's no __sum__ dunder.
I know that
it is not too bad to make the sum over range formula, IIRC it is just
def sum_range(start, stop, step):
size = stop - start
return start * size + step * size * (size - 1) // 2
in practice you would test it with hypothesis
https://github.com/search?l=Python&o=asc&q="sum+range"&s=indexed&type=Code
I feel like it's ubiquitous enough that that kind of abstraction should definitely be handled by a compiler... But yeah, I digress, thanks for the help!
I also think that due to name sum having it do something outside of the typical mathematical sum would be odd
I mean at the end of the day if yo really need cusotm sum behavior, you do something with __iter__ and pass a generator expression that is using your class to sum
and if it was really that frequent of a thing, you make a method to return that generator or abuse __call__
like sum(class_obj())
Right... I guess I'm just interested in Python being faster by default. Do they not want those kind of zero cost abstractions because it clutters the source code or because it could somehow introduce unexpected behavior?
Its because __sum__ would be too specific to just classes that are iterable. Its not general enough. If you read PEPs or discussions that may lead to a PEP the biggest rejection I see is "this is too specific to just your use case" and/or "this breaks backwards compatibility, and therefore not worth it"
It'd either be a big change to how sum works, or a possibly unexpected optimization; it's fairly trivial to implement yourself when you do need to cut down on the cost of the operation
you cant even use sum currently with iterables holding non-numeric types. If they implemented a dunder where you could change that, it would confuse so many beginners, and learning material. That is something else they talk about often too. How is this going to complicate it for newbs.
sum([[1],[2,3]], []) does work in CPython, but not as per the language spec afaik.
You have to set the default val
Oh, default is zero, huh?
still cant sum strings lol
tells you to use ''.join()
I think thats a poor design personally, but whatever. Must be GvRs
there should be one clear way to do it
inconsistant policy
Sum just adds up things so for example even this works
class Addable:
def __init__(self, val):
self.val = val
def __add__(self, other):
return Addable(self.val + other.val)
print(sum((Addable("a"), Addable("b"), Addable("c")), start=Addable("")).val)
strings are disallowed directly because adding them up is inefficient and join does it properly
but you could do 'a'+'b', __add__ is defined for strings
You're not going to do it more than 2 or 3 times in code through
though there is a CPython optimalization which makes
s = ''
for string in strings:
s+=string
```the fastest way a lot of the time (if the refcount of LHS of string `+=` is 1, it will mutate the string inplace rather than copy)
There's also the overhead of a custom python object but the difference is quite noticeable
In [5]: import itertools
...:
...:
...: class Addable:
...: def __init__(self, val):
...: self.val = val
...:
...: def __add__(self, other):
...: return Addable(self.val + other.val)
...:
...:
...: my_strings = ["".join(elem) for elem in itertools.permutations("abcdefg")]
...:
In [6]: timeit "".join(my_strings)
38.9 µs ± 1.34 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
In [7]: addables = [Addable(string) for string in my_strings]
In [8]: timeit sum(addables, start=Addable(""))
6.44 ms ± 279 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
>>> sum("test", "")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: sum() can't sum strings [use ''.join(seq) instead]``` interesting
thats probably there because most of the time summing strings is probably user error
ugh sometimes I wish Python had a do-while operator. I don't like doing "while True... if condition: break"
I could use walrus operator, but that just looks a bit too messy
while True and condition?
I mean the only way to emulate a do-while is like this (or variations of this):
while True:
if some_condition:
break
# do stuff
while not some_condition:
Sure, but then I'd have to do:
response = ...
while response.status = StatusCode.OK:
# do stuff
response = ...
oh if youre doing that
then thats a bad idea in general
Requests + Loops == Bad
Walrus operator could work
while (reponse := ...).status = StatusCode.OK:
# do stuff
response = ...
Not necessarily, especially if the API expect you to requests in pages
Im following all the requirements the api set with time-outs, polling, etc. But in this case a do-while like Java had would look so much prettier
I mean youll be able to get a status code easily as a int
so you could do while code == 200 for example
sure, but that's not the point. It's the structure of the code that looks bad. I either have to do a while-True and break it explicitly, generate a response before and during the loop, or use the walrus operator.
I mean i wouldnt say its a bad structure in the case where you know you should only be iterating while the code is x
walrus is generally a bad idea if you intend to make it available to others because 3.8+ only
Yeah I agree with that. It's mostly just that this is one of the few cases a do-while actually makes sense it it's kinda weird that python doesn't have one. But I digress
I mean, the only use case I'm aware of from a do-while versus a while loop is that a do-while will always execute at least once.
Oh, status code
If you hate the break, maybe wrapping the loop in a function and returning would feel better?
wishing Python had a do-while is a feeling I haven't felt yet
Yeah I get it. It's not often useful and it's easily rewritten using normal whiles and conditions.
first_run = True
while first_run or <actual condition>:
first_run = False
<body>
That's the cleanest parody of a do-while loop I can come up with, and it still looks pretty icky
why not wrap your body into a function, call it once, then do a check and start a loop?
That would be the optimal solution in the majority of cases, yeah
Guys what's the different between VGG16 model and efficientdet model ?
Are explicit parenthesis when defining a tuple considered good practice ?
Yeah, I'd say so
Why would you type on a snake
Are explicit parenthesis when defining a tuple considered good practice ?
@final whale in general, yes
main exception I can think of is returning what are conceptually multiple values from a function
if I make a venv and just copy the venv folder into some random computer that doesn't have python installed, would the binary in the venv folder still work?
Not unless it's set to portable iirc
qq: is distlib mature enough or setuptools is still the standard
as a distribution library
The answer to both of those is yes
It is mature, but setuptools is still dominant.
it says that your icon is invalid
Try in a help channel
how variables works in python?
I read a post (pt-br) talking about and I have a lot of doubts about this subject
the variables are "names" that point to the object?
Yes, and when you pass the name to a function for example it creates an another reference to the object
sure when i create a parameter
this parameter is another name equals the argument?
ya so if you have:py def func(arg1, arg2): ... func(x,5)
you're passing x and 5 as those arguments
in other words your defining arg1 = x = whatever and arg2 = 5
i'm confused
how so
i'm creating a project that uses a "cache" to save some data
and this data is used in varios parts of the code
with diferrente identifiers, when i moddify one, all is modified
i want to undestand how its works
Every time you pass the name somewhere, it creates an another reference to that same object, just like in the function.
So if you then modify the object in one place without reassigning, everything pointing to that object will also see the change, since it's still the same one
and if the name is not an variable, is it and object type an integer?
Not sure what you mean with a name not being an identifier
def function(parameter):
pass
function(1)```
parameter will be receive the address of 1?
Yes it'll get a reference to that 1 you created. The object is created by python and gets 1 reference from the literal, then an another reference is created for the function body and the reference from the literal is removed since it's no longer used
hi. could you recommend me some hub for macbook pro 2019? i want to plug 2 monitors. it would be nice if it has multiple thunderbolts because my first monitor is plugged by thunderbolt.
from __future__ import <something> isn't an import statement according to the compiler, that's interesting
I'm guessing that the compiler will catch any future_stmt syntax node and activate the feature, so it doesn't have to execute code to do so
Makes sense, you aren't really importing anything (the way you normally do), but you are changing the way the compiler and maybe the interpreter treats your code
Yup
Here are the docs if you are interested :
Import statement : https://docs.python.org/3/reference/simple_stmts.html#import
Future statement : https://docs.python.org/3/reference/simple_stmts.html#future-statements
is there a functional reason bool("string") returns True?
all non empty sequences return true for their boolean value
so it's just to check if a string exists?
essentially, if len(sth) == 0, then bool(sth) is False, otherwise True, unless len is inapplicable
this holds for the entire stdlib AFAIK
hm that's neat?
"" is falsy, as is [], {} and set()
super handy for checking if a string is empty, list is empty, etc etc. makes those boolean checks almost beautiful
I don't really like implicit coercion to bool in conditionals
and it might cause some bugs
def register_with_options(cluster_name, options=None):
"""
Register cluster with optional options.
:paaram options: is a dict. If the dict has some
missing options, they are added in the dict (with mutation)
"""
options = options or {}
...
notice the bug here?
Yea I am ambivalent about it. It's super handy most of the times but if I'm writing a library I am doing very explicit checks. In this case if options is None. Or even using a sentinel so padding None is possible.
In this case the bug is that if options is {}, the function will not mutate the passed dict, contrary to what's said in the docstring.
I find it makes the code more readable in almost all cases, and for cases where the behaviour is not desired it can be easily done explicitly
The problem is to know a priori if you need to.
I guess
Sir how the internal code of the input() print()
Subprocess ()
Cgi.Fieldstorage ()
Getvalue()
are working behind the screen
How ipconfig command output is being generated
How can we know how the internal data flow is going on
its all writing to the Stdout Stdin and Stderr pipes
input() is just reading Stdin with a \n delim
print() is just writing to Stdout with a \n end (by default that is)
Subprocess is built off of os.exec if i remember
Pep 505 should happen. As clear as x = y if y is not None else [] is, its also very verbose. Considering how widely used None is to indicate the nonexistance of something, I think the none-coalescing operators would be really good. Even if they wil appear strange at first sight. Of course I bet because of the examples using none-coalescing to process default positonal args and kwargs, that GvR and likes believe it could be handled by PEP 622 (pattern matching)
The whole idea of maybe type operations is really fucking interesting imo.
one could argue that x = y or [] is a thing
In a lot of cases you specifically need to check for None as a falsy value would still be considered valid input
hey
so, the function getattr is a heavy loaded to be executed?
so, i made an registry system for my game engine
getattr is just a dict lookup
and i am updating some code to make it more efficient
getattr is pretty much
object.__dict__.get(key, default)```
oh
def Get_RegKey(self, keyName, valueType=0):
return getattr(self, 'KeyType_' + str(valueType))(keyName)
def KeyType_0(self, keyName):
return self.LoadedRegistryKeys_Data[self.LoadedRegistryKeys_Names.index(self.CorrectKeyName(keyName))]
def KeyType_1(self, keyName):
return int(self.LoadedRegistryKeys_Data[self.LoadedRegistryKeys_Names.index(self.CorrectKeyName(keyName))])
def KeyType_2(self, keyName):
return float(self.LoadedRegistryKeys_Data[self.LoadedRegistryKeys_Names.index(self.CorrectKeyName(keyName))])
def KeyType_3(self, keyName):
return self.LoadedRegistryKeys_Data[self.LoadedRegistryKeys_Names.index(self.CorrectKeyName(keyName))].lower() in ("true")
@radiant fulcrum this is what my code looks like
so, the function Get_RegKey is that is called
insted of making an if-else ladder
but yes that would work
@radiant fulcrum but it does work faster than using a if-else ladder?
sorta?
you could do a dict instead
@radiant fulcrum i tryied this, and it does not worked, the first case executed and then it started executing every case listed, insted of executing the first one and stopping there
its python is never gonna be 'fast' to the point where speed shouldnt be sacrificed over readability
and a block of ifs are never really gonna be that readable if theyre big
hmm
Hey anyone good in machine learning?
this is not the right question for this channel @forest stone
please note the topic description
!paste
Pasting large amounts of code
If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pydis.com/
After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.
Trying to understand why Python does this looping behaviour
Maybe related to avoiding circular references for garbage collection
But I dont understand why loop as 12131213 rather than 1212
hmm, cool
manually run gc in between?
def f():
return lambda: f()
cur = f
res = []
for i in range(100):
res.append(id(cur))
cur = cur()
#print(res)
s = sorted(list(set(res)), key = res.index)
print([s.index(el) for el in res])
[0, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1]
actually, wait, it does look obvious why it's so
The first element is f itself. The second (1) is f(), so lambda: f(). The third is f(), so lambda: f() again...
so the real question is why doesn't it loop 0,1,1,1,1,...
It's interesting because your example loops differently to just calling it over and over again in the same line
oooh more fuckiness
>>> y = f()()
>>> z = f()()()
>>> a = f()()()()
>>> b = f()()()()()
>>> c = f()()()()()()
>>> x
<function f.<locals>.<lambda> at 0x7f9b004fc790>
>>> y
<function f.<locals>.<lambda> at 0x7f9b004fc8b0>
>>> z
<function f.<locals>.<lambda> at 0x7f9b004fc820>
>>> a
<function f.<locals>.<lambda> at 0x7f9b004fc9d0>
>>> b
<function f.<locals>.<lambda> at 0x7f9b004fc940>
>>> c
<function f.<locals>.<lambda> at 0x7f9b004fcaf0>
>>>
All different
Ah yeah ofc
That's not what undefined behaviour means
It's because the interpreter stores the value returned to _ as a ref
Implicitly
So after the first call, the result is referred to by _ as well
That would explain some looping if Python recognises that the functions are identical and just returns _ from before
But that doesn't really explain 12131213
That should have a max memory of 2
1,2,1,3 is cursed, yeah
like, it implies that it can't just be determined by the object itself
Look into how pythons garbage collector works and you'll start to understand the pattern
1() returns 2 or 3 depending on ...something, in your example.
The _ returning might be a good lead, because
>>> a = f()
>>> b = a();c=b();d=c();e=d();f=e()
>>> a
<function f.<locals>.<lambda> at 0x7f431881f700>
>>> b
<function f.<locals>.<lambda> at 0x7f4318584af0>
>>> c
<function f.<locals>.<lambda> at 0x7f4318584b80>
>>> d
<function f.<locals>.<lambda> at 0x7f4318584c10>
>>> e
<function f.<locals>.<lambda> at 0x7f4318584ca0>
>>> f
<function f.<locals>.<lambda> at 0x7f4318584d30>
Is there a way to detect if a builtin class (ei int) is called I can’t seem to find anything
hm
override builtins.int?
Is there a way to detect if a builtin class (ei int) is called I can’t seem to find anything
@signal tide yeah, assigning tobuiltins.intappears to work
so you replace it with a hook
or do you mean in general
That too but thanks
hello, does anyone knows django?
That too but thanks
@signal tide I don't think it's possible in general since there's no unifying interface...? you'd have to hook each of them individually
hello, does anyone knows django?
@ancient tide try #web-development
Wdym by each of them
you mean all the type objects (int, float, etc., right?)
Ya
okay I am guessing you'd need to replace each of them individually
though you could automate the process of course
Ok thanks
is there a way to link a variable to a value of a dictionnary to avoid typing all the way of the value when I want to modify it?
A function?
@signal tide you could hook thier init using something like forbiddenfruit
I’m still a bit lost on the whole hook thing but ty I’ll have a look
is there any way without extending to C++/C to read memory addresses in Python?
@north karma id()
You can use ctypes for that
Sorry, more specifically of non-python objects/memory
Like from other processes?
Ohhh you'll need ctypes
!d ctypes.string_at
ctypes.string_at(address, size=-1)```
This function returns the C string starting at memory address *address* as a bytes object. If size is specified, it is used as size, otherwise the string is assumed to be zero-terminated.
Raises an [auditing event](sys.html#auditing) `ctypes.string_at` with arguments `address`, `size`.
You can use string_at
thank you 😄
@north karma if it's a struct you can do some cleaner stuff too
it should just be a string, but it's returning b'1' instead of the actual memory value. any ideas?
Is your string null-terminated?
did you set the size?
shouldn't default size be okay? not sure if null terminated
If size is specified, it is used as size, otherwise the string is assumed to be zero-terminated.
so like
presumably the value you want to read is of a known type and therefore has a specific width, right?
you should specify that
otherwise it's just going to read memory until it hits a null byte (default behaviour for C strings)
which could be whenever
shouldn't default size be okay? not sure if null terminated
@north karma in other words, no, it would not
If you read until null you can also read until segfault. That's very bad obviously.
Lol
https://bellard.org/pi/
at the top it says they computed 2700 billion digits of pi with a regular computer
but it also says the best algorithm for finding n digits of pi is O(n^2)
how is this possible
shouldn't n >= 10^4 be infeasible
it could have a small constant
but it also says the best algorithm for finding n digits of pi is O(n^2)
that doesn't seem right
ent_types = sorted(set(list(sum(matrix_data.keys(), ()))))
this is a line that I just wrote
I demand function composition.
set(list(?
this is trying to get all the strings in Counter[Tuple[str, str]] into a sorted list with no repeated elements
Can i ask a question here? i wasn't lucky enough in the help section
Its about custom context-manager
@edgy tulip this particular channel is especially not for help sessions but if you ping me in the one you have open I might be able to help.
Where do I get started on machine learning in python?
@trim gull sounds like a good question for #data-science-and-ml. However I don't know that there's a one stop shop for "machine learning in Python" because there's a lot of applications of machine learning and it requires a solid foundation in the language itself and linear algebra.
@trim gull the Hands-on Machine Learning in Scikit-learn, TensorFlow and Keras book is really good. But you can get a long way with some tutorials on YouTube. Either Corey Schafer or sentdex is bound to have a series on the topic
Well, where would I learn maybe basic machine learning revolving around weighted matrices and decision trees?
Sounds like you want reinforcement learning or something.
machine learning
it's probably best to start with specific types though
rather than trying to do it all at once
reinforcement learning + supervised learning is so effective!
I don't think this discussion is on topic for this channel, but it is for #data-science-and-ml
Where is the best location for pandas assistance?
Probably also data science, coincidently
Though you can always open a help session. #❓|how-to-get-help goes over that.
How do python linters typically handle the possibility that an object might be None?
Should you always be using Union or is it a special case in some way?
I don't think there's any special handing; typing provides Optional which is an alias to Union[type, None]
Well, maybe not an alias, but equal to
My friend strongly prefers languages with explicit typing but he mentioned that even languages that he uses have issues with null
Not sure what
The issue with null is that it effectively is a hack that breaks static type systems, null is a possible value for (almost) every type yet calling a method from it will crash your program inadvertently
The meaning of null is also convoluted, sometimes it means an error, sometimes a normal return value, and you have no real way to know
Java, for instance, has NullPointerError despite not having pointers.
hi
And the inventor of null in the Java object model famously said he regretted it. https://medium.com/@hinchman_amanda/null-pointer-references-the-billion-dollar-mistake-1e616534d485
It was copied from Java into C#, so you can blame Tony Hoare too, @spark magnet 😄
adds Tony to his list
Java, for instance, has NullPointerError despite not having pointers.
@raven ridge aren't pointers still an implementation detail for non primitives?
<script type="text/javascript"><!--
google_ad_client = "ca-pub-1234567890123456";
/* My Ad Unit Description */
google_ad_slot = "1234567890";
google_ad_width = 728;
google_ad_height = 90;
//-->
</script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script>
<!-- END Google AdSense Ad Unit code, provided by Google -->
</div>
was dis?
dis is js
@halcyon shoal this isn't a help channel
Yeah man
just wanted to say
that i like js more than python
you know, because it's similar to C++
I must be missing the context here
I have in limited circumstances but this channel is specially for advanced python discussion. You might want to check out an off topic channel.
isn't there that xkcd about this
Bye 👋
@raven ridge aren't pointers still an implementation detail for non primitives?
@boreal umbra sure. ButNullPointerErroris public interface. That elevates the concept of pointers to be more than just an implementation detail.
Fair enough
Strictly speaking I think the name of that exception doesn't have to be true to what it means as long as the spec stays consistent
But it's weird that they'd refer to an implementation detail in the interface
Yeah, exactly.
Is that basically the same as the attribute error you get if you call a method with none?
Yes and no... Unlike an AttributeError, which is an ordinary part of Python's data model, NullPointerException is specific to accessing attributes or methods on null
It's a special case needed because of a special case, that is.
!e print(None.__class__.__name__)
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
NoneType
In Python it's possible to access attributes of None - there's no special casing that leads to that AttributeError; you get it if you try to access an attribute that doesn't exist, but accessing one that does works just fine.
Contrast that to Java, where NullPointerException is a special case, that can happen only on null, and that happens whenever you try to access any attribute of it (at least, AFAIK)
As I recall NoneType and None don't have any attributes native to that type, just what it inherits from object
speaking of which, why can you only arbitrarily change the class of objects that didn't start out as builtin types?
I use that ability in a library I've been working on where I sometimes want objects to have extra methods, so I switch them over and then switch them back
or deep copy them and switch them.
what do you mean by "change the class of objects"?
like, monkeypatch new attributes onto obj.__class__?
yesss
well, not necessarily
!e
class A: pass
class B: pass
a = c = A()
print(type(a))
a.__class__ = B
print(type(a))
print(a is c)
@boreal umbra :white_check_mark: Your eval job has completed with return code 0.
001 | <class '__main__.A'>
002 | <class '__main__.B'>
003 | True
so as long as the attributes a given instance are compatible with the attributes (and their respective properties) of another class, you can change to that class without any errors
!e
d = {}
d.__class__ = list
I had no idea __class__ was writable.
@boreal umbra :x: Your eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 2, in <module>
003 | TypeError: __class__ assignment only supported for heap types or ModuleType subclasses
huh I guess that error message is informative
my naive guess would have been that it only works with non-__slots__ classes, but it even works with __slots__.
Changed in version 3.5: __class__ module attribute is now writable.
I missed that change entirely 🙂
I wasn't even using Python when 3.5 was the latest version
I'm still mostly writing Python 2+3 code. My company is still working on the Python 3 migration, so while many app teams have moved to Python 3, I'm writing infrastructure-level libraries that still need to work for the teams still on 2.
I quickly ran into some things I missed from python 3 when I had to use 2 for one project
there's not a lot I regularly wish I had - f-strings are probably the biggest, and maybe subprocess.run
And maybe super() without arguments...
if only because the alternative is sorta verbose.
Nothing really was a huge dealbreaker, but things like the super you mentioned, lack of a proper range object, typehinting not being in the syntax etc. Haven't work with strings much but from some of the questions on the server those also look like they can be a bit of a pain
lack of a proper range object
Hm - why was this a problem for you? You can userange()exactly the same in 2 and 3, and the only difference is that some operations will have different performance characteristics.
but for small N they're pretty irrelevant...
I needed to do a lot of containment checks on certain ranges, was easily replaced by some math but lost some of its clarity
you could also use xrange for that in 2
The non constant time complexity was an issue in this case
I thought xrange also had an optimized __contains__
Seems to be a normal iterative approach
yeah, looks like it - https://github.com/python/cpython/blob/master/Objects/rangeobject.c#L601 is missing from https://github.com/python/cpython/blob/2.7/Objects/rangeobject.c#L165
Is there a dunder method that gets called on a class when a subclass of it gets defined? Or can it only be done with metaclasses?
__init_subclass__? I'm not sure
Whenever a class inherits from another class,
__init_subclass__is called on that class.
Oh, that actually looks perfect.
EDIT: Yup, that was exactly what I needed, thanks!
you can also use __init_subclass__ to grab keyword arguments from class definitions
which is cool
If I wanted to make a subclass of ordered dict, which allowed for positional indexing or by key. How would I tell have it return slices?
For example, id like to be able to return like odict_subclass[0:3] and have it return an odict_subclass with just first 3 k-v pairs while retaining the order
Would that be a situation where id use __new__?
How are you going to differentiate between key access and slice access?
{slice (None): 6}[:] is valid python
right, yeah
I was gonna overrite __getitem__
Oh, nvm
Restrict keys to str
Apparently slices are not hashable
yeah, I just checked
You could just check if it's an instance of slice and not bother restricting it
Although you want to use integers for indexing, probably
No, that's what I meant. This may help differentiate slices, but not regular index-like access
Although I'm personally not sure of having an API in which [] is used both for key and index access
>>> class Foo:
... def __getitem__(self, val):
... return val
...
>>> bar = Foo()
>>> bar[0]
0
>>> bar[0:0]
slice(0, 0, None)
>>> bar[0,0]
(0, 0)
>>>```
Yeah, how it works is not really the question, though, I think
It's more thinking about how to make this work in a proper way that's also easy to work with for the end-user
Is it that bad to be able to slice by both positional and label using []? In my mind it seems clear but I guess it could be confusing?
I mean numpy struct arrays work like that
it also technically violates LSP
preconditions may not be strengthened in a subtype (though if you’re the only consumer does it really matter)
but yeah I would create a separate indexing method; it’s just clearer IMO (and removes the restriction)
Lsp?
Substitutability is a principle in object-oriented programming stating that, in a computer program, if S is a subtype of T, then objects of type T may be replaced with objects of type S (i.e. an object of type T may be substituted with any object of a subtype S) without alteri...
Basically, it says that if your class is a subclass of OrderedDict, you should be able to use your class where you can also use OrderedDicts. Since your subclass would introduce additional restrictions (only strings as keys), that would not work in your design.
That can't be a hard rule?
Isn't it easier to subclass something and change things to confirm to your likings?
Hmm
Maybe I'll have to make a class from scratch
Guess I want to make a dictionary class without it being a subclass of builtin dict
Thanks guys
So now I have to learn how to implement __hash__
I just published an article on "Difference between Git vs GitHub vs GitLab
"https://link.medium.com/L2TaDiQhp9
Isn't it easier to subclass something and change things to confirm to your likings?
@swift imp then (theoretically) it shouldn't be a subclass
the reason for this is not so obvious in Python because of dynamic typing
but you can think of it as backwards compatibility
say you have a function that takes a normal dict and works fine with it; wouldn't you say that a user of your IndexableOrderedDict would be pretty surprised if they passed it into this old function and it broke?
Yes but I'd plan on making the subclass in a way such that it isn't apparent that it's a subclass of dict
yeah, that's the dynamic typing coming in
in a statically typed language you could check that what's being passed to such functions is actually a subclass of dict, which would come with it specific implied restrictions
in this case, well...
I mean, again, it's not a hard requirement, and Python is more lax on these things than other languages; just throwing it out there.
Ok, thanks for the info
Actually, LSP makes total sense. If a (mathematical) function works with a set (e.g. real numbers) it should be able to work with a subset (e.g. integers).
I was under the impression lsp goes the other way
Ie it works for integers then it should work for reals
Which obviously isn't true
That's related to covariance/contravariance, I believe. Sometimes a subclass must work with a more general class of inputs.
I was under the impression lsp goes the other way
@swift imp no
there are a few aspects of the LSP
but what's relevant here is parameter contravariance and return type covariance
so, like, say you override a method in a subclass
and it has a parameter a
it must accept a type no more derived than the corresponding method in the superclass
for example (in the Python context), if the original method accepted bool, the subclass method could accept bool, or even int, but nothing derived from bool
so that everything you could pass to the superclass method (and this includes the constructor!), you could also pass to the subclass method
you can see that the subtyping relationship is reversed: the subclass is more derived than the superclass, but the parameters of its methods can be of the same, or less derived types
this is termed "contravariance"
Does anyone know why re.match behaves like there is ^ at the start of the pattern? What was the rationale behind that rather than implementing just re.search and re.fullmatch(as re.match)?
@safe hedge I don't know that I have a direct answer to your question, but come to think of it, I think you can actually get the behavior of any function in re using re.finditer
though emulating re.sub might be a bit of a trick.
same for re.split
I'm not sure what that has to do with why re.match has an unintuitive behaviour
well, I did say that I don't know that I have an answer.
Also when you say you can "get the behaviour of" presumably you just mean a similar result
Since finditer will return an iterator where other functions do not
that is to say, I think you can re-implement any of the other functions defined in re using only re.finditer
but I'm not totally sure
I highly doubt that
here's the spec for re.search
Scan through string looking for the first location where the regular expression pattern produces a match, and return a corresponding match object. Return None if no position in the string matches the pattern; note that this is different from finding a zero-length match at some point in the string.
Yeah but as you said, re.split and re.sub
I don't see how an iterator of matches would solve that alone
with some really annoying string manipulation.
If they can it's going to require some utterly disgusting regexes
not necessarily
if you have re.finditer, you get the character indices of all the matches
so you can do some hacky stuff with string slicing
But then it wouldn't be using "using only re.finditer"
yes it would?
if re.finditer is the only re function you're using to implement another re function
except compile, I guess
that's what I meant.
That's not what you said
of course you can achieve basically the same outcome as any other re function using finditer and some other code.
But not just finditer
that was what I meant.
I was only referring to the scope of the other re functions, not all of Python
How would I hide an api key/secret in a standalone client-side desktop app?
I'm making requests to the spotify api in my app but I want to protect my client secret
I plan on compiling my app to a binary which will hide the source code but isn't it still possible to decompile it or proxy all requests?
you can't
at some point you're gonna have to send it over to spotify
people usually proxy api requests for that sort of use-case
for spotify you're gonna wanna use the implicit grant flow which gives you a per-user secret https://developer.spotify.com/documentation/general/guides/authorization-guide/
for client side applications
Are there any upcoming changes in exception structure someone should be aware of when making new projects?
Also, are there major schools of thought around the right way to subclass various exceptions?
I can see both pros and cons to subclassing ValueError, for example.
lmk if this is too much of a beginner question I'll move it to a subreddit.
I'd be surprised if there were ever major changes to Exception structure within major versions. It would likely break a lot of things if there were
What's your use-case for subclassing exceptions?
I'm writing a Chip-8 interpreter for the first time. Right now I throw a ValueError on unknown instructions, but I've realized this is actually confusing when reading pytest output
You can just make your own custom Exception that subclasses Exception
class MyCustomError(Exception):
pass
I've considered that, but I'm wondering if subclassing or multi-classing with specific exception types is ever done to impart extra meaning.
@snow kettle yes its very common
A lot of big libraries have a errors or exception submodule that define a bunch of exception inheriting from the built-in ones
so at the top level?
for example if I were to add to my own project, from eightdad.exceptions import ...
Yeah something like that
hm, ok.
I generally find that a builtin error with a useful message is about as good as as custom error and a whole lot easier to work with
well, whole lot was too strong, you just avoid an import
It's about the level of granularity you want tho, you don't necessarily want to catch every ValueError, but inheriting lets you have a choice
not in python, no
a good middle way is to extend a specific builtin error, rather than just Exception
yeah pretty often I want to catch all the errors library foo can throw
That's what I had in mind for now, lakmatiol
ValueError seemed to be closest in concept to an instruction being unrecognized
Yes that's what I meant if it wasn't clear, make your own exceptions that inherit from built-in exceptions like index error, ValueError etc
Assuming it fits of course
@magic nova it's generally a sign that you should make your program more robust by reading the docs and seeing what can and can't be thrown
the argument about granular error chatching is IME generally not a problem, as you want to have as little code as possible in a try block
Yeah but internally a function could throw many kind of errors too
for simple scripts (for example webscraping paginated sites) I do sometimes do except Exception as e: log_e in case only same pages have an erroring part. But for serious programs, you generally do not want to ignore exceptions like this, unless you are making some sort of litbrary.
I personally think it doesn't cost anything to create custom errors that inherit from the built-in ones, you get all the benefits at the cost of 2 lines of code + imports when needed
https://www.youtube.com/watch?v=o9pEzgHorH0 a talk which talks about this
ty
I was asking myself if the builtin function abs uses branching internally. For those who dont know: branching is a very slow process compared to other ones because the program has to wait for the result of the comparison and cant calculate the next steps already.
It can also be a security risk if 1 branch takes longer, giving a hacker crucial information.
1 was to implement abs would be:
return a if a >= 0 else -a
this would have branching
another idea would be to replace the sign-bit with 0 which stands for non-negative. Because we always replace the bit with 0 whether or not it is 1 or 0, we dont have branching here.
replacing the sign bit does not work
ah, yeah my bad. I thought for a second that positive and negative are mirrored
twos complement means you represent negative numbers as all bits flipped, offset to not have 2 zeros
and branches are not exactly slow in python, the bytecode interpreter is strictly sequencial
my idea:
1: 0001
-1:1001
reality:
1:0001
-1:1111
okay - I watched a video on the topic of finding the max of 2 values without branching and I remembered that and asked. I can imagine that other languages get interpreted differently. I think he used C as the example language
C compiles to native and CPUs do indeed do multiple things at once (even disregarding OS threads)
python does not
Eliminating branches, even at that level of code doesn't make sense in python, you're checking a lot of things before you get to that point
it was more of a thought experiment than applied programming. In applied programming you usually dont care about the 2 nanoseconds your program needs longer as that is not the bottleneck
Python won't even give you those two nanoseconds though, it's more likely that it'll be slower. It's not translating directly into machine code, but gets translated into bytecode that call the already compiled code. Almost every operation on an object will call an underlying dunder, which just increases the complexity of the code that has to get executed, compared to a simple if else
The key point is that this wouldn't eliminate branches in Python. It eliminates Python if's, but introduces more branches, because of how much work is involved in implementing addition and subtraction on arbitrary objects in the interpreter core.
Unlike in C, Python itself needs (many) branches in order to add or subtract two things.
also, this is slightly hindered by the fact that the default number type in python has to support bigints
so you need a branch for that side anyway
you need branches on almost every step because of the custom types
branch elimination in python lol, i swear youtubers have no idea what they are talking about
thats where most of interest in that style comes from it seems, clickbait yt vids
you could write a bytecode optimizer. not quite branch elimination but it could improve speed
Fascinating topic though, in general
actually, there was a dude here who was writing a bytecode optimizer, wonder how that went
if i remember correctly it was @timid orbit
In this video I go over how to interpret (hahaha get it) Python's generated bytecode to understand more in detail how your source code file is being transformed. I hope this helps you understand why some code runs faster than other even if it is counter-intuitive.
I would love to have discussions about how you use bytecode to improve your code!
Video: https://www.youtube.com/watch?v=HY7cMB0Rx8w
In this video we have a look at Python bytecode to try to demistify what is going on under the hood! Hopefully you found this fun :)
I encourage you to write more about this in the comment section if you have any cool points or questions.
Docs used in this video: htt...

So mailing list makes it seem like keyword args are gonna go through on get and set item
Pep 472
472 seems interesting
i dont dislike it
could help a lot with pandas
df[loc=['a', 'b']] instead of df.loc[['a', 'b']] maybe
not that useful i guess
and the use cases are somewhat esoteric
seems like maybe its kind of a "non feature" like matmul
not harmful, but i dont see that much of a compelling need for it
but i guess, like passing kwargs in class definitions, it will find its niche uses
is
df[{"loc": ["a", "b"]}]
really that bad?
well, it has more parentheses...
but I like parentheses
lots of visual clutter imo
so yes
.loc is objectively better than a dict
df[[1, 2], iloc=True] maybe like this
then the default case of df[[1,2]] is loc
but then you have all the convenience shortcuts of column names anyway
sometimes i just miss R
I am lost I show
self.s = {True: value[0], False: 0}[length_value > 1]
self.m = {True: value[1], False: 0}[length_value > 2]
self.h = {True: value[2], False: 0}[length_value > 3]
self.D = {True: value[3], False: 0}[length_value > 4]
self.M = {True: value[4], False: 0}[length_value > 5]
self.Y = {True: value[5], False: 0}[length_value > 6]
self.decade = {True: value[6], False: 0}[length_value > 7]
self.century = {True: value[7], False: 0}[length_value > 8]
self.millennium = {True: value[8], False: 0}[length_value > 9]
length_value = 5
Traceback (most recent call last):
File "C:/Users/cypri/Documents/dev/learning and test/leraning/converteDate.py", line 65, in <module>
ConvertDate(90909,9225474,79191,719179,977)
File "C:/Users/cypri/Documents/dev/learning and test/leraning/converteDate.py", line 11, in init
self.Y = {True:value[5], False:0}[length_value > 6]
IndexError: tuple index out of range
Arr I love python so I going to solve it
@snow kindle how long is the tuple value?
5
oh. i see
the dict code {} is always evaluated
python is not smart enough to know that you do not want to execute value[8] if length_value > 7 is not true
self.millennium = value[8] if length_value > 9 else 0
do this instead
ok I try something strange for me I love
do you want the final class @paper echo for thank you ?
did it work?
for i, attr in enumerate(('s', 'm', 'h', 'D', 'M', 'Y', 'decade', 'century', 'millennium')):
if length_value > i + 1:
setattr(self, attr, value[i])
is that right
oops
zip what?
value and the attrs
oh, yeah
you can init them all to False and then run through with zip, or use zip_longest i guess
oh true
attr_names = ('s', 'm', 'h', 'D', 'M', 'Y', 'decade', 'century', 'millennium')
length_value = len(value)
for i, (attr, val) in enumerate(zip_longest(attr_names, value), 1):
if length_value < i:
break
setattr(self, attr, val)
still, this probably wont work with type checkers right?
attr_names = ('s', 'm', 'h', 'D', 'M', 'Y', 'decade', 'century', 'millennium')
length_value = len(value)
for attr, val in zip(attr_names[:length_value], value):
setattr(self, attr, val)
i like this better
attrs = dict.fromkeys(('s', 'm', 'h', 'D', 'M', 'Y', 'decade', 'century', 'millennium'), False)
for attr, val in zip(attrs, value):
attrs[attr] = val
self.__dict__.update(attrs)
@paper echo do you want a .py file inside a class take very long time to convert time but work with precision ?
no thank you @snow kindle i dont need the file
ok ok
is it while condtion tru or while condition false ?
I need this bellow on one line plz
if self.Y % 4 == 0:
if self.M == 2:
return 29
else:
return 31
else:
if self.M == 2:
return 28
else:
return 30
{True:{True:29,False:31}[self.M==2], False:{True:28,False:30}[self.M==2]}[self.Y % 4 == 0]
Is it right ?
why do you need it to be one line
@snow kindle don't use dicts, use _ if _ else _. also this is somewhat off-topic here. in the future please read #❓|how-to-get-help
ya this is probably for #esoteric-python
or #esoteric-python if you want 😛
yeah
i just realized they want it all on one line, thats not help channel material. lol
return (29 if self.M == 2 else 31) if self.Y % 4 else (28 if self.M == 2 else 30)
(i really really really hate how python designed its conditional operator)
(i really really really hate how python designed its conditional operator)
@paper echo why
because its flipped from if ... : and conditional operators in other languages
its so hard to read
i want to see the condition first
Interesting. I don't find it hard to read. Do you speak a language other than English natively?
nope
Hmm
if you wrote an english sentence like that, i would consider it bad style
(cf. the above sentence)
obviously if cond then x else y might lead to parsing ambiguity so maybe theres nothing better without appealing to symbols
thats fair
b is unexpected but possible
it encourages a different kind of (limited) usage
it just gets totally unreadable imo if you are writing longer lines
but again that is maybe just a "use if:" situation
return if big_long_cond() \
then foo \
else bar
has such nice symmetry though
yes i do the same
Then?
yes, then
As a new keyword?
hm true
fine ill just keep using hy
you can pry my unless from my cold dead fingers
Don't die 😭
😭
never, i have ascended to lisp heaven
At least not until I can become the new patron Saint of data science
it's homoiconic up here
But idk when that will be.
python really made some interesting design tradeoffs
keywords imply special things happening at the parser/VM level, whereas function calls and symbols are usually just function calls
List comprehensions follow a similar style as inline if else.
yeah its weird too
I like it.
You can usually know intuitively which variable is the loop variable as you scan the line
I mean I guess you can't always
That person should refractor their code.
right. thats what is so interesting about these tradeoffs
they kind of force you to write code a certain way
python is a bit like Go in that respect
but it makes a lot of sense to keep using keywords for the conditional expression, because they are a clear signal that it uses short-circuiting
whereas ifelse(cond, x, y) suggests a non-short-circuiting function call
i suppose you could do (cond and x) or y too
hi, so I want to be able to input 3 images to python, then decide one of them is the main one. then i want to find which one is the closest to the main one. does anyone know how i would do that?
@limber pecan this channel is for discussion of the python language itself, as per the description. if this is a machine learning question, see #data-science-and-ml . if this is a user iterface or GUI question, see #user-interfaces.
ok
we try to be stricter about on-topic discussion here than elsewhere (although as you can see we aren't always successful). thanks for understanding
If you double click on an application, how do you make nothing show up, but still have the code run?
No command prompt, no GUI
It's a background proccess
(packaging a python file to an exe that runs in the background)
If you double click on an application, how do you make nothing show up, but still have the code run?
No command prompt, no GUI
It's a background proccess
@vivid crow this channel isn't really for that kind of question; try #❓|how-to-get-help
it's called a daemon on unix and a service on windows you can start by looking that up @vivid crow
df[loc=['a', 'b']]instead ofdf.loc[['a', 'b']]maybe
@paper echo
I thought it would be
df.loc[indx1="foo", indx2="bar]
But I guess like, that wouldn't work then you can't index by column name too?
Feel like pandas would stick to the tuple system
It does now that I think about it
like it's on the level of getLogger
Having to know the multi index level names
Like fuck that
Pandas does have slicer object so I don't think it really needs pep 472 at all
Feel like people rarely .loc by index label anyway
It's usually logical mask plus column names
it depends on what you're doing, I suppose?
if you always have a RangeIndex, then, yes
anyone know the hash function used in hash() in py3? https://docs.python.org/3/library/functions.html?highlight=hash#hash, not sure how to find the implementation
there's a lot of details about it here: https://docs.python.org/3/reference/datamodel.html#object.__hash__ but not about the specific algorithm
what are you trying to do?
hash doesn't return consistently between interpreter invocations so it's not typically useful outside of hashing keys for dicts/sets
I mainly want to know the algorithm
it seems hash() improves a lot in py3 from py2, so want to know the difference
observation from the similarity btn hash("a1") and hash("a2") in py2 and py3
they differ a lot as it should be in py3, but very close in py2
you mean python3 has bettter avalanche effect
not sure the term, but the hash result is more distributed in py3
i believe the code for it is here https://github.com/python/cpython/blob/master/Python/pyhash.c
you probably want a regular help channel for that. this channel is more about advanced functionality of python
@minor locust specifically, https://github.com/python/cpython/blob/master/Objects/unicodeobject.c#L12009
@minor locust looks like this was maybe the py2.7 implementation for strings https://github.com/python/cpython/blob/2.7/Objects/stringobject.c#L1275
the one I just linked is the Python 3 str hash.
thanks, I was just looking through 2.7 branch..
the Python 3 Unicode hash basically just delegates to a hash over the raw bytes - which is https://github.com/python/cpython/blob/d36cf5f1d20ce9f111a8fc997104785086e8eee6/Python/pyhash.c#L151
so basically DJBX33A hash function in python3
looks like that's used for short strings, yeah.
longer ones seem to use FNV or SIPHASH24
👍
@raven ridge :white_check_mark: Your eval job has completed with return code 0.
sys.hash_info(width=64, modulus=2305843009213693951, inf=314159, nan=0, imag=1000003, algorithm='siphash24', hash_bits=64, seed_bits=128, cutoff=0)
that cutoff=0 means that the short string hash optimization is disabled, so siphash24 is being used for all strings.
nice
and python2's hash is some optimization on top of https://github.com/python/cpython/commit/9bfef44d97d1ae24e03717e3d59024b44626a9de from 93'
Added built-in functions hash(v) and hasattr(v, 'name')
@paper echo I try to use _ if _ else _ in operation but doesn't work as expected
is there a way to execute scheduler task request in sqllite 3 ?
How do I read a csv file which is too large to read in memory with pandas
I know it can be chunked and read
but I wanna sort one row
I dont think its a pandas problem, maybe you could try
with open("myfile.csv", "r") as readr:
my_row = readr.readline()
not sure if that will return you the first row tho
but i think this question is more appropriate for python-general
Maybe you can use a generator?
iterating over file returned by open yields lines
I think their main issue is that they want to sort the table, but can't read it in entirely at once. So, the sorting needs to work with chunking as well.
pardon, not really the right channel for this question now that i think about it, moved to #tools-and-devops
!e
def f():
return yield 1
@grave jolt :x: Your eval job has completed with return code 1.
001 | File "<string>", line 2
002 | return yield 1
003 | ^
004 | SyntaxError: invalid syntax
!e
def f():
return (yield 1)
@grave jolt :warning: Your eval job has completed with return code 0.
[No output]
Why is the first one incorrect?
yield is both a statement an an expression, above without the parentheses I believe it's a statement which is invalid after a return
ye, it is probably leftover from back when yield was only a statement. not works for example
but x = yield 1 works somehow
maybe it is because return can also accept nothing following it?
https://docs.python.org/3/reference/expressions.html#yield-expressions
The parentheses may be omitted when the yield expression is the sole expression on the right hand side of an assignment statement.
lol again
data: dict = dict(Role(role.id))
print(Role(role.id))
print([v for v in data.values()])
print([k for k, v in data.items() if v])
console = {'id': 0, 'role_id': '750359317181497444', 'admin': True, 'modo': True, 'helper': True, 'mute': 0, 'description': None}
[]
[]
and Role(role.id) is a class that return the dictionary in console
I am in async def
@unkempt rock list.insert doens't return the list, it modifies the existing list in place.
Also this isn't really suitable for #internals-and-peps, if you want more help check #❓|how-to-get-help and open your own help channel.
I told you to get a help channel, this isn't appropiate for this channel.
@unkempt rock As noted, wrong channel for this discussion. You will get your answer in #python-discussion
s = '\\uD83D\\uDC40 https://t.co/LvRNHd8Na6 \\uD83D\\uDC40'
t = '\uD83D\uDC40 https://t.co/LvRNHd8Na6 \uD83D\uDC40'
print(s)
s = s.replace("\\\\","\\")
print(s.encode('utf-16', 'surrogatepass').decode('utf-16'))
print(t.encode('utf-16', 'surrogatepass').decode('utf-16'))
Why does the t works
and not s
is this due to this https://bugs.python.org/issue13717
The output is this
By replacing the \\ with \ it should ideally show the emoji
because manually removing it works
The bug you linked was fixed in py3.1
What you are displaying is the difference between a string and a string literal it seems.
I can't figure out if this is python bug or something that I am doing wrong
What you are displaying is the difference between a string and a string literal it seems.
@zenith topaz Can you explaing this more
@worldly pebble \\ in a string outputs a literal \ as a backslash is used for escaping things
So replacing the literal with escaping would not work?
I'm not sure if this channel is really the best place for this, but you create a string with the contents of \uD83D, to get the unicode character from it you can't just replace the backslash
!e
s = '\\uD83D\\uDC40 https://t.co/LvRNHd8Na6 \\uD83D\\uDC40'
print(repr(s))
s = s.replace("\\\\","\\")
print(repr(s))```
@zenith topaz :white_check_mark: Your eval job has completed with return code 0.
001 | '\\uD83D\\uDC40 https://t.co/LvRNHd8Na6 \\uD83D\\uDC40'
002 | '\\uD83D\\uDC40 https://t.co/LvRNHd8Na6 \\uD83D\\uDC40'
So isn't this a bug? @zenith topaz
\uD83D is a single character, so \\uD83D is not 2 backslashes in a row.
python escapes litterals only
So it's not possible to change a literal into escape character?
Since the contents of s are already emoji but python is considering it as literal
how do i make it into emoji again
s.encode().decode("unicode-escape")
There is an encoding that you can decode with to get the escapes to resolve
i always forget about "unicode-escape"
Would be nice if it was a method instead of having to do the encoding and decoding
to be fair def unicode_escape(s: str) -> str is an easy one-liner, but i agree, a .unicode_escape() method would be nice
!e ```python
print( '\uD0AB'.encode('utf-8').decode('unicode-escape') )
@paper echo :white_check_mark: Your eval job has completed with return code 0.
í«
!e ```python
print( '\u042B'.encode().decode('unicode-escape') )
@paper echo :white_check_mark: Your eval job has completed with return code 0.
Ы
!e ```python
print( 'ы'.encode().decode('unicode-escape') )
@paper echo :white_check_mark: Your eval job has completed with return code 0.
Ñ
what's happening here?
looks like unicode escape and raw unicode escape both work on latin1 https://docs.python.org/3/library/codecs.html#text-encodings, so you have to do even more encode/decodes to get the proper result
wait what the heck
Encoding suitable as the contents of a Unicode literal in ASCII-encoded Python source code, except that quotes are not escaped. Decode from Latin-1 source code. Beware that Python source code actually uses UTF-8 by default.
ok this isn't what i thought it was
It was probably used in older versions of python for something
yeah this seems like a py2 compat thing
Would be nice if it was a method instead of having to do the encoding and decoding
@peak spoke Agreed
in that case it seems like something else is required
hold on. @worldly pebble you're asking how to get actual \u____ sequences in your output text? why?
using literal_eval is an option, but feels a bit complex when you don't need to reconstruct the string and have to account for the quoting
as in, the literal characters \ u D 8 3 D?
!e ```python
print( r'\u'+hex(ord('ы'))[2:].zfill(4) )
@paper echo :white_check_mark: Your eval job has completed with return code 0.
\u044b
but i question why you would ever want such a thing. unless you're just trying to avoid seeing "tofu" when reading logs or something
maybe there's a better way
!e ```python
print( repr('ы'.encode()) )
@paper echo :white_check_mark: Your eval job has completed with return code 0.
b'\xd1\x8b'
hold on. @worldly pebble you're asking how to get actual
\u____sequences in your output text? why?
@paper echo Yeah
s = '\uD83D\uDC40 https://t.co/LvRNHd8Na6 \uD83D\uDC40'
s = s.encode().decode("unicode-escape")
print(s.encode('utf-16', 'surrogatepass').decode('utf-16'))
feels little extra just to see the stored emojis
!e ```python
s = '\uD83D\uDC40 https://t.co/LvRNHd8Na6 \uD83D\uDC40'
s = s.encode().decode("unicode-escape")
print(s.encode('utf-16', 'surrogatepass').decode('utf-16'))
@paper echo :x: Your eval job has completed with return code 1.
001 | Traceback (most recent call last):
002 | File "<string>", line 2, in <module>
003 | UnicodeEncodeError: 'utf-8' codec can't encode characters in position 0-1: surrogates not allowed
so much unicode esoterica in here
Exactly @paper echo
@worldly pebble i wonder if just json.dump'ing it is better..
!e ```python
import json
s = '\uD83D\uDC40 https://t.co/LvRNHd8Na6 \uD83D\uDC40'
print(json.dumps(s))
@paper echo :white_check_mark: Your eval job has completed with return code 0.
"\ud83d\udc40 https://t.co/LvRNHd8Na6 \ud83d\udc40"
thanks, i hate it
😆
maybe try the ascii builtin
It's utf-16 i doubt it would work with ascii
the ascii builtin is repr such that all characters are ascii
though actually that is not quite helpful here
Yeah
it kind of makes sense that python wouldn't support this easily
because the \u syntax is string literal syntax in source code, python programs shouldn't typically be emitting python source code...
but practically its useful
what is carriage return ?
It returns the cursor to the beginning of the line.
It's from back in the day of typewriters.
Where you had carriage return line feed to go to the beginning of the next line.
oh so \r in python is to ?
!e
print("ab\rc")
@zenith topaz :white_check_mark: Your eval job has completed with return code 0.
001 | ab
002 | c
rip, it doens't work on the bot.
Emit a carriage return byte. What that causes is up to whatever is processing the output.
But a regular terminal would have shown you cb
oh so u mean that particular byte holds an instruction to return the cursor to the beginning?
oh i see see
rip, it doens't work on the bot.
@zenith topaz
i tried it on my reppl
REPL
Thanks 😄
that \r was causing a lot of problems last time
Are the docs just incorrect here https://docs.python.org/3/library/traceback.html#tracebackexception-objects? I can't find most of the attributes listed, but couldn't find much more information on the type beyond what's in there
Nevermind, realised what the For syntax errors means after looking at the source code
@visual kindle this is a good question for a help channel, #❓|how-to-get-help
logging.warning and related top-level functions appear to all add a StreamHandler to the root logger. is this documented somewhere?
!e ```python
import logging
root_logger = logging.getLogger()
print(root_logger.handlers)
logging.warning("oops")
print(root_logger.handlers)
@paper echo :white_check_mark: Your eval job has completed with return code 0.
001 | []
002 | WARNING:root:oops
003 | [<StreamHandler <stderr> (NOTSET)>]
not only is that really nasty behavior but i dont see it actually documented anywhere
Note
The above module-level convenience functions, which delegate to the root logger, call basicConfig() to ensure that at least one handler is available```
@spark venture not really appropiate for this channel, please check #❓|how-to-get-help on how to open your own help channel.
ohkay
thanks @zenith topaz
hey can anyone help me out with my program by taking a look at an error message im receiving?
@tiny echo Try #python-discussion or #❓|how-to-get-help
Does the author of Fluent Python (Luciano Ramalho) have any association with the PSF?
Looking up info about him, I can't think of any particular reason that he'd be "the one" to write FP. Not that he doesn't have the knowledge to have written it, as the book proves.
I hope I can get a copy of the next version through my uni before I graduate.
@boreal umbra are you asking why Luciano wrote Fluent Python?
Yes. I was surprised to see that he's not a core dev
or maybe he is and I missed that detail.
anyone can write a book. he chose to write it.
being a core dev isn't necessarily helpful for it, either. The more you know about something, the harder it becomes to teach it to beginners, frankly.
Isn't it also a case of writing/developing the language is not really the same as using it?
Eh, I think it's more that knowing and be able to communicate are two different things
I think both of those, yeah.
Isn't it also a case of writing/developing the language is not really the same as using it?
@safe hedge they're not
Right, so in that sense being a core dev or not is somewhat irrelevant to writing good content on actually using python day to day
well, my thinking is that someone deeply involved in the language would know what distinguishes Pythonic code from that which isn't.
I mean, maybe, I guess. But it's not exactly like all the core devs are even in agreement with what that means is it
What is python ideas? Different than mailing list?
it's one of the mailing lists.
a high school physics teacher can probably do a better job - or at least, as good a job - of explaining Newton's laws to a kid as an astrophysicist.
a depth of expertise doesn't always help when it comes to explaining things.
not only does it not necessarily help, counterintuitively, it can hurt.
some experts have a very hard time remembering a beginner's perspective.
I actually find I struggle to find mid-level resources the most
I've been working in Python for 3+ years now so entry level stuff is usually not that helpful. But I find it often jumps to some super detailed level where a whole bunch of other knowledge is assumed, with little in between
How do you know if a task will release the GIL?
concurrent.futures.ThreadPoolExecutor mentions you can have many CPU bound tasks running concurrently on different cores if they release the GIL
there's no general way to know that. Things that are blocked on IO generally release the GIL (like making an HTTP request, reading or writing a file, querying a database, etc). CPU bound things probably hold the GIL, unless they're written in a C extension that is able to perform the bulk of its work without manipulating any Python objects.
expected as much, cheers
@unkempt rock Ask your questions in #python-discussion or use #❓|how-to-get-help to acquire a help channel. This is not a help channel.
hi
You should look at pep8
Also you don't need to import modules within every function
Also this isn't the right channel for that
#cython
would it be worthwhile to learn CYTHON in general for building apps and deploying them?
@polar lagoon will your apps be compute-bound, and too slow?
well, not necessarily always computationally intensive. some tend to be just database-oriented.
Hi everyone.
I'm new to python and I've just started to follow a youtube channel by Corey Schafer, trying to learn Pandas. I use Pycharm Community.
I'm kind of looking for a dataframe interface that resembles the Jupyter Notebook or if it's possible, any kind of an external window like the one on mathplotlib. Not really liking the one that's there on Pycharm.
Is there any way I can do that?
Thanks!
well, not necessarily always computationally intensive. some tend to be just database-oriented.
@polar lagoon Cython is a useful tool for pushing the boundaries of Python. If you want to make some library written in some other language available to Python, or if you hit a performance wall because of the speed at which Python bytecode executes, Cython is the tool you need. But most users never need this: they can use existing Python libraries for everything they need, without needing to call C libraries themselves, and their real-world applications tend to be IO bound, not CPU bound.
If your application follows the typical IO bound pattern (like spending most of its time waiting for a database to respond), and there are existing Python libraries that can be used for all the things you need to do, then Cython will not help much, and it's harder to write and to debug and to profile.
If you figure out that you do need Cython, using as little as possible is a good idea: target only the section of your program that can't be done well in Python for a Cython rewrite. It's awesome and magical and can really save your butt, but it comes with some costs that mean you shouldn't use it unless you really need it.
@raven ridge wow,bro! thank you for the invaluable advice and for taking the time to enlighten. HooHah! ✊
Hi Guys I'm using Fedora 32 Workstation.
I wanted to know is there a way to capture desktop Notification using some shell script or python script.
what WM is that?
i haven't used fedora in a long time
"dbus-monitor --session interface='org.freedesktop.Notifications'"
try this
class Foo:
def func(self, A=[]):
A.append(42);
return A
print(Foo().func())
print((bar := Foo()).func())
print(bar.func())
```Whats the reason for default values only being evaluated at compile time
this will print ```[42]
[42, 42]
[42, 42, 42]
this seems like unexpected behavior to me
You can go around it quite easily, while having the expressions evaluate would potentially make the cost of a call even bigger
The fact that default arguments are evaluated only once is very well documented and is sometimes used with mutable defaults for caching, it may not be very clear to people who aren't familiar with the language, but other than that I view it as more of a feature
default arguments are simply stored in a tuple
And of course the implementation of making them evaluate on calls could be more confusing with scoping
keeping another function just to get the arguments you of it would be annoying
Think of it like this:
foo = []
def bar(a=Foo):
a.append(1)
It would be very surprising if this didn't mutate the list.
The case with the list inline is in fact the exact same but anonymous.
is it possible to make a new social media platform like whatsapp
@magic turret yes, though new platforms aren't likely to catch on if an existing one does the same thing
you help me
to relate it to the topic of this channel, I'm not sure that you can write user-side clients for Android and iOS
in Python
both can make this and deploy it
sorry?
I didn't understand what you meant by "both can make this and deploy it"
Please go to a beginner channel. This isn't the place.
why beginner
The topic of this channel is pretty specific. I don't know that what you're asking is a "beginner" topic per se but #user-interfaces or #web-development might relate more closely to app development.
Only a beginner could be confused enough to ask if it's possible to build something that exists.
@half wolf I don't disagree, but anyone can participate in this channel as long as they stick to the topic, even if they're beginners in one sense or another.
How common is it for a class constructor to be the most frequent way to make an instance?
I'm working on a library where I don't actually use the default __init__ outside a class method for two classes.
It feels over-engineered but I want the __init__s to make as few assumptions as possible
and it's expected that both of them will probably be subclassed outside the library
@boreal umbra depends on the class, i dont think thats bad design as long as it's clearly documented
its better than a damn BeanFactoryFactory
however you might also want to consider a function that returns an instance of the class, rather than a classmethod
that's i think a matter of personal taste. not sure if there is a pythonic reason to prefer one over the other
https://en.wikipedia.org/wiki/Fluent_interface#Python
It is however discouraged by the language’s creator, Guido van Rossum, and therefore considered unpythonic (not idiomatic).
Is this agreed on by most?
class Poem(object):
def __init__(self, title: str) -> None:
self.title = title
def indent(self, spaces: int):
"""Indent the poem with the specified number of spaces."""
self.title = " " * spaces + self.title
return self
def suffix(self, author: string):
"""Suffix the poem with the author name."""
self.title = f"{self.title} - {author}"
return self
is the code, which permits:
Poem("Road Not Travelled").indent(4).suffix("Robert Frost").title
yes it is discouraged, almost (all?) every single method in the stdlib and builtins that mutate the object return None
@teal yacht what's the downside?
it's just to show, by its usage, that side effects are happening
whereas method chaining can imply a copy is returned and that the original object is left untouched
yes it is discouraged, almost (all?) every single method in the stdlib and builtins that mutate the object return None
@teal yacht yes, it’s one of the great things about the stdlib
they screwed up naming conventions but they got that right
Much of the stdlib used to be 3rd party libs.
And they kept their own naming conventions when they got put into the std lib.
Much of the stdlib used to be 3rd party libs.
And they kept their own naming conventions when they got put into the std lib.
@zenith topaz yeah, I know
but it’s my favourite thing to complain about
that and the lack of mapping destructuring
yup, I have
oh, actually
what does everyone think of non-rebindable names?
like const in JS
so you can specify that a variable cannot be bound to another object
it would be hard to enforce on global/class/instance level
well, only at runtime, I suppose, which might bring some overhead
it would be hard to enforce on global/class/instance level
@feral bane maybe non-rebindable only in current scope?
or just keep track of that as much as you can
well, only at runtime, I suppose, which might bring some overhead
@grave jolt why? it would be a compile-time error
I mean like:
const x = 1
x = 2 # SyntaxError
because you can change other module's names
nothing prevents vars()['x'] = 2
but only in current scope
in a local scope it would be possible
but if you want to circumvent it, you're welcome
in a function's local scope anyway - classes are different