or you could check the library page
#internals-and-peps
safe hedge
How terrible of an idea is it to have parallel modules with the same name?
raven ridge
What do you mean by "parallel"
safe hedge
So like I have:
-pkg
-subpkg
-moda
-subpkg2
-moda
Also if your top level package has an __init__.py do those subpackages then also need them or can you rely on implicit namespaces? I ask because I seem to be getting ModuleNotFoundError: No module named <module> for modules in folders without an __init__.py despite being on 3.8
main yarrow
Hello
Is it possible to run some processes on the GPU rather than CPU?
raven ridge
@safe hedge having two submodules with the same name at the same depth under different packages is fine. No one will get confused between
chef.appetizer.recipes and chef.entree.recipes, and as long as the names don't confuse readers I'd say you're fine.
As for the other question, what you're making is a regular package, not a namespace package. You need an __init__.py
safe hedge
Ok thanks. Basically I have a “functionality/core” package and then a “commands” package with submodules reflected
As for the other question, what you’re making is a regular package, not a namespace package. You need an
__init__.py
@raven ridge so yeah namespace packaging is all or nothing?
raven ridge
Yes. It has to be, just by the very nature of what namespace packages are.
You can't put a namespace package inside a regular package, because namespace packages don't have just one location on disk, and regular package do.
If you could put a namespace package inside a regular package, you would no longer be able to use it as a namespace package
PEP 420 made it so that if you were going to make an __init__.py that just contains
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
``` you no longer need to create that file at all.If you weren't going to put that into your __init__.py, then PEP 420 didn't affect you. If your __init__.py would have been empty in a pre PEP 420 world, you still need an empty one in a post PEP 420 world.
unkempt rock
420
raven ridge
Implicit namespace packages make it easier to do the wrong thing than the right thing, unfortunately. Namespace packages are an advanced feature of the language designed for allowing a single Python package to be composed of many individual PyPI packages installed into different directories. This isn't a thing average developers need to do. But now that it's harder to create a regular package than a namespace package (creating an empty __init__.py versus not creating an __init__.py at all), regular developers are creating the wrong type of package all the time.
safe hedge
Yeah I didn't really want a namespace package I thought maybe I just could take advantage of it in some of my nested subpackages which would have empty inits
unkempt rock
If anyone is good with datetime module come to #help-broccoli
grave jolt
@unkempt rock This is not a help channel, read the channel description. Don't advertise your help channel anywhere, please. If someone wants to help, they will help.
paper echo
@safe hedge I think it's fine to have empty init files
@raven ridge I wish they would have just created a new dunder file instead
Or some other explicit way to declare that a package is a namespace package, but without manually hacking around with the path
__init_namespace__.py
Idk
I get that the problem is, you need to prevent users from putting actual python code in that file
Heck, just call it __init_namespace__ and require that it be empty (or ignore its contents) - like py.typed
Or call it py.namespace analogously to py.typed
raven ridge
Or some other explicit way to declare that a package is a namespace package, but without manually hacking around with the path
@paper echo The problem with the old status quo was that some PyPI package needed to create the__init__.py, which meant that in practice when multiple packages all wanted to install into the same namespace, all of them would create an identical__init__.py, but then some packages are overwriting files created by other packages, and if you package those PyPI packages up as Debian .deb packages or RedHat or Arch packages or whatever, the__init__.pythat made the namespace package would conflict across all of those packages.
in other words, any solution that involves creating a new file wouldn't have solved the problem.
paper echo
Yeah i get the need for namespace packages & the desire to make them implicit
I imagine that a tool like setuptools could reconcile all that stuff
So the deb version would in fact have an empty file
Then again having a file whose existence is a sufficient check also solves the problem, no?
raven ridge
no, it doesn't. because if two different .deb packages both contain site-packages/myns/__init_namespace__, then Debian won't let you install both of those packages, because they conflict with each other: both want to own the same file.
raven ridge
yes.
in practice the solution to this was making a package that provides only site-packages/myns/__init__.py and doing the extend_path magic in there, and making every myns-foo and myns-bar package declare a dependency on the myns package, which needs to be independently published.
which worked, but was terribly inelegant, so implicit namespace packages are a solution for the problem that doesn't require creating artificial packages just to make an explicit namespace that other packages can depend on.
unkempt rock
Do I have to setup a special configuration to read all lines from asyncio.subprocess (create_subprocess_exec)
because as soon as i try to read all lines from process.stdout.readline() with a while True
I end up in an infinite loop
For no reason... I guess
unkempt rock
finding someone to help out with this topic is quite hard, im now trying to use a third-party lib that makes the same, anyio
@sacred yew but this is the code I was having problems with https://dpaste.org/odK1
if you're able to figure out whats wrong i can try again
gleaming rover
would it really be bad if frozensets were created with f{}?
cloud crypt
well, {} is not a set, to begin with
gleaming rover
like f{1, 2, 3}
I know {} isn't a set
empty frozensets can still be created with frozenset()
raven ridge
My take is that frozenset isn't a common enough type to warrant new syntax just for it
raven ridge
Because code is meant to be read, not written.
And reading it is already harder than writing it. So there's a cost to adding new syntax
Also, when you hit a function you don't know in some code that you're reading, you can Google it. That's much harder to do with syntax.
swift imp
Why not frozendict
gleaming rover
Why not
frozendict
@swift imp rejected a while ago
there was once or twice I actually needed it (mappingproxy is not a perfect substitute)
steady remnant
Hello everyone, I’m sorry if this isn’t the correct channel to post in, but I’m an intermediate python programmer, and I’m stuck for project ideas. Can anyone help?
mint forge
it is not the channel, but there is a link for project ideas, ask in #community-meta for that link.
steady remnant
Thanks!
ionic fulcrum
Can someone help me out for an app development for my university project?
languid dagger
!rule 5
fallen slateBOT
languid dagger
We will not help with graded assessment
blissful hornet
hey guys i am trying to make a software -in python- that when you double click on desktop it opens the FILE EXOLORER, any one have an idea about how to do this??
unkempt rock
hey guys i am trying to make a software -in python- that when you double click on desktop it opens the FILE EXOLORER, any one have an idea about how to do this??
@blissful hornet subprocess.Popen('explorer "C:\path\of\folder"')
Like that?
blissful hornet
yep
but am asking about how to double click on the desktop to open the explorer
i looked a lot for that
unkempt rock
So you're asking how to detect double clicks?
blissful hornet
yeah, on DESKTOPS ONLY
unkempt rock
okay, sorry, someonelse might answer that
blissful hornet
k this it the problem
loud summit
This is not a help channel @ancient lintel
See #❓｜how-to-get-help if you need help
ancient lintel
oh my bad
wintry radish
@tropic gale i dont think u can compile python to an exe for example
because some features wouldnt work if it was not intrepreted
like eval() or exec()
maybe try https://pypi.org/project/auto-py-to-exe/ ?
wintry radish
np
just be careful some stuff wont work @tropic gale
pure python without import might be fine? i havent tried it myself
tropic gale
It needs imports to run. But I'll see what I can do though.
wintry radish
maybe try this @tropic gale
brave badger
Anyhow, not really on-topic for this channel. I could suggest getting a help channel though.
tropic gale
@wintry radish okay. 👍🏿
rare ocean
Hello. So the scenario is that I’m trying to link a python script as backend to a front end. How do I make a front end with very decent UI. Like what language and how do I link it with python
Sorry if it’s not advanced
Please don’t suggest tkinter please no don’t
unkempt rock
@rare ocean there are many options but the simplest will be that you wrap your script in API and create a front end for it.
You can use flask(simpler) for it or django (more complex).
There are also some simpler alternatives if you don't have much experience with flask or django like framework. Like streamlit.
You can check it out and is nice if your script is for personal use or just for some presentation purpose.
quick snow
Or you could use a non-web UI that's not tkinter, such as PyQT
vast junco
Not sure if I am asking in the right channel. But, I want to use a namedtuple to cleanly combine a bunch of filter parameters passing into and then used inside a class. Example of the parameters:
minEfficiency = 1.25
minPercentile = 3
maxPercentile = 50
minDays = 3
maxDays = 5000
minStrike = 0.25
maxStrike = 3
I like namedtuples because they are immutable and their indexing is succinct compared to dicts. For example, I much prefer filter.minDays over filter['minDays']
However, I am struggling with how to overcome two things in my implementation.
- I will be assigning the namedtuple outside of a class and then passing it in as a parameter, but then using it a bunch inside of that class. I feel like the namedtuple's structure should be defined inside of the class. What is the best way to do that?
- Related to #1. Is there a way for the class to define the structure of the named tuple w/o necessitating the use of
selfeverytime I want to access my tuple data inside of the class, for example, Badself.filter.minDays, goodfilter.minDays?
quick snow
Re 1: Why do you think the namedtuple should be defined inside the class? Re 2: Not really.
vast junco
I am thinking what I want to to create a simple class that inherits NamedTuple... I think that gives me what I want... never done that before.
quick snow
Yes, that's quite a common pattern. You can inherit from a specific namedtuple even; class MyTuple(namedtuple("Foo", ["x", "y"])): ...
vast junco
Re 1: Why do you think the namedtuple should be defined inside the class? Re 2: Not really.
I know I said it should be defined inside of the class, but I guess it doesn't have to be. It could be defined outside of the class but in the same package.
quick snow
Yes, that seems more reasonable. Think of a namedtuple like a class.
vast junco
So, then, I don't get this...
class MyTuple(namedtuple("Foo", ["x", "y"])):
# what goes here? Is it just empty?
Not being a CS guy... I am having a Inception level confusion about how this inheritance works.
brave badger
The namedtuple function creates a class object for you, very much like type
Although I think typing.NamedTuple is better suited for that purpose
vast junco
I think you are right @brave badger, as you typed that response, I was reading this...
http://zecong.hu/2019/08/10/inheritance-for-namedtuples/
which provided this example, which seems perfect...
from typing import NamedTuple
class Point(NamedTuple):
x: int
y: int = 0
p = Point(1)
print(p.x, p.y) # 1 0
The relationship between collections, collections.abc and typing always perplexes me
vast junco
What is the point of naming a named tuple? To clarify...
Foo = namedtuple("Foo", ["x", "y"])
The second "Foo" seems redundant.
peak spoke
It's a class factory, and the class it creates needs to have a name
It's useful in in the string representations for example
vast junco
It's useful in in the string representations for example
@peak spoke
Do you have a link I could read that shows an example of where this is useful? Or something you would suggest I read.
cloud crypt
I’d rather use attrs/dataclasses instead of namedtuples
magic python
i was wondering about obvious attrs usage the other day -if you had several functions with similar parameters which had similar meanings, would you use an object for those instead?
def f(x, y, z):
....
def g(x, y, z, t):
...
replaced with
def f(location):
...
def g(location, time):
...
I’d rather use attrs/dataclasses instead of namedtuples
@cloud crypt
I'll have to check that out, if you have a link I can check out to read about them, please share.
cloud crypt
https://docs.python.org/3/library/dataclasses.html — standard library since python 3.7
https://www.attrs.org/en/stable — pretty much what inspired data classes in the first place
vast junco
Reading about dataclasses, it seems like I should use them instead of from typing import NamedTuple if I want mutable data.
@cloud crypt is that a valid conclusion on my part?
And, thankyou for the links
cloud crypt
it depends, really, but for classes based on manipulation with attributes, I think yeah, that’s a way to go
manic hatch
Does anyone know beautiful soup?
magic python
@manic hatch this not a help channel - and just ask a direct question rather than asking around the question #❓｜how-to-get-help
manic hatch
#help-pineapple is the help channel
vast junco
It's a class factory, and the class it creates needs to have a name
@peak spoke
Aha!!! I get it now. I have to name the tuple simply because (as you said) namedtuple is a class factory and is actually creating a class behind the scenes. So, Foo is the name of the instance and "Foo" is the name of the class. I guess, the authors of namedtuple could have made the name optional and given it a a GUID name or something guaranteed to be unique, but they didn't, or let you pass in Foo as a parameter and assigned it inline. But, that would have taken the understandable concept of a class factory and made things even more confusing.
I guess that is one of the reasons they created the typing library.
I am perplexed by why there are so many different ways to do these types of things...
typing.namedtuples, collections.namedtuples, dataclass...
And then dataclass uses decorators, collections.namedtuples uses a class factory, and typing.namedtuples doesn't use decorators but feels very similar to dataclass.
Seems like dataclass and typing.namedtuples should be more similar in how they are used in practice.
peak spoke
the NamedTuple from typing is just an another (nicer) way to construct a collections.namedtuple. Dataclasses offer a lot more compared to the namedtuples by not being a tuple at the base (mutable, field names matter etc.) the pep for dataclasses is quite extensive https://www.python.org/dev/peps/pep-0557/ and may be worth a read
swift imp
I can tell you right now dataclasses >> namedtuples and NamedTuple. dataclasses are soo freaking nice. Reduces so much boiler plate while simulatenously being really dynamic. You can do class factories with dataclasses.make_dataclass too. You can even make them hashable by doing frozen=True, eq=True in constructor
peak spoke
Sometimes you just need named fields, and namedtuples are perfect for that
spice pecan
namedtuples are exactly that, just fancier tuples, while a dataclass is a boilerplate class
So you'd use namedtuples when you'd do fine with a tuple, but attribute access would be nice
swift imp
true, but you can effectively make a namedtuple with a dataclass by doing dataclass(eq=True,frozen=True) and you get the added benefit of being able to add extra metadata and stuff
I mean I guess if you arent doing anything extra, just use a namedtuple but imo being able to add slightly extra metadata is worth it
spice pecan
namedtuples still have things like index access and iteration, and IIRC their overhead is less significant than that of a dataclass, so, again - if all you need is a tuple, but with added attribute access, namedtuple would be the right tool for the job
swift imp
True
but for example anyway, no iter, but thats ez
from dataclasses import dataclass, fields, make_dataclass
dc_namedtuple = make_dataclass("dc_namedtuple",[('x',int),('y',int),('z',int)],namespace={'get_fields': lambda self: fields(self), '__getitem__': lambda self, item: getattr(self, self.get_fields()[item].name)})
ex = dc_namedtuple(1,2,3)
ex[0]
ex.x
They're actual tuples and interchangeable with them, so you can pass them around to modules expecting tuples, in the (improbable) case that it matter them not being direct python objects helps with the overhead etc. Right tool for the job and all that
swift imp
Thats a good point, thats what they were used for originaly right? interfacing? namedtuples that is
peak spoke
Their main purpose is to bring the named fields to be self documenting instead of the indices which mean nothing without additional information
swift imp
here's an interesting piece typing.NamedTuple is actually a dataclass https://medium.com/@jacktator/dataclass-vs-namedtuple-vs-object-for-performance-optimization-in-python-691e234253b9#:~:text=NamedTuple is also DataClass%2C but,faster at only creating object.
vast junco
Thanks for all of the discussion, this has been very enlightening for me!
dry notch
Does anybody use task/project man. tools?
peak spoke
The attr access though namedtuples has to be delegated to tuple's getitem so it can't be faster there
vast junco
What if I wanted to be able to have nested attributes, is that something that is easy to pull off? For example something like this (had to make up some of the syntax because it is not apparent to me the right way to do this)...
from typing import NamedTuple
from dataclasses import dataclass
from collections import namedtuple
# Something like this
class MyTuple(NamedTuple):
x: int = 2
y: namedtuple('nestedtuple', 'a b c') = (5, 6, 7)
# Or something like this
@dataClass
class MyDataclass():
x: int = 2
y: namedtuple('nestedtuple', 'a b c') = (5, 6, 7)
myTuple = MyTuple()
myDataClass = MyDataClass()
print(myTuple.y.b) # 6
print(myDataClass.y.c) # 7
I only used namedtuple inside MyDataClass(): to get the idea across, I don't actually want it to be a tuple in that case.
peak spoke
Not sure if dataclasses have anything nicer as I haven't worked with them extensively, but you can simply create two namedtuples and then assign them inside each other etc.
vast junco
That makes sense. I guess the same would probably be true for dataclasses... Although, not having to define the nested class as a separate class would be nicer, or rather, doing it inline would be nicer and more succinct.
unkempt rock
#help-popcorn Its pretty advanced, just had to share pictures with explanation to make a understanding, aka reason why I am linking the help room here
peak spoke
The size stats from the article also don't seem right to me, tuples don't have to use dicts etc. so I assume it's because they used getsizeof which includes the fields for tuples but only counts the actual object with python classes (that don't define it)
vast junco
Just playing around with nested NamedTuples, doing something wrong:
from typing import NamedTuple
class Fees(NamedTuple):
flat: float = 0.0
percent: float = 0.1
contract: float = 0.2
class Filters(NamedTuple):
minStrike: float = 0.25
maxStrike: float = 3
fees: Fees() = (1.0,2.0,3.0)
filters = Filters()
print(filters.fees.flat)
Output:
Traceback (most recent call last):
File "c:/Users/HomeLaptop/Documents/FarmTrader/test.py", line 9, in <module>
class Filters(NamedTuple):
File "C:\Users\HomeLaptop\AppData\Local\Programs\Python\Python38\lib\typing.py", line 1616, in __new__
nm_tpl = _make_nmtuple(typename, types.items())
File "C:\Users\HomeLaptop\AppData\Local\Programs\Python\Python38\lib\typing.py", line 1590, in _make_nmtuple
types = [(n, _type_check(t, msg)) for n, t in types]
File "C:\Users\HomeLaptop\AppData\Local\Programs\Python\Python38\lib\typing.py", line 1590, in <listcomp>
types = [(n, _type_check(t, msg)) for n, t in types]
File "C:\Users\HomeLaptop\AppData\Local\Programs\Python\Python38\lib\typing.py", line 149, in _type_check
raise TypeError(f"{msg} Got {arg!r:.100}.")
TypeError: NamedTuple('Name', [(f0, t0), (f1, t1), ...]); each t must be a type Got Fees(flat=0.0, percent=0.1, contract=0.2).
nova meteor
can someone guide me to where there are discussions that are you know very low level
peak spoke
You need to use the class itself for the typehint, and then construct an instance for the value
vast junco
You need to use the class itself for the typehint, and then construct an instance for the value
@peak spoke
I can guess what you mean, but not with any accuracy. What would this line look like?
fees: Fees() = (1.0,2.0,3.0)
fees: Fees = Fees(...), the typhint needs to be to the type of the value you expect, not an instance
nova meteor
can someone guide me to where there are discussions that are you know very low level
?
maybe a helper?
vast junco
@peak spoke that did the trick, thanks!
grave jolt
@nova meteor If you want to get help with something, use a help channel. See #❓｜how-to-get-help
azure wraith
Hey y’all. So I was kinda just wondering if anyone out there has used Swift with python?
I’ve really been wanting to dive back into Swift, Swift UI but the reason I dropped it was because it just wasn’t practical for me. It served no purposes. Python however did at the time. But now I want to like intertwine the two. Maybe use python as a backend and use SwiftUI as a front end.
But yeah, just wondering if anyone has ever used python and swift together
stuck barn
i need a python module for console stuff, such as cleaning the console without buffering and mouse input relative to the console
spice pecan
Not the right channel, but curses should at least be similar to what you're looking for @stuck barn
stuck barn
whats the right channel then?
spice pecan
#python-discussion or #user-interfaces probably, this channel serves a completely different purpose
gloomy rain
@stuck barn Check out #❓｜how-to-get-help
stoic sandal
Anyone familiar with scrapy ?
coarse basalt
Never used it but have done plenty of scraping with requests, beautifulsoup, and selenium.
@stoic sandal Just read your thing in #general. Honestly, Selenium would be way easier.
stoic sandal
@coarse basalt I used selenium before but I think scrapy is much faster
novel vector
Even dead turtle would be faster than selenium, use js scrapers for speed, python ones for flexibility
pseudo cradle
Anyone have any thoughts about Dataclasses vs Named Tuples? It looks like named tuples are a little faster to create but dataclasses have faster read times.
spark magnet
i think it would be a very rare application that would choose based on speed.
novel vector
Dataclasses are supposed to be about 10% faster if my memory is not corrupted, but anyways it is very marginal difference in such slow language as python is
spark magnet
i'm not sure why people are so drawn to micro-optimizations
spark parcel
If there’s a choice between two things, where one performs better and they both complete your task, why not choose that one?
Obviously, it’s best to choose the best suited one, but if we assume that the alternatives work in close to the same ways
pseudo cradle
Well, micro-optimizations in my case might be making a difference, I am performing some pretty computationally expensive algorithms. My thoughts so far are that Data Classes might look a little nicer in the autodocs and be slightly faster, while I'm already using namedtuples in the project and am more used to them. Wanted to see if there were other considerations I might be missing between the two
Although, now that I think about it, I'm performing all of those algorithms outside of the data structure, lol.
Most of them, anyway
spark magnet
@spark parcel theere didn't seem to be any comparison of behavior, only speed.
pseudo cradle
My mention about speed was just referring to my limited knowledge of the subject.
lost nexus
I am performing some pretty computationally expensive algorithms.
if you're interested in the best optimization with Python & computationally expensive algorithms then something like Cython may interest you
pseudo cradle
It's something on my plate to learn 🙂 Right now I'm just leaning heavily on numpy/scipy
spark parcel
Yeah. I assumed that the behavior would be the same or very similar, since they were compared
pseudo cradle
In terms of accessing the data, it's extremely similar. The difference is that with a dataclass you're actually creating a class to store the data while you're just creating an object of type/class namedtuple for the other.
It looks a little different in documentation and behaves slightly differently with autodoc generators
I think what namedtuples have over dataclasses is that they are indexable
gleaming rover
i'm not sure why people are so drawn to micro-optimizations
@spark magnet in general?
spark magnet
yes
gleaming rover
I'd say it's because a) humans are very bad at identifying bottlenecks
pseudo cradle
Well, there's a difference between micro-optimization and making the best design decisions.
gleaming rover
and b) very bad at understanding just how fast computers are nowadays
pseudo cradle
The choice of data structure is a design decision
That may have implications down the road of your project
for you*
gleaming rover
(entirely unrelated to your specific case @pseudo cradle)
pseudo cradle
for your*
gleaming rover
just making a general statement
spark magnet
sure, which needs to consider behaviors
gleaming rover
with regard to namedtuple...
lost nexus
I mean, I think people often overoptimize as well.
often
spark magnet
a drawback of namedtuples: they are considered equal if they have the same elements, regardless of their types
gleaming rover
I mean, I've said this many times before; some form of name-destructurable container would be really nice
and also that
spark magnet
they are also accidentally iterable.
gleaming rover
example:
class Point(NamedTuple):
x: float
y: float
pt = Point(1.5, -2.0)
# which is right? you can't tell without looking up the definition
x, y = pt
y, x = pt
It has been proven that the most of 'too slow' applications has some flaws on the algorithm level, like let say traveling salesman problem, u can't optimize code good enough to handle the mathematical problems so i was never into micro-optimizations
spark magnet
a downside to dataclass: you can't do x, y = pt
pseudo cradle
I mean sure, you reduce time complexity before you optimize the code beyond that
gleaming rover
compared to:
scala> case class Point(x: Int, y: Int)
defined class Point
scala> val pt = Point(1, 2)
pt: Point = Point(1,2)
scala> val Point(x, y) = pt
x: Int = 1
y: Int = 2
Ah, you can't unpack dataclasses like you can namedtuples?
gleaming rover
You could do pt = Point(x=1.5, y=-2.0) though, right?
@pseudo cradle perhaps I was not clear
Ah, you can't unpack dataclasses like you can namedtuples?
@pseudo cradle nope
the point is not how a Point object is initialised
but how it is destructured
pseudo cradle
Gotcha
spark magnet
you can use x, y = dataclasses.as_tuple(pt)
gleaming rover
you need to visit the definition to know the order of fields
spark magnet
or something like that
pseudo cradle
True
gleaming rover
and there's no simple syntax for destructuring
on the other hand, functional languages can usually pattern match on this kind of abstraction
like the Scala example above
pseudo cradle
nice
I do need to get more comfortable with dataclasses at some point, but I think for my current project/purpose that namedtuples are better suited. Thank you for the conversation, it was enlightening.
raven ridge
rather than the performance differences, I'd be more concerned with the maintainability differences. The fact that namedtuples are iterable and unpackable means that they can't be extended - the fields they contain, and the order that those fields occur in, is now part of your public interface. Reordering fields or adding new ones can break your users.
gleaming rover
and actually, another point: with knowledge and experience comes an ability to understand what optimisations should be made right off the bat.
for example, if you only need to do membership testing with a collection, and that collection will be big, it would be an obvious design decision to use a set.
pseudo cradle
Damn gg, right when I had made up my mind, haha
gleaming rover
but if you never learnt about the differences, you would probably micro-optimise something totally unrelated
raven ridge
if it's a private type used only within your own module, that's nothing to sweat - you have all the flexibility you want to change the type however you want. If it's something public you're exposing to users of a library, though, dataclasses are safer.;
pseudo cradle
In the short term, some users may be exposed to this. In the long term, it's just going to be done through API calls.
It probably needs to either be easily expandable or trivially expandable by myself. It's a tool/library in high demand right now (internally).
So I guess that's something I need to consider, how easy it would be for me to personally maintain/expand
It's true that any use of its iterative capability I'd have to be careful of
raven ridge
the entire reason we're using Python is because we care more about maintainability and speed of development than speed of execution. There's definitely some amount of coders needing to use their best judgment, but if there's ever a tension between maintainability and performance, it's probably better to err on the side of maintainability.
pseudo cradle
Dataclasses would force me to resist the urge to iterate through it, lol
raven ridge
yep.
pseudo cradle
I might need that. It's just so tempting, lol.
While you're hear, gg, do you have any thoughts about Jupyter Notebooks?
here*
raven ridge
Nothing informed. I've barely used it.
pseudo cradle
Gotcha. People either seem to love it or hate it.
raven ridge
I've started using IPython a little bit, but I've hardly done anything with Jupyter.
I don't like the idea of something that shoves me out of the terminal and into the browser, though I get that it's lower friction for others.
pseudo cradle
I've found it useful for presentations and executives seem to love them
But it has downsides
no linter, hidden states, etc.
gleaming rover
I don't like the idea of something that shoves me out of the terminal and into the browser, though I get that it's lower friction for others.
@raven ridge it's really good for data scientists because of the visual aspect IMO
pseudo cradle
Yeah, I'll be working closely with a data scientist in the future, so I need to up my familiarity with it
Also my boss LOVES them for some reason. Like, he literally bugged me every day until I set one up
raven ridge
@gleaming rover yeah - I've seen some cool stuff done with it that blends in HTML, too. It's not a need that I tend to have, so learning Jupyter, and getting used to using the browser as a REPL, isn't appealing to me.
pseudo cradle
You don't ever have to give presentations?
raven ridge
but it's not a "love it or hate it" thing for me. I'm pretty indifferent to it.
gleaming rover
it depends on the kind of presentation that you have to give.
raven ridge
Presentations of data? No.
raven ridge
My presentations are either no slides, screenful of code, and explaining what it does and taking questions, or a powerpoint deck.
gleaming rover
it's a great thing to have, but it's not a Swiss knife
pseudo cradle
I heavily used draw.io and now jupyter notebooks for my presentations
Oh, I have to present things to people who will just look at me blankly if I show them a screenful of code
gleaming rover
yeah, which is why it depends on the kind of presentation.
if your primary audience is nontechnical and you're showing stuff like data visualisation, then Jupyter tends to be a lot more helpful
pseudo cradle
If someone is tech savvy I'll just give them the help docs and tell them to hit me up if they have any questions
peak pollen
I found it quite useful back when I was taking notes for CS lectures and stuff. Having pandoc-flavoured markdown lets you integrate latex expressions in your markdown cells, which was quite handy.
gleaming rover
I found it quite useful back when I was taking notes for CS lectures and stuff. Having pandoc-flavoured markdown lets you integrate latex expressions in your markdown cells, which was quite handy.
@peak pollen oh yes I loved this
pseudo cradle
Oh, I bet they'd be pretty nice for CS classes
gleaming rover
yeah I taught a DS bootcamp and Jupyter notebooks were basically the only place Python code was run
pseudo cradle
Nice
gleaming rover
(which I don't really agree with as a teaching method, but it was helpful)
peak pollen
aha nice
gleaming rover
like there was this perception that Jupyter notebooks were the way to run code and "scripts" (meaning modules) were like an alternative for edge cases
but well. mass-market education.
gleaming rover
thank you 🙂
pseudo cradle
I've found Jupyter notebooks helpful, but pretty awful for actually developing the code
gleaming rover
I've found Jupyter notebooks helpful, but pretty awful for actually developing the code
@pseudo cradle yup, precisely
I mean, I use them to poke around with code sometimes
the fact that you have cells is nice for experimenting real quick
then after a while I move to PyCharm
once I have an idea of what I want to write and what it should look like
pseudo cradle
It is pretty cool that you can just rerun cells without having to rerun the program if you get an error
gleaming rover
yeah, that's a great function
peak pollen
yep, pretty much the only thing I use it for these days is messing about with APIs and/or visualising data
raven ridge
like there was this perception that Jupyter notebooks were the way to run code and "scripts" (meaning modules) were like an alternative for edge cases
@gleaming rover I've taught classes that used repl.it and had pretty similar experiences - the advantage of having something that students can try out right in the browser is that it's really low friction to get started, but it does lead to confusion about how programs are actually written in the real world.
unkempt rock
does _init_ returns None or the class object?
teal yacht
there's a great talk called "i don't like notebooks" if it hasn't been mentioned yet
https://www.youtube.com/watch?v=7jiPeIFXb6U on the caveats of notebooks
raven ridge
@unkempt rock __init__ returns None, __new__ returns an instance of the class.
peak pollen
oh interesting, I'll check it out
teal yacht
funnily enough, it was presented at jupytercon 2018
unkempt rock
then when a new class instance is created, the _new_ method is called and it tries to call _init_?
raven ridge
when you call f = foo(), the interpreter calls f = foo.__new__() to create a new instance, and then calls f.__init__() on the instance to initialize it.
gleaming rover
then when a new class instance is created, the _new_ method is called and it tries to call _init_?
@unkempt rock use `code_here` to format your code
__ will underline stuff
pseudo cradle
Yeah, I've watched presentation
unkempt rock
yep, i messed the backslashs
pseudo cradle
It's really good and did cover a lot of the stuff I was running into as a Notebooks beginner
unkempt rock
when you call
f = foo(), the interpreter callsf = foo.__new__()to create a new instance, and then callsf.__init__()on the instance to initialize it.
@raven ridge also thanks for the explanation :D
gleaming rover
also, __init__ is not called if __new__ does not return an instance of the class
pseudo cradle
Being able to quickly run strings of code I feel is kind of offset by the lack of a linter. Like, I think you can get one in there but it's a massive pita
gleaming rover
e.g.:
class X:
def __new__(cls, *args, **kwargs):
return 0
def __init__(self):
raise RuntimeError
X() # returns 0, doesn't raise RuntimeError
I'd rather have my text editor let me send code to the interpreter directly
gleaming rover
because it wouldn't make sense to run __init__ on an instance of another class
teal yacht
that way I have linting, autocompletion etc
brave badger
!pep 634
fallen slateBOT
mystic cargo
cool
wet obsidian
Well, actually seems like 3 new peps were made
brave badger
The spec, rationale, and tutorial are separate
Which makes me suspect that this might actually be added
wet obsidian
I just assumed it was gonna
btw, I was wondering, what exactly is meant with https://www.python.org/dev/peps/pep-0622/#don-t-do-this-use-existing-method-dispatching-mechanisms
Like, how can oop replace this sort of matching?
brave badger
It's essentially the difference between matching between structures and types
e.g. why dispatch based on the "shape" of sometuing if you can simply dispatch on the types?
wet obsidian
So like, by calling isinstance?
brave badger
and/or using methods
wet obsidian
soooo like.... using polymorphism and sorta replacing the matching with a single call to a method?
stoic sandal
I'm trying to login into a website using request, i post the information but it seems that it just loads the login page again without any errors
can someone help me please
grave jolt
@stoic sandal This is not a help channel. See #❓｜how-to-get-help
wet obsidian
Soooo, wouldn't it be nice if python had support for arbitrary base numeric literals?
and like, there's built in int(string, base), but not the inverse, just hex(), oct() and bin()
spark magnet
the asymmetry does seem odd
unkempt rock
heroku cant import exceptions from asyncio #help-chestnut
@grave jolt can you help me?
peak spoke
@swift imp It means it has been replaced by a newer PEP, https://www.python.org/dev/peps/pep-0001/ has some information on how PEPs are strutured
wet obsidian
see 634, 635 and 636
swift imp
Google isn't finding them
So it's confusing
I found them, man it's crazy they need to break it into 3 peps it's do big
vast lark
can anyone give a look at #help-cake
peak spoke
The peps that supersede something usually have their assigned numbers around the original one, so the index should help there
grave jolt
@unkempt rock Don't ping random people. I just said in my last message that this is not a help channel. See #❓｜how-to-get-help
vocal bluff
Guys is there a float command in pseudocode?
feral cedar
you can make whatever command you want in pseudocode
note that this channel is meant for discussion, you should check out #❓｜how-to-get-help for future questions @vocal bluff
magic python
does anyone know of a good resource for explaining why there is pyproject.toml and setup.cfg, why I would use one over the other, and what the defined purpose of each is... I have projects which use both, but I just follow what was already there, I use flake in setup and pylint in toml, absolutely no idea why
wide shuttle
pyproject.toml is the result of a desire for a better system for specifying build dependencies. There are two PEPs, PEP517 and PEP518, which together explain why pyproject.toml became a thing. Other tools started using pyproject.toml as a configuration file by allowing users to add settings to it. This was initially "forbidden" by PEP518, but that was changed after the authors realized that is was happening anyway and it probably wasn't a bad idea.
Since then, more and more tools allow you to add configurations to pyproject.toml instead of, say, a tox.ini or other config file
We're now in a situation where some tools support pyproject.toml, some tools support setup.cfg, some tools support tox.ini, and some tools support custom config files as well (such as .flake8).
This is a slightly dated write-up on the subject: https://blog.pilosus.org/posts/2019/12/26/python-third-party-tools-configuration/
Slightly dated in that some tools may have added pyproject.toml support since, I haven't checked
What you should use is up to you, although I'm hoping to see something like the pep517 build tool to come out of its "experimental" phase and into real use. My guess is that more and more tools will start supporting pyproject.toml, which means that you may end up with fewer config files in the future
unkempt rock
there's any way to get an object from the superclass of an instance without inner class calls?
wide shuttle
You should be able to access it directly using b.b, as class a will be part of the MRO of class b
!e
class A:
attribute = 10
class B(A):
pass
b = B()
print(b.attribute)
@wide shuttle :white_check_mark: Your eval job has completed with return code 0.
10
@wide shuttle thanks a lot for the context and links 🙂
pseudo cradle
Does anyone else find themselves using python more and more like a typed language?
With type annotations, mypy, heavy reliance on typed libraries like numpy, etc
Even dataclasses rely on fields being typed
teal yacht
numpy doesn't have type stubs yet
sacred tinsel
yes, I annotate all my code, but I don't use mypy much
usually without the annotation you have some kind of type documentation in the docstring, to me the annotations are a more effective way of doing that can also be used by tools such as mypy or your IDE
grave jolt
I'm using annotations in most of my code, and I'm using pyright in real-time (in VSCode)
pseudo cradle
It feels like I'm going against one of the fundamental tenets of the language by only living in the typed areas
sacred tinsel
it's part of the language's syntax, you're using a native feature
pseudo cradle
I'm using a native feature, but one that feels like a concession as opposed to a primary component
teal yacht
i mean at this point, the benefits of dynamic typing are close to non-existent, it's just now up to the language to provide us with the tools we need to express statically typed programs without it being ugly and verbose
pseudo cradle
I never was a fan of the dynamic typing. I'm sure there are uses for it, but I can't think of any
grave jolt
Well, you can still have not-fully-typed (like Dict[str, Any]) things, and you can have untyped areas/modules.
If your code is fully annotated and has no magic/unspecified relations, then the code probably could've been rewritten in a statically typed language.
pseudo cradle
I guess that's useful, but it seems like a union would do just as well, unless you truly needed it to be anything
as opposed to just some selection of types
peak spoke
The dynamic part makes it easy for others to pop in their own behavior and testing that's easily implemented
teal yacht
Any in general is either you doing something weird or you being lazy
pseudo cradle
Maybe it's just not useful in the stuff I do. I deal with complex algorithms and there's never a time I want something untyped
teal yacht
or the type system being too limited, which happens way to often with the current state of python
pseudo cradle
Yeah, but then, that's a more fundamental question. Whether or not typing should be more supported in Python
teal yacht
i like the idea of adding gradual typing in existing dynamic languages, but one issue here is that the core maintainers are strictly against using them for optimization or doing any kind of static analysis directly in the interpreter
so it doesn't encourage people to use them
pseudo cradle
Brief aside, does anyone have any opinions on creating a dataclass vs importing the make dataclass method?
grave jolt
huh?
teal yacht
i'm not sure i understand
make_dataclass is an alternative way of creating dataclasses, it's meant to be used instead of defining a new class, i'm not sure what you mean by "importing" here
pseudo cradle
Sorry
What I meant was, defining a dataclass versus using make_dataclass
Could have been more clear on that
teal yacht
by default, use a proper class definition
grave jolt
Well, make_dataclass won't play nicely with static analysis tools, right?
like namedtuple vs NamedTuple
pseudo cradle
Right, it's similar to namedtuple's method
It's probably better to use a class definition if you're not creating the dataclass algorithmically
grave jolt
Yes
teal yacht
dynamic creation of types is ugly
grave jolt
I haven't used that in non-esoteric code, I think
I had a piece of code that created a metaclass and a metametaclass dynamically
@pseudo cradle Some people recommend Julia as a statically typed alternative for data scientists, but I haven't tried it. It surprised me with some cool concepts a while ago, but I don't remember what exactly. Maybe you could try it.
pseudo cradle
Maybe I'll check it out
I've definitely heard the name being tossed around enough
same with Rust
grave jolt
It probably doesn't have the same amount of libraries, though.
pseudo cradle
Yeah
I'm not a fan of Python's dynamic typing, but I am a fan of the ecosystem and the sheer amount of open source libraries with permissive licensing
teal yacht
julia isn't statically typed
i have some tiny hope in nim in that regard, but i doubt python is going anywhere
it'll probably stay the industry standard for at least a few decades
swift imp
julia doesnt interest me, its supposed to be a simulink/matlab replacement but its just meh
I get it, its free but like why would want to write C/C++ code for Julia when I can just use Simulink and it generate C/C++/HDL code for me.
teal yacht
I am not following
one benefit of julia is that you get decent performance without having to wrap C++ code
that's one of the main selling point of flux.jl, it's written in pure julia, making it easier for people to contribute
unlike python libraries
grave jolt
julia isn't statically typed
really?
swift imp
If you want to SIL/PIL/HIL testing you have to write your own C/C++/HDL code in Julia and then use it in Julia
The most stringet of engineering implementations using model-based designs is a MIL (Model in loop, i.e. raw simulink) -> SIL/PIL (software in the loop/processor in the loop) -> HIL (hardware in the loop)
grave jolt
really?
Oh, it seems so.
But I've heard that it's faster than Python. Is it? Wikipedia says it has a JIT/JAOT compiler
swift imp
You want to test how your production code works in your model you have tow rite it yourself in Julia
pseudo cradle
Also, one of the biggest reasons for julia over matlab is that matlab is $$$
teal yacht
it uses type hints to optimize code and do multiple dispatch on it, but there is nothing ensuring your code is properly typed
swift imp
Simulink can auto generate it directly from your model, and can even port it direct boards such as specific FPGA/Processors
teal yacht
julia is much faster than native python in almost every regards
pseudo cradle
Yes, there are industry matlab shops where the company is the one paying the licensing, but still
teal yacht
it's about as fast as numpy for numerics
swift imp
teal yacht
the performance imo is a non-argument, the main selling point is that you can actually write good old regular idiomatic julia and not depend on the quirk of a library
swift imp
Well if you're doing that sort of modeling you're probably in big industry and have the money
grave jolt
(just a reminder: if this goes wildly off topic of python, let's move to #ot-*)
teal yacht
numpy becoming so prevalent is a curse but also a blessing, as if you know it, you know how to use almost every tensor library
pseudo cradle
I can't remember the last time I wrote a module that didn't import numpy
swift imp
Very regularly, pandas can get done 90% of my work, which uses numpy but still I find myself not directly importing it
pseudo cradle
Simulink is pretty neat though, I'm a fan
But when my student license expired, that was it lol.
swift imp
Yes, its expensive but most engineering design software is. Freaking any kind of EDA tool is going to cost an arm and leg.
pseudo cradle
I'd have to justify the license to my company and in good conscience I can't.
Because I can get plenty done with numpy/scipy
and pandas, etc
swift imp
The problem with bare simulink in industry is that it isn't enough. So unless you want to shell out the money for the CodeGen support and application specific toolboxes its not worth it.
pseudo cradle
Yeah, precisely
The head of our science division has a license, and to the best of my knowledge he's it
and he can do codegen and all that jazz
swift imp
Yeah, its really good if you do that. The turn around on implementations can be fast
pseudo cradle
I'm a little jealous with how much faster his algorithms run than mine, even though we have like the same implementation.
We're talking seconds, but still
swift imp
Ive modeled digital fitlers and controllers have directly ported it do an FPGA of my choice, dictating where and how to use the built in DSP/FFT cores
pseudo cradle
Nice. I work quite a bit with digital filters and do a lot of dsp
swift imp
Again its big $$$
pseudo cradle
Maybe it just depends on industry. I'm in neuroscience and we're pretty well funded, but maybe not as well funded as other industries.
This is getting off-topic though
swift imp
for sure, interesting talk though : )
pseudo cradle
Definitely
unkempt rock
can somebody tell me the use cases of partial functions in python?
sacred tinsel
mm, the way I think about partials is that they allow you to "pre-fill" some arguments to a function, while leaving others to be filled in later
let me give a super simple example, let's say we have this function:
>>> def say_hello(animal_a, animal_b):
... print(f"{animal_a} says hello to {animal_b}!")
...
>>> say_hello("Cat", "Bird")
Cat says hello to Bird!
now let's imagine that in some part of your code, you want to call this function many times, but animal_a will always be the same
we could therefore create a partial, which will automatically fill in animal_a, and will wait for animal_b to be specified
>>> import functools
>>>
>>> elephant_says_hi = functools.partial(say_hello, "Elephant")
>>>
>>> elephant_says_hi("Snake")
Elephant says hello to Snake!
>>> elephant_says_hi("Lizard")
Elephant says hello to Lizard!
now elephant_says_hi is a partial which when called, will pass "Elephant" as the first argument, and whatever you call it with as the second argument
however in practice I usually come across partials being used to "delay" the call to some function, but with the arguments already filled in
for example, if we do:
>>> say_hi = functools.partial(say_hello, "Elephant", "Dog")
say_hi will now, when called, call say_hello with those two arguments, so we can for example pass the partial as a callback to some other function
>>> say_hi()
Elephant says hello to Dog!
I hope that makes sense
for another use case, in one project that I worked on, I had a function return a dict mapping from ints to partials, and the caller would then choose the partial to call based on the key - the justification for this was that the function wrapped by the partial was very expensive, and it only needed to run once, but within the function I didn't know which arguments to use (because the decision partly depended on another function)
so I prepared them for the caller, i.e. "choose which one to call"
of course, this could have been avoided had the whole thing been designed better
deft pagoda
i used them in my truthtable project for specific binops
And = partial(BinOp, 'and', lambda p, q: p and q)
Or = partial(BinOp, 'or', lambda p, q: p or q)
Implies = partial(BinOp, '->', lambda p, q: not p or q)
Iff = partial(BinOp, '<->', lambda p, q: p == q)
Xor = partial(BinOp, 'xor', lambda p, q: p != q)
could just subclass, but this was less typing
grave jolt
Actually, in Python, methods are kind of partial functions.
class Duck:
def quack(self, loudness):
print(f"{self}:", "qu" + "a"*(1 + loudness) + "ck")
dave = Duck()
dave.quack(42)
dave.quack works like Duck.quack.partial(dave)
>>> dave.quack
<bound method Duck.quack of <__main__.Duck object at 0x7fab47705b80>>
thanks @sacred tinsel
yeah so now I see them like less typing with the same functionality
brave badger
Sometimes I wish lambdas are way nicer than what they are currently
deft pagoda
always
brave badger
smh I just want to be able to declare predicate functions inline without lambda all over the place
deft pagoda
lambda p, q: p and q
-->
p and q from p, q
also pep 8 discourages lambda like
f = lamda x: x * x and opts for
def f(x): return x * x
is this due to readability?
brave badger
Debugging as well
deft pagoda
well, lambda's are meant to be anonymous -- if you're assigning them to a variable, you should use normal def
unkempt rock
hmm so instead it is preferred to use them inside a function like map or filter
right?
deft pagoda
usually
unkempt rock
right
unkempt rock
def logger(msg):
def log():
print('Log', msg)
return log
log_hi = logger('Hi')
log_hi()
Isn't this stub confusing? I mean why would you do that? I was reading about First class functions and closures btw.
deft pagoda
my ipython console accepts λ
add this to your startup file:
import re
__λ__ = re.compile(r'λ[a-z]+[.]')
def __λ_sub__(m):
m = m.group()
return f'lambda {", ".join(m[1:-1])}:'
def _transform_λ(lines):
return [__λ__.sub(__λ_sub__, line) for line in lines]
get_ipython().input_transformers_cleanup.append(_transform_λ)
this expects a . after single-letter variables though: λpq.p and q
grave jolt
@unkempt rock Well, educational/explanational examples are often useless.
unkempt rock
okay can you show me a real world example?
mint forge
gloomy rain
@grave jolt I've often used classes as partials in Java.
mint forge
3.9 is launching in oct 5th apparently. Didn't Know that
gloomy rain
Like, prepopulate certain parameters in the constructor and supply others on the fly through method arguments.
grave jolt
@unkempt rock example of what? first-class functions and closures?
unkempt rock
yeah
deft pagoda
do you want real world example of nested functions?
or this partial-like use of it specifically
unkempt rock
nvm I will just skip this part and come to it later
grave jolt
@unkempt rock Have you already learned about decorators?
unkempt rock
nah its still in TODO, the prerequisite was first class functions and closures.
grave jolt
Well, decorators and decorator factories are one application of nested functions.
And functions being first-class in general.
gloomy rain
@unkempt rock In the case of logging, actually executing the log call, or constructing the log string, can be expensive, so you might only want to do that if the log level is set high enough.
So it would make sense to construct the function that creates the log entry in one place and running it in another (and only if it's actually necessary).
unkempt rock
I see
lusty patrol
Woah. Wait. Does Python's print() function use sys.stdout? And does Python's input() use sys.stdin?
deft pagoda
here's a example decorator you might see in the wild:
In [15]: def run_once(func):
...: def wrapper(*args, **kwargs):
...: if not wrapper.has_run:
...: wrapper.has_run = True
...: return func(*args, **kwargs)
...: wrapper.has_run = False
...: return wrapper
...:
...: @run_once
...: def hi():
...: print('hi')
...:
In [16]: hi()
hi
In [17]: hi()
# No output
@lusty patrol Yes.
lusty patrol
Dude.
deft pagoda
you can redirect print with file kw
gloomy rain
You can read from a file with input(), or from a string, or from a socket probably
cloud crypt
Hello everyone, little question about typing.
so uh, I have a zip function in a class that has generic T
and so, when I write out with several types: python class Iter(Generic[T]): @overload def zip(self, __iter_1: Iterable[T1]) -> Iter[Tuple[T, T1]]: ... @overload def zip(self, __iter_1: Iterable[T1], __iter_2: Iterable[T2]) -> Iter[Tuple[T, T1, T2]]: ... @overload def zip(self, __iter_1: Iterable[T1], __iter_3: Iterable[T3]) -> Iter[Tuple[T, T1, T2, T3]]: ... ... and I can't seem to find a way to type it for a variadic amount of arguments
since Tuple[T, Any, ...] is not valid
deft pagoda
yeah, i think overload only works for single dispatch
or maybe not
did you try it?
cloud crypt
no like
not that
I have @overload everywhere
but the thing is, if you write like python def zip(self, __iter_1: Iterable[Any], __iter_2: Iterable[Any], __iter_3: Iterable[Any], __iter_4: Iterable[Any], *iterables: Iterable[Any]) -> ? it won't accept Iter[Tuple[Any, ...]] as the return type
telling that it is incompatible
i.e. Overloaded function implementation cannot produce return type of signature n
deft pagoda
-> Iter[Tuple[Any]] doesn't work?
cloud crypt
should be -> Iter[Tuple[Any, ...]], going by that logic
deft pagoda
why
unkempt rock
It unfortunately does mean a Tuple of exactly one element with type Any
cloud crypt
either way, does not work
unkempt rock
Example:
Tuple[T1, T2]is a tuple of two elements corresponding to type variablesT1andT2.Tuple[int, float, str]is atupleof anint, afloatand a string.To specify a variable-length tuple of homogeneous type, use literal ellipsis, e.g.
Tuple[int, ...]. A plainTupleis equivalent toTuple[Any, ...], and in turn totuple.
cloud crypt
this is so infuriating lmao
deft pagoda
that's inconsistent with the Iterators but ok
unkempt rock
It's not inconsistent, no
graceful junco
Hello, I have a problem with the Visual Studio Code editor, namely Rich Presence does not start.
cloud crypt
well tuple is not a true iterator in many langauges
since it is defined to be pretty much a well-known group of different types
@graceful junco that's a wrong channel for that
graceful junco
I was not answered in the main channel
graceful junco
Ok
deft pagoda
another reason not to type things
cloud crypt
hah
deft pagoda
better to leave__annotations__ for esoteric meta classes
unkempt rock
@cloud crypt based on the fact this issue is still open there doesn't seem to be a "neat" way to do what you want to achieve https://github.com/python/typing/issues/193
cloud crypt
F
worldly pebble
Does python release the memory once you del a object?
paper echo
Arguably variable length tuples are semantically inconsistent
flat gazelle
Maybe just do Sequence[Any]
teal yacht
if the refcount reaches 0, then yes @worldly pebble
cloud crypt
in the final return?
paper echo
Tuples are supposed to be for fixed/known length records imo
cloud crypt
well zip is really a special case
unkempt rock
The tuples are fixed-length, based on the input parameters
cloud crypt
yeah
worldly pebble
import pandas
df = pandas.read_csv('large_txt_file.txt')
del df
Let's assume the df size is 2gb
Does deleting is efficient
or is there some other way to free up?
unkempt rock
a given function call to zip will only return tuples with size N
flat gazelle
I mean, you are asking for dependent typing
Since the amount of args to zip can vary
paper echo
@worldly pebble if that is the only code referencing df then it should free that memory
worldly pebble
What if there is some processing between read and del? @paper echo
flat gazelle
What type should
zip(*['123']*randint(1,10))
``` havealso you should not read the whole file instead read chunks by chunks ig
worldly pebble
import pandas
df = pandas.read_csv('large_txt_file.txt')
Some processing on df
dictionary = {some_processed_parts_of_df}
del df
Is the above one more efficient
or
import pandas
df = pandas.read_csv('large_txt_file.txt')
Some processing on df
dictionary = {some_processed_parts_of_df}
df = pd.DataFrame()
Or doing implicit garbage collect
import pandas
import gc
df = pandas.read_csv('large_txt_file.txt')
Some processing on df
dictionary = {some_processed_parts_of_df}
del df
gc.collect()
if a SINGLE thing references to df, it won't be deleted
worldly pebble
what if all the references are also deleted @cloud crypt
cloud crypt
if you are sure about it, then yeah, it will be deleted
boreal umbra
@worldly pebble you can't force something to be deleted at a specific time.
worldly pebble
How do I handle it then
unkempt rock
The gc.collect() version is the one most close to "perfect"
as it forces a gc-cycle
worldly pebble
Thought So
boreal umbra
How resource constrained is your environment?
unkempt rock
If memory is really an issue you can disable garbage collection completely and write your own manager for the memory, this is what we did at our company
worldly pebble
Because I have a huge csv file like 2gb
I am storing some transformations into dict
and I don't want to have a 4gb in usage
So how do i free up the csv file memory read as it's not required
teal yacht
gc.collect doesn't interact with reference counting
paper echo
@unkempt rock you can do that in python? Or did you modify the C code?
worldly pebble
and when a refcount reaches 0, it directly destroys the object
@teal yacht can we do it explicitly
paper echo
@worldly pebble the 2nd makes more sense
teal yacht
yes, using del
cloud crypt
del is simply obj.refcnt -= 1
paper echo
Also I recommend experimenting. Just try it and look at memory usage
cloud crypt
^
boreal umbra
del doesn't guarantee that you've fully decrement the reference count
teal yacht
of course not, there's no way to guarantee that
but there's no way to guarantee anything in python, and we live with it
worldly pebble
@worldly pebble the 2nd makes more sense
@paper echo Thanks
boreal umbra
We had a chat a while ago about what it would take to have forced object deletion
It would be really bad, apparently
peak spoke
@paper echo you have to go in C to interact with the ref counting GC, I recall an article from instagram that did something with it to increase their performance by quite a lot
teal yacht
you either have to tank performances by scanning every objects to find every reference to the object you want to delete, or you directly free it, risking invalid objects later on
grave jolt
@cloud crypt If you really need something like this, you could use immutable linked lists :D
Tuple[int, str, str, *Any]:
V = TypeVar("V") # or something like that
Vec[int, Vec[str, Vec[str, V]]]
The implementation could actually use a contiguous array (i.e. tuple/list).
cloud crypt
did I hear recursive annotation here haha
grave jolt
yeah 😎
no
grave jolt
from typing import TypeVar, Generic
T = TypeVar("T")
U = TypeVar("U")
V = TypeVar("V")
class Vec(Generic[T, V]):
def __init__(self, hd, tl):
self.hd = hd
self.tl = tl
def prepend(x: U, vec: Vec[T, V]) -> Vec[U, Vec[T, V]]:
return Vec(x, vec)
a: Vec[str, Vec[str, None]] = Vec("hi", Vec("hello", None))
b = prepend(4, a)
b is inferred to be Vec[Literal[4], Vec[str, Vec[str, None]]]
in Pyright, at least
Alternatively, you could just do a nested tuple (Tuple[int, int, str, Tuple[Any, ...]]) as a compromise
cloud crypt
not like I can apply it in my Tuple[T, T1, ..., TN] case
signal tide
T being a tuple?
unkempt rock
can i take an object from a function that i'm decorating inside the decorator?
since the object only exists inside the function, and is not passed into the parameters
signal tide
Maybe I’m misunderstanding but could you not do Tuple[(T: Tuple[Types])* n] or whatever the syntax would be for that
cloud crypt
there is no such syntax
thing is, there is a fixed T and zip function which returns Tuple[T, ...] where ... is Any, ..., and it isn't compatible with T, T1, T2, T3 signatures
signal tide
Could you not redefine them to fit the case?
solar delta
Has anyone applied multivariate testing in python? ( Multivariate in the context of AB testing )
somber halo
#data-science-and-ml might be more adequate for that question.
covert saddle
is it a good idea to call shell commands using subprocess.call?
sacred yew
wrong channel
covert saddle
not really, we are having a heated debate about it in #ot0-psvm’s-eternal-disapproval. Just wanted to know what the pros think about it.
empty kite
Iirc subprocess.call is the standard for Python <= 3.5. Nowadays, you would use subprocess.run instead @covert saddle
spark magnet
@covert saddle what is the alternative? Calling shell commands with something else? Not calling shell commands? Using Popen directly?
covert saddle
if the shell command is Linux only, i tend to write bash scripts that run the python script, capture the output and then pipe it back to the script as an argument
but I don't know if this is the correct way to do it, and that's why I asked
spark magnet
i don't understand how that description relates to "call shell commands using subprocess.call"
covert saddle
I agree, i framed the question incorrectly. The subject of my question is calling shell commands from python scripts, not subprocess.call.
paper echo
Shell builtins wouldn't be accessible from there... right?
Since the whole point is that it sends the arguments to the kernel and bypasses the shell
@cloud crypt i still think you should just use Sequence here
cloud crypt
@paper echo well eh, not really... doesn't work kek
paper echo
Why doesn't it work
signal stratus
do we have pattern matching yet 
cloud crypt
nah
@paper echo Tuple[T, T1, T2, ..., TN] and Tuple[Any, ...] are not matching -- and so is not Sequence[Any]. In the end, @no_type_check will do I guess lmao
unkempt rock
is there a library that can like for 15 or 10 times per second so its not intensive , check if part of a screen looks the same , maybe can even recognize text ?
cloud crypt
not really #internals-and-peps, tbh
cloud crypt
just, nevermind, it's not going to function that way, I tried it and it didn't work
shadow musk
Hi, can someone explain why I can't do f = lambda n: eval('all(n % i != 0 for i in [2,3,5])') to get a function that returns that n is not divisible by 2, 3 or 5? When I do f(10) I get "NameError: name 'n' is not defined". However g = lambda n: eval('n > 10') works fine to define a function that returns that n is greater than 10 and so g(10) returns False
And yes I explicitly want to use eval since I actually need to evaluate user provided strings
uncut sage
@shadow musk this isn't a help channel
shadow musk
@hollow crane Sure, same question for def f(n): return eval('all(n % i != 0 for i in [2,3,5])')
hollow crane
ah i see
i mean scoping with eval is kind of weird anyway
i wouldnt have thought your g example would work
shadow musk
g(10) works in my python 3.8.2 at least
grave jolt
@shadow musk Why are you using eval at all? 👀
eval accepts globals and locals as extra parameters, I think it doesn't receive information about the enclosing scope as well.
Oh, wait, the issue may be that a generator expression/list comprehension creates a new stack frame, so n is actually a nonlocal variable.
shadow musk
@grave jolt Hmm, so what about def h(n): return eval('n == 0 or h(n-1)') ? h(10) creates a new stack frame since it does a recursive call to h but this evaluates fine
grave jolt
I have no idea how eval works here. What you're doing belongs to #esoteric-python
raven ridge
eval('all(n for _ in "abc")', {}, {'n': 3}) fails too.
I'm inclined to agree, this seems specific to generator expressions and list comprehensions.
In [20]: eval('[n]', {}, {'n': 3})
Out[20]: [3]
In [21]: eval('[n for _ in "abc"]', {}, {'n': 3})
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
yep. ```
~>python2
Python 2.7.17 (default, Apr 15 2020, 17:20:14)
[GCC 7.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
eval('[n for _ in "abc"]', {}, {'n': 3})
[3, 3, 3]
it's probably related to the change that made comprehensions' loop variables not leak
grave jolt
yep <#esoteric-python message>
flat gazelle
Seems like you cannot construct closures over non-global variables from inside eval.
f=lambda n:eval('lambda:n')()
``` also errorsthis seems like a bug to me, honestly
grave jolt
kind of yes... everything in eval for which scope is not resolved should be looked up using LOAD_NAME
flat gazelle
!e
s='Hello'
def print_(s):
return print (eval('lambda:s')())
print_('Bye')
@flat gazelle :white_check_mark: Your eval job has completed with return code 0.
Hello
This seems pretty buggy to me
raven ridge
In [30]: def foo():
...: n = 10
...: print([n for i in "abc"])
...: eval('print([n for i in "abc"])')
...: foo()
[10, 10, 10]
NameError: name 'n' is not defined
``` sure don't seem right.whoa:
In [32]: def foo():
...: s = "abc"
...: print(eval('[c for c in s]'))
...: print(eval('[s for c in s]'))
...: foo()
['a', 'b', 'c']
NameError: name 's' is not defined
that makes sense to me 🙂
because the iterable is passed as an argument to the list-comprehension-function, not accessed as a nonlocal variable from inside of it
I mean, it doesn't really make sense and is a WTF thingy, but it is explainable
!e
import dis
def f():
return [c for c in s]
dis.dis(f)
god damit
4 0 LOAD_CONST 1 (<code object <listcomp> at 0x7f9c8a2aabe0, file "<string>", line 4>)
2 LOAD_CONST 2 ('f.<locals>.<listcomp>')
4 MAKE_FUNCTION 0
6 LOAD_GLOBAL 0 (s)
8 GET_ITER
10 CALL_FUNCTION 1
12 RETURN_VALUE
Disassembly of <code object <listcomp> at 0x7f9c8a2aabe0, file "<string>", line 4>:
4 0 BUILD_LIST 0
2 LOAD_FAST 0 (.0)
>> 4 FOR_ITER 8 (to 14)
6 STORE_FAST 1 (c)
8 LOAD_FAST 1 (c)
10 LIST_APPEND 2
12 JUMP_ABSOLUTE 4
>> 14 RETURN_VALUE
As you can see from the first few lines, it's doing <listcomp>(s)
raven ridge
I mean, it doesn't really make sense and is a WTF thingy, but it is explainable
Yeah - I get why it is what it is, but I think it's pretty surprising from a user POV.
But my go-to core dev friend says it's supposed to be like that. 🤷
it seems really surprising to me for a single syntactic construct to have multiple distinct scopes. I get why it needs to be that way, but still - really surprising.
swift imp
What is the purpose of ast
boreal umbra
!doc ast
fallen slateBOT
swift imp
Yeah I read that, it means nothing to me
gleaming rover
Yeah I read that, it means nothing to me
@swift imp an AST (abstract syntax tree) is a representation of code halfway between source code and bytecode.
if you could explain what you did not understand about the doc @boreal umbra linked, it might help
boreal umbra
for the record, idk what ast is for either, I just like how you can link to the docs with the bot.
spark magnet
@swift imp if you want to write a program to process code, the ast gives you a structure to work with
swift imp
Thank you, I see now
brave badger
ParamSpec sure is interesting
wet obsidian
So many peps about typing
Really like it, just think it's really a pity that duck typing is so prevalent
it kinda undermines the typing
imo duck typing should be considered bad practice. For other reasons too
quiet stone
Does anyone know R here?
Struggling with a line of code.... older plyr function I cannot figure it out
brave badger
Structural subtyping + composition 
sacred yew
#ot0-psvm’s-eternal-disapproval @quiet stone
wet obsidian
What's there even to gain with duck typing? Like, what do you lose by extending the proper abc(s)?
brave badger
Structural subtyping would allow you to avoid inheritance entirely I think
wet obsidian
polymorphism
because not everything has an abc
I don't get this argument. You can make an abc anytime you want
mystic cargo
it is exactly what it sounds like
you create new functions that will do certain tasks and you use it with your existing functions.
Usually done for not repeating code and making your program more readable
@shrewd bolt however, this is not the right channel to ask that question
wet obsidian
Structural subtyping would allow you to avoid inheritance entirely I think
So? There's nothing inherently bad about inheritance
inheriting from the proper classes create a meaningful "signature" for the class
and has a lot of informational value, much like the signature of a function
it's more powerful, since you have all the goodies of non abstract classes available
and can do stuff similar to default methods in java interfaces
brave badger
Fair enough, although it can also be noted that not all libraries explicitly expose ABCs, but then again, there's already a reason why they aren't exposed in the first place
Similarly, mixin classes typically don't inherit upstream since they're supposed to hijack methods/super calls in the first place
raven ridge
it's more powerful, since you have all the goodies of non abstract classes available
@wet obsidian Inheriting from concrete classes is risky business... You as a user of someone else's class have no way of knowing whether it not correctly handles cooperative inheritance, or whether it will delegate to the wrong class instead of the next class in the MRO, or accepts arguments incompatible with the next class in the MRO, or things like that. Designing for correct behavior under inheritance isn't something you get for free, and it's something you can't assume all classes have done. And as inheritance hierarchies get deeper, it can often be tougher to make sure that the most derived classes are still Liskov substitutable for all their base classes.
It's not that inheritance is inherently bad, it's that there are a lot of gotchas to designing and maintaining correct inheritance hierarchies. Like, what happens if you subclass a class from a library, and you add a new attribute to your derived class, and later the author of the library adds the same attribute to their class with a different data structure, or something.
Inheriting from ABCs is safe, and does have some documentation value, but I wouldn't extrapolate from there to extending concrete types too freely.
undone hare
modern frigate
Yay!
spice pecan
Ooh, and an Alpha of 3.10 with | syntax for unions, nice
unkempt rock
yo i have a unclosed client session error with aiohttp and its annoying me ive tried everything someone help
teal yacht
Finally
spice pecan
It's listed as a coming feature on the "What's new in 3.10?" page
And the status is now accepted, niice
teal yacht
I definitely have complained about that exact same feature in the past month
I'm just a crybaby who can't use google
young oak
how does python3.9 typing work under the hood? it doesnt optimize variables, right? is it only for debugging?
brave badger
They don't affect runtime at all
unkempt rock
guys flask vs django
brave badger
They're there for documentation and for type-checking tools like mypy or pyre
modern frigate
guys flask vs django
@unkempt rock Its a difficult long answer
modern frigate
Basically flask is a build from scratch thing, while django is batteries included
brave badger
@unkempt rock Might be more appropriate to ask in #web-development, not really on-topic for this channel
spice pecan
@mystic cargo | for dict unions came in 3.9, 3.10 will introduce type.__or__ to allow you to write, say, int | str instead of Union[int, str]
spice pecan
I though it was already there for a while, no?
>>> a, b = {1, 2, 3}, {4, 5, 6}
>>> a | b
{1, 2, 3, 4, 5, 6}```| for union, & for intersection, ^ for symdiff, - for diff, probably no complement though
I think there was a pep (or was it just an idea in the mailing list?) for universal sets, that would make it possible to get complement with, say, ~
feral cedar
yeah those existed
violet jetty
can someone help me find some resources for constructing qualitative spacetime models in using Python? things like: classes, github repos, notebooks, and so on...
obsidian bough
Between Pandas, Numpy, Matlab, Scipy, ect.... which do you focus to learn first? Also, how long does it take to learn pandas, Numpy, MatLab?
somber halo
Why Matlab in the middle there? (also, this is not the right channel for this question)
grave jolt
@spice pecan well... a complement wouldn't be a set anymore, right?
could still be useful, though
spice pecan
It could a subclass of set
Something along the lines of universalset, which inverts the membership tests and keeps the items that are excluded from it
Although it does pose a lot of questions, like "How would it be printed?" and "How would it interact with regular sets?"
grave jolt
Well, it would be a completely different data structure, I suppose. Because it would get messy when you | and & and ^ and - it with regular sets and other universal sets. Although you could store it as a tree of predicates:
!e
class Predicate:
def __init__(self, is_true, name="<anon>"):
self.is_true = is_true
self.name = name
def __call__(self, *a, **kw):
return self.is_true(*a, **kw)
def __repr__(self):
return f"<Predicate {self.name}>"
def __or__(self, other):
if not isinstance(other, Predicate):
return NotImplemented
return Predicate(lambda *a, **kw: self(*a, **kw) or other(*a, **kw), f"({self.name} | {other.name})")
def __and__(self, other):
if not isinstance(other, Predicate):
return NotImplemented
return Predicate(lambda *a, **kw: self(*a, **kw) and other(*a, **kw), f"({self.name} & {other.name})")
def __invert__(self):
return Predicate(lambda *a, **kw: not self(*a, **kw), f"~{self.name}")
def __xor__(self, other):
if not isinstance(other, Predicate):
return NotImplemented
return Predicate(lambda *a, **kw: self(*a, **kw) != other(*a, **kw), f"({self.name} ^ {other.name})")
A = {1, 2, 3, 4, 5, 6, 7, 8}
B = {3, 6, 9, 12, 15}
in_a = Predicate(A.__contains__, "A")
in_b = Predicate(B.__contains__, "B")
p = in_a & ~in_b
print(p)
for x in [2, 3, 5, 12, 100]:
print(x, "=>", p(x))
@grave jolt :white_check_mark: Your eval job has completed with return code 0.
001 | <Predicate (A & ~B)>
002 | 2 => True
003 | 3 => False
004 | 5 => True
005 | 12 => False
006 | 100 => False
the realization of a universal set is strange
Like how do you know apriori what all the elements are
grave jolt
Well, you're usually working in some domain -- like real numbers, strings, players etc.; and the universal set is just your entire domain.
swift imp
yeah but like
Its not like set theory where U\A where A is subset of U and U is universal set
like
U\A would give you everyhthing in U after removal of A
you cant realize that in a program
bc you dont know apriori what the elements of U are
grave jolt
I mean, it depends on how you implement sets. If sets are hash sets, then that, of course, is impossible.
If you implement them as functions T -> bool, then it's easy
swift imp
Ive done something like
class UniversalSet:
def __contains__(self, other):
return True
...But then you may not be able to list all the elements in a set, which you might not need
swift imp
but its not useful outsideof just satisfying a part of my program where I couldnt construct a finite set of values
grave jolt
If you can combine this object with other sets and predicates (like in my example), it could be pretty useful if you need some fairly complex conditions -- now you can work with conditions are first-class objects
swift imp
eh
grave jolt
Well, I never needed that 🙂
swift imp
Tell me what you're gonna do with a set or predicate that cant be done with straight boolean algebra, they all literally have the same base operations just described in different domains
grave jolt
wdym?
Well, yes, of course
if you're only concerned with checking membership, a set of T is the same as a function T -> bool
swift imp
I'd love to see applications of it being useful
grave jolt
well, I haven't seen any application of it per se, I just pulled it out of my ass
...as an example of how sets with ~ can be implemented
swift imp
I think the absence of ~ for sets is due to the fact that the majority of complementary operations for mathematical sets i.e. ~ devolve into either the empty set, universal set, or some set difference involving the unversal set and given that you can't have a universal set when your sets are based on hashes theres no need for it
anyway im rambling
I suppose it is possible to construct a universe U in programming and have fullblown set notation for that would be cool as hell
thinking like
some parent class that has a __init_subclass__ that verifies every element of child is in parent and then you can have that fun
stiff gust
Does anyone know how to find a peek in a 2dimensional array
feral cedar
@stiff gust this channel is meant more for discussion. #algos-and-data-structs is a much better channel for that question
forest flicker
I'm very surprised to learn that pypy is written in python. How can pypy be generally faster than cpython, given that pypy is written in python and cpython mostly written in C? how can scripting language be faster?
spark magnet
@forest flicker the algorithm is more important than the language.
PyPy is a JIT compiler.
raven ridge
pypy turns the code that it runs into machine code. CPython does not do that.
boreal umbra
So why is pypy not the canonical implementation?
sacred yew
pypy lags behind cpython on versions
I think its only 3.6 rn
plus pypy doesn't work well with c extensions and short programs
safe hedge
Also is the speed improvement that big of a deal for like 90% of users?
flat gazelle
Pretty sure pypy also has worse startup times, so worse for short scripts
raven ridge
plus pypy doesn't work well with c extensions
It needs to emulate CPython's reference counting and API just to make C extensions work, and that emulation makes it slower than CPython for running those extensions.
Which has the paradoxical effect that the libraries where performance mattered enough to make them as fast as possible in CPython are the exact libraries that are slowest in PyPy.
safe hedge
So effectively it’s faster for heavy work being done pure Python?
visual shadow
Aye. At the end of the day, effectively heavy work is usually "outsourced" to C or something similar, so Cpython wins out simply because you're no longer running number crunching through native python anyways. (native python being the area where pypy smokes Cpython)
safe hedge
Well in pypys case yes
@radiant fulcrum I mean that is what we were discussing...
radiant fulcrum
Yeah but it's the discussions between PyPy vs cpython,
C lib will still be faster than anything PyPy could reasonably do
paper echo
Is it safe to assume that the memory layout of data in PyPy is incompatible with that in CPython?
red solar
is the memory layout not an implementation detail? i don't see how this could be an issue in any code
(but also it's always safe to assume incompatiblity)
C extensions need to be recompiled for PyPy in order to work. https://www.pypy.org/compat.html
from what i can gather, they're incompatible
paper echo
Just curious, it would be fun if you could have a %pypy magic in iPyKernel
But that would be kind of silly if you had to copy data from one runtime to another
formal echo
Is there an equivalent of sys.modules but for spec's? (Not sure Python internals actually need to store it some where, but perhaps it does and if so, I'd like to find them)
unkempt rock
I have an eazy question...
minor sinew
does the ** operator basically mean "give me all of the entries from this dictionary"? that's what it seems like, but i just want to be sure lol
sacred tinsel
when calling a function, it unpacks a dict into keyword arguments:
args = {"a": 1, "b": 2}
myfunc(**args)
This is equivalent to:
myfunc(a=1, b=2)
dense roost
it gives a dictionary of all the arguments passed, I think
sacred tinsel
it can be used in a function signature as well to capture all kwargs (that don't have a corresponding parameter) into a dictionary:
>>> def myfunc(a, **kwargs):
... print(f"Received a: {a}")
... print(f"Kwargs: {kwargs}")
...
>>>
>>> myfunc(a=1, b=2, c=3)
Received a: 1
Kwargs: {'b': 2, 'c': 3}
this means that your function can take arbitrary keyword arguments
notice that a is not in the dict, because the function takes a param named a, only those that "remain" will be captured into kwargs
minor sinew
i was asking because i know that's the syntax for kwargs, and i came across a way to merge dictionary entries into a single dict by just doing:
dict1 = {"a": 1, "b": 2}
dict2 = {"c": 3, "d": 4}
dict3 = {**dict1, **dict2}
It's not really an operator
It's special syntax that is recognized in a handful of places
But yes in general it means something like "unpack keys and values from a dict-like thing"
raven ridge
Or exponentiation, when it's used as a binary operator, heh
hollow crane
And gives correct results for some numbers 
God I've had some unfun debugging involving that
covert saddle
python 3.9 has a new way of merging dictionaries @minor sinew dict3 = dict1 | dict2
obsidian bough
With pandas_datareader, what is a free service to use that dosnt need a key?
spice pecan
Are there any plans to allow using * and ** in comprehensions? i.e. ```py
a = [[1, 2], (3, 4)]
b = [*item for item in a] # Not possible currently, would love to have this
print(b)
[1, 2, 3, 4]```
Currently, this produces a SyntaxError:
>>> [*item for item in [[1, 2], (3, 4)]]
File "<stdin>", line 1
[*item for item in [[1, 2], (3, 4)]]
^
SyntaxError: iterable unpacking cannot be used in comprehension```you can use nested comps, so it is not a high priority I would think
spice pecan
Unfortunate, IMO it'd be a big improvement
This does work, but I feel like * would be more flexible
>>> a = [[1, 2], (3, 4)]
>>> b = [v for item in a for v in item]
>>> b
[1, 2, 3, 4]```I have to admit, I simply can't understand flat nested list comprehension kind of things
teal yacht
Just write the regular for loops in the same order
flat gazelle
what helped me was formatting it like so
[v
for sl in l
for v in sl
]
``` and just read top to bottomexcept first line is last
teal yacht
for v in item:
if predicate:
result.append(expr)```i personally just dont use list comprehensions if i need to do more than like 2 things
teal yacht
Equiv to [expr for item in a for v in item if predicate]
minor sinew
it gets too jumbled up imo
flat gazelle
I find that 2 at a time when the second one just creates multiple values is fine, but it can get horrible for sure
concat=lambda x,y: [
x
for x in [x[:]]
for yv in [[y]]
for _ in y
for a,*b in yv
for _ in [x.append(a)]
for yv[0] in [b]
][-1]
i can see why some people dislike making apis now lol
teal yacht
I mean this would be terrible even with regular for loops
normal field
@spice pecan Python has an external package called more-itertools that handles a lot of these processes involving lots of iterations. There's one called flatten that does what you are asking
https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.flatten
spice pecan
I know that, it was just an example - I'm still rooting for that syntax to be legal
normal field
ah gotcha!
gusty marsh
is there a way to set custom variable names like we set custom strings with .format()
like is there a way using some library or it just can't happen
grave jolt
@gusty marsh This is not a help channel. See #❓｜how-to-get-help
I don't really understand what you want -- can you elaborate or give an example?
gusty marsh
so like (this is just an example):```py
#so if i set this variable
x="Abc"
#i want to do this
2{x}=1234
#that means
2Abc = 1234
there are ways to do that
but don't
use a dictionary or something
and 2Abc isn't even a valid var name
flat gazelle
it is possible with more effort, but ew
gusty marsh
how?
i got one way with dictionary but how to do without that
whatever = {}
x = "yes"
whatever[f'2{x}'] = 1234```Variables are dict keys under the hood
You can access it with __dict__ and do some wonky stuff with it, but you really shouldn't
modern night
you could access locals itself but thats a dict anyway so why not just use your own dictionary
peak spoke
dynamic names aren't something you should really be using, maybe in the case of some classes and their attrs but it's confusing even then
grave jolt
@gusty marsh The whole point of a dictionary is to associate names with values dynamically.
So there's nothing wrong with just using a dict.
sacred yew
any other method is just the dict method with more magic
grave jolt
Anyway, this is not a help channel.
floral forum
is Guido van Rossum in this server?
scarlet pumice
hey im looking for someone interested in helping me make a very flexible learning AI for now its very simple but i will probably enhance it in the future
grave jolt
@scarlet pumice @floral forum This is not a help channel and not #community-meta, please read the channel description. For meta stuff use #ot-* or #community-meta, check out #❓｜how-to-get-help if you want to get help with Python.
grave jolt
@scarlet pumice if you're searching for collaborators, try #python-discussion or an off-topic channel
scarlet pumice
ok thanks
minor locust
curious how it's decided that assert is a keyword assert 1<2 instead of a function like assert(1<2)
hmm.. actually both works
I wonder if it's because assert(1<2) is parsed as assert (1<2)?
flat gazelle
yup
assert is a keyword so that it has no deeper stacktrace
much better than sys._getframe
unkempt rock
yo im having errors with my shit im basically tryna figure out what do u add to await member.edit(name=rename_to) for it to rename someones nick name like their name, but then the thing after it so lets say someone was named callum and i did !nick @callum lol it would make it callum lol not just lol
feral cedar
this isn't a help channel @unkempt rock you might have better luck in #discord-bots
boreal umbra
@minor locust assert is special because assertions can be turned off. So if assertions are off, the code after assert won't even be evaluated.
unkempt rock
does collections.UserDict inherits from dict?
boreal umbra
On a related note, and you might already know this, but there's nothing you can do to stop people who are running your code from turning them off. So you'll never want to write something that depends on assertions.
@unkempt rock I'm on my phone and my phone is almost literal garbage, but you can follow the docs to the source code. I tried and my phone froze.
!docs collections
fallen slateBOT
boreal umbra
Oh here we go. https://github.com/python/cpython/blob/b664a1df4ee71d3760ab937653b10997081b1794/Lib/collections/__init__.py#L1040
unkempt rock
yep
boreal umbra
Looks like it inherits from something else.
unkempt rock
i was looking into it
the mutable mapping comes from the default mapping
and the mapping from collection
collection from sized, iterable, container
i was wondering it because UserDict also have the dict _missing_ method
boreal umbra
I thought dict didn't have dunder missing
tender hinge
unkempt rock
@boreal umbra well, here is it, then i wondered that the dicts from collections inherits from this dict
unkempt rock
even if the item doesn't exist, _missing_ is not automatically called
hollow crane
well how does it determine if item exists?
does getitem have to return something funky
boreal umbra
@hollow crane it depends on how __getitem__ is defined for a given class. Are you working with lists or dicts? Or something else?
hollow crane
was referring to the example
i guess maybe if you define getitem it assumes you know what you're doing
teal yacht
if it raises KeyError
boreal umbra
The actual lookup table for the dict is in C
teal yacht
wait no i'm wrong
boreal umbra
I don't believe it's possible to interface with the lookup table in pure python so I think you have to inherit from a class that can, like dict
teal yacht
or call it explicitly, meh
boreal umbra
Call it directly?
teal yacht
you can do self.__missing__(key)
woeful quest
!codeblock
fallen slateBOT
paper echo
Why not just use in?
violet cedar
im very very new to python and I was wondering if it is possible to like my teachers messages, basically my schooling system uses an online platform to display our grades. On this platform they can also send messages, in order for them to take attendance we must like the message they send out that day before a respective time on mondays(we don't have school on mondays but they still take attendance) basically I want to know how I can make a bot to like all their messages on monday when they come out. Thank you! 😄
shut raft
Is codeacademy good to start learning python?
forest flicker
python v3.9 recently was released. What are the new features that excite you?
feral cedar
str.removeprefix
safe hedge
Is that really that much of a benefit?
Those funcs are getting touted but they don't exactly seem to add anything new
jovial fable
The stated benefit was clearing up confusion about rstrip and lstrip, and adding common functionality people have been implementing themselves
paper echo
also set operations for dicts
and enabling type annotations like list[int] whereas before you needed typing.List[int]
pep 593 is weird to me, but i guess i see the benefit... that is one thing i do not want to use in my code
although this is an interesting use case https://www.python.org/dev/peps/pep-0593/#lowering-barriers-to-developing-new-typing-constructs
oh yeah, the new parser is very exciting
and zoneinfo is very very nice
quick oar
why is this in the advanced discussion channel
paper echo
@quick oar the channel topic is
Discussion on the use cases, implementation and future of the Python programming language including PEPs, advanced language concepts, new releases, the standard library, and the overall design of the language.
safe hedge
This is about as on-topic as you can be for this channel
raven ridge
I'm by far most happy about the zoneinfo. That's something that's way overdue for Python. The language has had the concept of a timezone for a very long time, but to actually represent a timestamp in some real timezone you've always needed a 3rd party library. This is something that very obviously should be supported by the language itself.
gleaming rover
I'm by far most happy about the zoneinfo. That's something that's way overdue for Python. The language has had the concept of a timezone for a very long time, but to actually represent a timestamp in some real timezone you've always needed a 3rd party library. This is something that very obviously should be supported by the language itself.
@raven ridge it's honestly actually ridiculous and at odds with the batteries-included philosophy
slow wadi
makes a bit more sense when you realize that timezones are a nightmare and can potentially change at any time when some country decides to change it up
raven ridge
nah, that's exactly why I don't want a 3rd party library to be in charge of it. Having Python provide it, and having it hook into the OS's database wherever possible (since it's kept up to date by automated updates), is way better than needing to rely on remembering to pip install an updated 3rd party module to get it.
slow wadi
i don't think it was good, i just think it makes sense how the can got kicked down the road so far
raven ridge
yeah. I get that it's a hard problem to solve, for sure - and that's exactly why I'd rather it be solved in a single consistent way for everybody, in the part of the ecosystem that's best vetted.
slow wadi
now the really hard problem... convincing your boss to upgrade Python
safe hedge
I never really understood the fuss over timezones
Do people actually use location offsets instead of just using UTC with a numerical offset?
slow wadi
humans typically don't use UTC
so if you are trying to communicate a time with a human... you need it in their time zone
safe hedge
Do you?
slow wadi
this sounds like the unicode debate