11 if you ignore binary files
#internals-and-peps
sacred yew
radiant garden
Oops, I was in the same directory as my site-packages, plus there were some pyc binary files.
smoky turret
Heres a fun little puzzle for those who haven't seen it before
!e
print(False == 0 in [False])
fallen slateBOT
You are not allowed to use that command here. Please use the #bot-commands channel instead.
smoky turret
False == 0 is True
brave badger
Precedence rules are tricky
brave badger
ye
sacred yew
huh I thought they would've changed then in py2->3
smoky turret
in python3 bools are a direct subclass of int
sacred yew
oh yeah that
smoky turret
True + True is 2
smoky turret
10 points to whoever can explain the puzzle problem without just googling it directly
brave badger
Explaining it myself is cheating, but if no one does it in an hour or two I'll have a go for the sake of discussion lol
urban quiver
probably binary related ?
dire hull
the only wae i think it would work like this is if the in gets evaluated first
but im stupid so idk lel
smoky turret
a close reading of the operator precedence docs might be revealing
dire hull
hum it says == and in are of the same precedence so i was wrong lol
sacred yew
wait is it comparison chaining
smoky turret
ding ding ding
sacred yew
why is in even considered a comparison
smoky turret
equality, comparison, and "in" can chain
sacred yew
doesn't make sense for chaining at all
smoky turret
so
False == 0 in [False]
is evaluated as
False == 0 and 0 in [False]
my best guess would be for multiple subset testing with sets
sacred yew
when would a in b in c make sense
smoky turret
but honestly idk why
there was probably some rational back in the day, but idk who to ask
radiant garden
a in b in c => element in set in set-of-sets
in which case the current behavior is expected
for when your inner set theorist goes for a walk
safe hedge
You can't have a set-of-sets
>>> set([{1,2,3}, {'a', 'b', 'c'}, {4,5,6}, {'d', 'e', 'f'}])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'set'
peak spoke
Not sure if it makes logical sense with the above check, but you can easily make one with frozensets
smoky turret
^
past crow
:0
ornate trench
wouldn't a in b in c evaluate to (a in b) in c, thus only being true if c contains either True or False, depensing on the value of a in b?
flat gazelle
no
slow isle
no and don't be the guy who does that
ornate trench
oh, okay
safe hedge
So I have been trying to help in #help-peanut on an issue with @functools.wraps. Effectively it seems not to play nicely with discord.py @commands.command and so I figured maybe setting assigned=[] and updated=[] would reduce it to being almost like not having the decorator at all. But it still broke the discord code, and I don't know why?
Doing that should seem to only add __wrapped__ to the function which I would be surprised if it broke something?
Does anyone here know what I'm missing?
radiant fulcrum
Probably because you're trying to wrap a coroutine which needs a bit of diffrent setup for decorators
D.py in this case has a set of utilities todo checks or decorators to go into of commands but yg
safe hedge
Well I figured that, but removing the functools.wraps part made it fine. So I as confused as to why simply adding __wrapped__ broke it
Not my code btw, I was the one trying to help but I got super confused
idle gazelle
I'm the one with the fucky wucky situation/question, if you need anything from me
safe hedge
I guess I just don't understand how __wrapped__ by itself can break something, assuming I'm correct in thinking setting assigned/updated to empty lists makes the rest of update_wrapper redundant
radiant fulcrum
@safe hedge you sure it's not because of slots?
safe hedge
Maybe? But update_wrapper doesn't change that explicity
unkempt rock
#python-discussion for this
heavy bear
k thx
mint forge
I have some questions about the builtins.py is this the right place to ask about them?
visual shadow
I suppose it may depend on the questions themselves, I'd say go for it and let's see.
drowsy mauve
does anyone know if scripting to pull files from a shared drive to local tmp (edit them) and back to shared drive could cause issues?
Im thinking there could be conflict in the double connections created within the script
mint forge
so when we go the definition of the class Exception it shows us this:
class Exception(BaseException): ...
and when we go to the definition of BaseException it shows us this:
class BaseException(object):
args: Tuple[Any, ...]
if sys.version_info < (3,):
message: Any
if sys.version_info >= (3,):
__cause__: Optional[BaseException]
__context__: Optional[BaseException]
__suppress_context__: bool
__traceback__: Optional[TracebackType]
def __init__(self, *args: object) -> None: ...
def __str__(self) -> str: ...
def __repr__(self) -> str: ...
if sys.version_info < (3,):
def __getitem__(self, i: int) -> Any: ...
def __getslice__(self, start: int, stop: int) -> Tuple[Any, ...]: ...
if sys.version_info >= (3,):
def with_traceback(self: _TBE, tb: Optional[TracebackType]) -> _TBE: ...
when we inherit from Exception how does it make the class an exception when there is only this much?
spark magnet
@mint forge builtins.py is not the implementation of those classes. Where did you find it?
mint forge
i went to the defintion
peak spoke
That's a stub file for typing, not the source
mint forge
@spark magnet Where my cursor is
spark magnet
@mint forge inheriting from BaseException is what makes something an exception.
peak spoke
Not sure exactly where, but exceptions will be defined somewhere in the C source
spark magnet
@mint forge ok, VSCode is showing you the type definition file it is reading, not the implementation.
mint forge
ohhh, I wanted to see how they are made an exceptoin
spark magnet
anything inheriting from BaseException is an exception. maybe we need to talk about what "be an exception" means.
spark magnet
@mint forge I think literally when code wants to know if someething is an exception, it does isinstance(thing, BaseException)
mint forge
hmm
spark magnet
Perhaps there is some behavior you are thinking of when you say, "be an exception"?
mint forge
could u explain that as well?
but my main question was how does BaseException make something an exception
but u answered that
peak spoke
Exceptions have some attributes python expects which get set with BaseException, to ensure you're not raising whatever python pretty much does the isinstance nedbat mentioned above and if it's not an exception or an instance of one you get a type error https://github.com/python/cpython/blob/master/Python/ceval.c#L4448
grave jolt
No, CPython is not Cython
mint forge
ye lol
grave jolt
CPython is the most popular implementation of Python
peak spoke
cpython is the reference implementation you're most probably using, the code I linked is what actually runs when you raise an exception
mint forge
I see
the language itself is really fabulous
the way it's compiled (even though i don't understand how it is compiled) it's really WOWing
teal yacht
@mint forge the code you're looking at is called a "type stub"
look at the file extension
mint forge
what file extension the one Numerlor linked?
teal yacht
no, the one where you found #internals-and-peps message
mint forge
oh, what about it
teal yacht
it's not .py
mint forge
it is
teal yacht
no, it is .pyi
teal yacht
pyi files are for types only, the implementation itself can be located in some other place
peak spoke
Stubs are for editors to know the types of things they can't infer
mint forge
I see
lyric marsh
hello
was just setting up pycharm if u look my file name is scratch.py but why is it not printing
it
it is printing test
any idea
brave badger
Hello @lyric marsh, this is not really the appropriate channel to get help from for such things. Feel free to check out #❓｜how-to-get-help though, you'll get better answers there.
lyric marsh
ok
inland juniper
can anybody tell me how can i make programmes like face recognition etc with tensor flow?
unkempt rock
im making an app for talking and chatting like mix beetween discord twitter and skype
grave jolt
@unkempt rock @inland juniper This is not a help channel, this is a channel for meta discussions about Python. For help, see #❓｜how-to-get-help.
weak solar
Hello. So quick question. I've seen many packages use [] in their installs for custom downloads, so for example, discord.py can be downloaded via pip install discord.py, but if you want to download the voice packages, you'd run pip install discord.py[voice]. I currently have an REST API, and I am about to re-write all of it in order to be compatible with async, so if users want to download it, I want them to run pip install PACKAGENAME[async]. My question is, how would I design and upload a package (or modify the setup.py file) in order to achieve this? Thank you for the help.
weak solar
Alrighty, thanks.
unkempt rock
Does someone have experience with unittest.mock ? I've been stuck with this package for hours now... You would definitely help me out big time!
mint forge
#unit-testing @unkempt rock
unkempt rock
#unit-testing @unkempt rock
@mint forge Sadly nobody is available to help 😦
mint forge
that's not the reason to ask in this channel
unkempt rock
And everyone who offered has no idea about unittest
mystic cargo
sad to hear that, but this is not the right channel for it
unkempt rock
that's not the reason to ask in this channel
@mint forge I thought this topic might be too advanced?
mint forge
no
unkempt rock
My bad
mystic cargo
"advanced: discussion channel is for discussion about THE language
unkempt rock
@unkempt rock It's ok, this channel is about discussion about the language itself
Ahh.... sorry I'm new here
stark pulsar
I worked with unittest and was really a pain. Saddly I didn't do anything with mock
unkempt rock
I understood it incorrectly
mint forge
@unkempt rock Ves Sums it up quite well
#community-meta message
true hollow
This bytecode here seems interesting: https://github.com/python/cpython/blob/e6b8c5263a7fcf5b95d0fd4c900e5949eeb6630d/Python/frozen.c#L17
Where did they get info to make it valid bytecode?
stone field
how do you mean? you can get bytecode of anything from the code object in CPython
true hollow
i mean
flat gazelle
looks like they just compile a simple module and put it into the source
true hollow
where did they get info to make the magic header, bytecode and etc
because making an array of bytecode is just crazy
stone field
It's not though?
Everything in CPython is bytecode, when it comes down to it
>>> compile('print("hello, world!")', 'filename', 'exec')
<code object <module> at 0x0000028B0013F710, file "filename", line 1>
>>> _.co_code
b'e\x00d\x00\x83\x01\x01\x00d\x01S\x00'```Basically you don't do it by hand.
true hollow
interesting
visual shadow
I think that's the key.
stone field
Doing CPython bytecode by hand is an interesting exercise, since it's fairly well documented.
peak spoke
It is mentioned in the comment that you should freeze and copy on changes
grave jolt
@stone field There's also a hax module on PyPi, it allows you to write bytecode in a more readable way.
from hax import *
@hax
def add_tuples(a, b):
LOAD_FAST("a")
UNPACK_SEQUENCE(2)
ROT_TWO()
LOAD_FAST("b")
UNPACK_SEQUENCE(2)
ROT_THREE()
BINARY_ADD()
ROT_THREE()
BINARY_ADD()
ROT_TWO()
BUILD_TUPLE(2)
RETURN_VALUE()
it adds two 2-element iterables elementwise (like
add_tuples((1, 2), (200, 100)) == (201, 102)), and it turns out to be faster than any built-in way of doing that
true hollow
that's cool
grave jolt
But doing so in a real production system is a sure way to shoot yourself in the foot.
Probably never a good idSegmentation fault (core dumped)
stone field
@grave jolt That's interesting, thank you.
final geode
Heh, my first time using Python Discord ever and I stumble into a conversation about one of my packages. Neat.
grave jolt
peak spoke
How accurate are the benchmarks here? https://docs.python.org/3/whatsnew/3.9.html#optimizations was looking through it and the read_boundmethod seems like a fairly big regression
local tests seem to agree, what could be the reason for it? Was looking through some PEPs like 590 earlier to speed up calls but couldn't find benchmarks for that
wide shuttle
There was a small discussion on twitter about it, including that to get closer to the subinterpreters being implemented, the language summit decided that it was okay to trade in some speed
Twitter
peak spoke
The access is consistently around 10ns slower locally, but it is trivial to go around if it actually was an issue in something that looped a lot, mainly just curious
stable path
I have a embedded system in my office, i created some for x86_64 with python 3.7. now i want do some testing on armv7 with python 3.7. i really need to cross compile python 3.7.9 to armv7. I tried many online ways to do so and failed. I think i didn't understand the concept of as of too much flags, any help will really appreciated. There is no internet in my office so i cant experiment. Is there any good docs on cross compiling python to armv7. And can i use binary of python compiled by rpi3b, i didn't try this? Please help.
grave jolt
it will hopefully get better with ParamSpec
zenith topaz
Yeah, ParamSpec seems to be specifically to solve that problem.
Already available in 3.10 alpha
shadow musk
Hi does anyone know how to identify all the # pylint: disable=XXX comments in a script that are redundant? i.e. if you remove them then pylint still passes since they weren't actually suppressing a warning / error
somber halo
you could use grep or similar in your repo.
shadow musk
@somber halo how would that help me identify whether a comment is redundant?
somber halo
Saw now your restriction on redundant pragmas... it wouldn't.
I'm not aware of a fully automatic approach for that that's available... but you could devise one
1- find those lines with those pragmas; 2- have some code that modifies that Python code that removes a pragma at a time; 3- Run pylint to see if it passes; 4- If it doesn't, keep track of that "necessary" pragma.
just something that appeared at the top of my head
spark magnet
@shadow musk pylint has a message: "unnecessary-suppression"
magic python
what was that recent PEP that had macros or something?
something about it being similar to C# and LINQ iirc... though I've never touched either of those
stray hollow
not sure if this is the place for this, but i want to see this changed for ease of use. i don't see any practical use of this being in the library. im talking about having to use a .join statement when using a for loop in a tupple. the photo below here shows what happens when you use/dont use .join. Another reason that i think needing to use .join with a for loop in a tupple is because it removes readability(refer to code under the photo for an example
flat gazelle
you want str([1,2,32]) to be 1232?
stray hollow
f"Not using a .join statement: {[i.mention for i in Reverse(list)]}"
f"Using a `.join` statement: {''.join([i.mention for i in Reverse(list)])}"
feral cedar
you can use a different string other than "" to separate them
peak spoke
what if I wanted the list and not a string?
feral cedar
did you write you own Reverse function?
flat gazelle
raku and lisp use (1 2 32)
feral cedar
there's a builtin reversed
flat gazelle
why not reversed?
stray hollow
ik, my function uses .reverse() on a list and returns the reversed list
flat gazelle
!e
for x in reversed([12,14,11]):
print(x, end=' ')
@flat gazelle :white_check_mark: Your eval job has completed with return code 0.
11 14 12
it already exists
wide shuttle
!e
print(*reversed([12,14,11]))
@wide shuttle :white_check_mark: Your eval job has completed with return code 0.
11 14 12
Anyway, when I print a tuple, I want to see that it's a tuple and I want to make sure it uses the repr for the elements so I know what the elements are
flat gazelle
anyway, back to the original point. What do you think str([1,2,3]) should return
wide shuttle
Your proposal trades that in for a little bit of convenience in some situations
stray hollow
fair
flat gazelle
honestly, I would not exactly despise str of list being ' '.join(map(str, l)) and repr being what we have right now
kinda like apl
peak spoke
str can be expensive in some cases so I don't really see that as a good use, and repr won't make as much sense
grave jolt
@flat gazelle you mean, str(*xs) == "".join(map(str, xs))?
that would make sense for me tbh
flat gazelle
no, I mean str([1,2,3]) == '1 2 3'
varargs in str would not be terrible either
but there is ambiguity with existing constructors
grave jolt
...that would probably add overhead to the current usage (when using only one argument)
flat gazelle
!e
print(str(b'abcdef', 'ascii'))
fallen slateBOT
@flat gazelle :white_check_mark: Your eval job has completed with return code 0.
abcdef
ooh
well, I didn't know about this
why is this not a staticmethod or something?
also, there's bytes.decode(), right?
flat gazelle
yeah, but this works on other things too afaik
such as a bytearray
though that also has a decode
idk why its there then
grave jolt
well, it's probably the same thing as with int (it's int as opposed to int.parse)
unkempt rock
what do you guys think about the dunder prefix for private class variables?
self.__x
molten onyx
if you don't intend on anyone inheriting the class I think it's unnecessary
Anyone happen to know what PyCF_IGNORE_COOKIE does
is it a legacy flag? is it an easter egg?
boreal umbra
@unkempt rock I've never needed the name mangling so I don't use it
molten onyx
I have also figured out the extremely strange way the new = "format specifier" is implemented
When you have a string like f'{NAME = :FORMAT_SPEC}' it actually gets transformed into f'NAME = {NAME:FORMAT_SPEC}'
>>> ast.dump(ast.parse("f'text {x:5}'"))
Module(body=[Expr(value=JoinedStr(values=[Constant(value='text ', kind=None), FormattedValue(value=Name(id='x', ctx=Load()), conversion=-1, format_spec=JoinedStr(values=[Constant(value='5', kind=None)]))]))], type_ignores=[])
>>> ast.dump(ast.parse("f'text {x = :5}'"))
Module(body=[Expr(value=JoinedStr(values=[Constant(value='text x = ', kind=None), FormattedValue(value=Name(id='x', ctx=Load()), conversion=-1, format_spec=JoinedStr(values=[Constant(value='5', kind=None)]))]))], type_ignores=[])```
deft pagoda
In [2]: ppast('f"text {x:4}"')
Expr
╰──JoinedStr
├──Constant
│ ╰──'text '
╰──FormattedValue
├──Name
│ ├──x
│ ╰──Load
├──-1
╰──JoinedStr
╰──Constant
╰──'4'
In [3]: ppast('f"text {x =: 4}"')
Expr
╰──JoinedStr
├──Constant
│ ╰──'text x ='
╰──FormattedValue
├──Name
│ ├──x
│ ╰──Load
├──-1
╰──JoinedStr
╰──Constant
╰──' 4'
This is also seemingly true at bytecode-level
The only difference between f'{x=}' and f'x = {x}' is that for the former, repr is explicitly called
>>> dis.dis("f'{x=}'")
1 0 LOAD_CONST 0 ('x=')
2 LOAD_NAME 0 (x)
4 FORMAT_VALUE 2 (repr)
6 BUILD_STRING 2
8 RETURN_VALUE
>>> dis.dis("f'x={x}'")
1 0 LOAD_CONST 0 ('x=')
2 LOAD_NAME 0 (x)
4 FORMAT_VALUE 0
6 BUILD_STRING 2
8 RETURN_VALUE
>>>
>>>
>>> class Dummy:
... def __repr__(self): return 'repr'
... def __str__(self): return 'str'
...
>>> f'Dummy() = {Dummy()}'
'Dummy() = str'
>>> f'{Dummy() = }'
'Dummy() = repr'```This also seemingly means you cannot tell whether or not the original code used the = or not
well actually you can 
because there's the repr() instruction without a format spec
rustic stag
How is that extremely strange?
cloud crypt
I didn’t know that f"{expr = format_spec}" used repr() explicitly; well, not like I really used it, anyways
molten onyx
because it's a compile-time transformation of the string
peak spoke
Makes sense as the contents may change after compilation and accessing it probably wouldn't be nice
rustic stag
f-strings are all about compile-time string transformations!
true ridge
This also seemingly means you cannot tell whether or not the original code used the
=or not
@molten onyx Indeed. They are indistinguishable, that is the reason whyast.unparseunparses both of them in the same way
>>> ast.unparse(ast.parse("f'{x = }'"))
"f'x = {x!r}'"
ast.unparse also can't tell where parens were in an expression when you unparse it (except for the string-ized expression, of course):
ast.unparse(ast.parse("f'{(x) = }'"))
"f'(x) = {x!r}'"
grave jolt
Well, that makes sense
((((x*y))))+z is the same as x*y+z
and () aren't stored in the ast because they don't provide any additional information
unkempt rock
I made an asynchronous version of map(), take a look: https://github.com/joumaico/asyncmap,
boreal umbra
@unkempt rock interesting concept. I'm not too familiar with async but shouldn't it be async def so that the function doesn't require its own event loop?
radiant fulcrum
i dont exactly understand the point of in respect to being 'async', i would see it a interesting or useful concept if the function you map to the list is async itself
but the 'blocking' function doesnt make a massive sense considering your application of map() having generally really rather lightweight tasks
spice pecan
a version of map that applies stuff to an async iterable would also be nice
radiant fulcrum
that would probably defeat the purpose of the async iterable though, atleast in my experience
generally its used if you have something that requests something or gets it as its being iterated over asynchronously which to map it would end up just listing it then mapping the result
spice pecan
fair point
unkempt rock
It's an inspiration from the newest method in Python3.9 https://docs.python.org/3/library/asyncio-task.html#running-in-threads
radiant fulcrum
thats just a helper method for run_in_executor
magic python
i'm a bit confused about what overload is doing here: https://github.com/pandas-dev/pandas/blob/master/pandas/core/reshape/concat.py#L40
from here: https://docs.python.org/3/library/typing.html#typing.overload it seems as though the only reason they're there is for type checking, at first glance I thought that there was some different functionality declared for differing inputs or something.
elder lodge
how do you guys differentiate between a database model like sqlalchemy model, and a non database model like pydantic models. theyre both models but are used differently. do you store them in your models.py or do you classify them differently?
gleaming rover
how do you guys differentiate between a database model like sqlalchemy model, and a non database model like pydantic models. theyre both models but are used differently. do you store them in your models.py or do you classify them differently?
@elder lodge did you mean to post that in #web-development or #databases? that seems like a Django question.
raven ridge
@magic python 100% for type hinting, yes. When evaluated as Python instead of parsed into an AST, each of those definitions overwrites the last.
grave jolt
For me, a 'vertical slice' model makes more sense, i.e. instead of having a models.py file, having a module for each domain type, like users/, posts/, comments/ etc.
raven ridge
The point of the overload decorator is to allow you to express dependencies between parameters. "Either both of these must be provided or neither", or things like that.
elder lodge
@grave jolt i have that too. typically like, users/models.py users/services.py
is that bad too?
desert peak
that seems like a lot of duplication
why not make a root module like orm/ where you put models and such?
or serialization/
grave jolt
yes
and when you need to change something, you typically change a bunch of stuff related to a particular domain (e.g. users), not a particular layer (e.g. models)
it's just a different kind of layout
elder lodge
Ayyy u right
grave jolt
I'm not saying it's the only way to do it
maybe there's some kind of convention in Django
tacit hawk
That vertical slices can be done with django apps (python packages). It's recommended to split the whole system in several apps. So that's a good advice.
charred wagon
Is abc only designed to be used with classes that are meant to be instantiated?
Like, you can make a static abstract method, but that doesn't actually get checked if the concrete class isn't instantiated
Therefore, if your concrete class consists of only static methods (i.e. there's no need to instantiate it), then abstract methods basically do nothing for you
smoky turret
I suppose you could concievably have just a big function collection class that still has to abide by some interface, but in practice I've never seen anything like that
charred wagon
That's kind of what I have, except it's not a big class
smoky turret
unnecessary qualifier on my part, doesnt have to be
charred wagon
I have a couple concrete classes that share the same interface
smoky turret
to answer you question, it will work, there shouldnt be any problems
charred wagon
They just happen to have no state - they used to
teal yacht
You can check the __abstractmethods__ attribute
It's kinda ugly, but it's one way to check if a class implements all the required methods without instantiating it
charred wagon
The only way I see to make something work defining __init_subclass__ to check
But that's clunky
It's not as nice as a metaclass + decorating some functions and off you go
teal yacht
You don't want all abstract classes to be checked at class definition
Otherwise you can't make partially implemented ABCs
charred wagon
That's a good point
Though from a more general perspective IDK how to work around that. Thankfully I wouldn't care in my specific case.
teal yacht
However, you could make a class that checks __abstractmethods__ in a metaclass or a decorator
decorator would be the easiest
smoky turret
Do you have enough volume of code or lots of teammates that would make an abstract class worthwhile?
teal yacht
def check_abc(cls):
if cls.__abstractmethods__:
raise ...
return cls```No. I could just code on an assumption that the classes will have the interface I expected
smoky turret
thats usually the python way
charred wagon
But it's "nicer" if it would fail quickly upon definition rather than when I try to access an attribute or call a missing function
It's not really nicer, since I have to actually write some metaclass or whatever myself to do this
if abc could already do it then i'd use it
smoky turret
I mean, you could just instantiate to check
even if you dont need to instantiate for normal use
charred wagon
That's true, feels kinda hacky though. I'd rather just get rid of the abc code
smoky turret
way less hacky than doing it yourself
charred wagon
Well, whether metaprogramming is hacky is contentious. If it's used for a sane purpose and has a nice interface, like abc itself, then it's alright
Hi starlord
In that case I wouldn't call it hacky, just a lot of work for little gain
charred wagon
Hey that's pretty cool. Simpler than I imagined
smoky turret
Nice
teal yacht
in __new__, you can make the check
charred wagon
But isn't new for new instances rather than for when the class is defined?
charred wagon
ah right, I understand
How would that be different than __init_subclass__
(without a metaclass)
teal yacht
I was afraid there would be issues with the order in which you specify the inheritance there, but I can't think of a case where it could break (other than overriding the method entirely in a way that breaks the MRO)
Guess it should be fine
I personally think the decorator solution is overall better tho
Simpler to understand, and I think decorators stand out more, syntactically speaking
So you know at a first glance which classes are checked, and which are not
charred wagon
I am already using __init_subclass__ anyway cause I need to have an abstract class attribute
Cause static abstract properties don't work if accessed via the type rather than the instance
teal yacht
now I'm curious
charred wagon
And by don't work, I mean it returns the property object rather than the actual value
teal yacht
from abc import ABC, abstractmethod
def implements_abc(cls):
if cls.__abstractmethods__:
raise NotImplementedError(f"{cls} must implement {list(cls.__abstractmethods__)}")
return cls
class T(ABC):
@staticmethod
@property
@abstractmethod
def foo():
...
class U(T):
...
@implements_abc
class W(U):
foo = 3
```works apparentlyW.foo returns 3
same with W().foo
charred wagon
Okay I misremembered
What happened was it returns the property object when the child class doesn't define it, rather than raising an exception
But of course the decorator takes care of this now
teal yacht
I think it's a matter of properly ordering the decorators (property, abstractmethod, staticmethod), I am managing to have it crash when it is instantiated without foo defined
(but either way we go back to your original problem that it needs to be instantiated to be checked, unless we use that implements_abc)
charred wagon
What works for me is py @property @staticmethod @abstractmethod
classmethod also works
charred wagon
Apparently __abstractmethods__ doesn't exist yet at the time __init_subclass__ is called
dreamy trail
wary galleon
ok, so ive got it to pass links to a browser, but i need a way to store them similar to where they are going...
smoky turret
sacred yew
decorators
low lagoon
@wary galleon #❓｜how-to-get-help
lunar glacier
How do classes work
mint forge
Class's are objects
unkempt rock
wdym
lunar glacier
Ok
fallen slateBOT
lunar glacier
or just in general
@unkempt rock general
unkempt rock
class Apple:
def __init__(self, color):
self.color = color
myapple = Apple("green")
print(myapple.color) # prints 'green'
you can add methods to it
lunar glacier
Ok
unkempt rock
better to read an online tutorial
lunar glacier
True
signal tide
is there any point to using __call__ if all I'm doing is accessing/mutating the attributes?
like I've got
class Foo():
def __init__(self):
...
def __call__(self):
return self```but that feels wrong/redundant
teal yacht
__call__ is for when you want to treat the - ^ yeah that
signal tide
the what lol
smoky turret
Theres a difference between a class and its instances
signal tide
I was just trying to figure out how to create an instance of the class
so ya it's a improper use
teal yacht
__call__ is not for instantiating objects
signal tide
let me rephrase
teal yacht
class T:
...
x = T() # T.__init__
x() # T.__call__```I was trying to figure out how to create an instance of a class and call that instance's attributes
teal yacht
call an attribute as in call a function that you have stored as an attribute ?
smoky turret
what do you mean by call an attribute?
like T.attr() ?
signal tide
armour_list, weapon_list, bag_list = Inv().inventory_setup()``` def inventory_setup(self):
x = self.NoItem() # Placeholder for a .name attribute if no item
armour_list = []
for item in self.armour_list:
if item is None:
item = x
armour_list.append(item)
...```honestly the whole thing's kind of convoluted but I also had:
class NoItem:
def __init__(self):
self.name = None
def __call__(self):
return self.name```Inv is a class right?
signal tide
Inv = Inventory() ye
smoky turret
ok, so Inventory is a class
Inv (lowercase it) is an instance
you can just do inv.inventory_setup()
signal tide
ya I got that now
slate rose
Is there any way to use bitset in python?
flat gazelle
you could make it with the int type, but why not use set instead? I guess if you really needed the space efficiency, it would have merit
raven ridge
https://m.youtube.com/watch?v=tGAngdU_8D8 was a pycon talk about implementing a bitset, and what tradeoffs it has vs a regular set
grave jolt
Does anyone have a favourite misconception/misunderstanding about Python?
Not something like misunderstanding indentation, but more high-level ideas
mint forge
Uses of @staticmethod @grave jolt does that count?
smoky turret
I mean, the age old range(len(...)) idiom never goes away
mint forge
not misunderstanding but it's usage
flat gazelle
Overuse of classes is pretty common
smoky turret
thinking assignment copys data is a common one
grand crag
the misunderstanding of super, I think.
cloud crypt
In python, we don't really operate on values, since everything is pretty much a reference
smoky turret
uhh, we operate exclusively on values?
cloud crypt
let vector = vec![1, 2, 3];
let maybe_copy = vector;
println!("{:?}", vector); // error: value has moved```
```python
vector = [1, 2, 3]
maybe_copy = vector
vector.append(4)
maybe_copy.append(5)
assert vector == [1, 2, 3, 4, 5]``` everything is a reference to this vector in pythonsame with objects
smoky turret
correct, variables are references, but to touch a reference is to immediately dereference it
the only thing that could be said to be an operation on a reference itself is assignment
everything else immediately dereferences it
grave jolt
I recently discovered that browsing supposed interview question aggregates is a very entertaining activity. Nowhere else can you find bizarre statements like "__init__ is a method that allocates memory" and "python is interpreted line-by-line".
smoky turret
to be fair, "python is interpreted line by line" is a pretty easy leap to make given how "interpreted languages" is usually described
- python is usually described as just an interpreted language, when the real answer is a bit more complicated
grand crag
if it was interpreted line by line, many code would be incorrect. I think that it is great to explain a bit how it works, because quite no langages can be interpreted line by line.
"
__init__is a method that allocates memory"
I think that many people are seeing__init__as the__new__method, because they never heard of__new__
flat gazelle
Bash, batch and such do afaik go line by line, as does forth (sometimes)
grand crag
there is no scope nor functions in bash, batch, ... ? Right ?
grand crag
oh ok
solar grove
would this be the right channel to have a jovial discussion and have my old code ripped apart and discussed?
its an interpreter for a subset of python written in c++
cloud crypt
line-by-line is a weird way to describe that
grave jolt
@solar grove Probably not this channel... The closest match would be #c-extensions, but not really. It's more of a C++ question rather than a Python question
grave jolt
line-by-line is a weird way to describe tha
I'm not kidding about "line-by-line"
https://www.interviewbit.com/python-interview-questions/ -> Question 4
An Interpreted language executes its statements line by line. Languages such as Python, Javascript, R, PHP and Ruby are prime examples of Interpreted languages. Programs written in an interpreted language runs directly from the source code, with no intermediary compilation step
It's also funny how later on it explains something about bytecode being an intermediate representation on Python code.
grand crag
he wants to discuss about python, not C++
unkempt rock
Try both
solar grove
ah thanks been a while but i guess that would be where to go too
cloud crypt
An Interpreted language executes its statements line by line. Languages such as Python, Javascript, R, PHP and Ruby are prime examples of Interpreted languages. Programs written in an interpreted language runs directly from the source code, with no intermediary compilation step
oh now THAT is incorrect
solar grove
been off discord for a few years
cloud crypt
interpreted languages get compiled to byte-code or JIT compiled sometimes
solar grove
yea by an interpreter i wrote one lol its college cpode though
cloud crypt
so there is intermediary compilation step
solar grove
i want it ripped apart lol
grave jolt
Well,
Programs written in an interpreted language runs directly from the source code,
this part is wrong
-- sorry, misread @cloud crypt's message
solar grove
there can be any number of intermediary steps it depends on the toolchain
grand crag
line-by-line is a weird way to describe that
I think the confusion comes from codes like :
print(5)
func()
``` This codes run well up to `func`, because it is not defined.
I think the confusion comes from the fact that everything in python is dynamic, and you can create all types of objects at runtime.i write c mostly bu i just love languages in general
python is interesting
y
I think the confusion comes from codes like :
print(5) func() ``` This codes run well up to `func`, because it is not defined. I think the confusion comes from the fact that everything in python is dynamic, and you can create all types of objects at runtime.
@grand crag python is ultimatey c code
you need to define types
err well not exactl;y that lol
grand crag
hmm... you can create all at runtime if you want
solar grove
if you want...
ive thought about combining compilation with an interpreter
best of both worlds type stuff
still have to whiteboard it
grand crag
Python cannot be sure that a variable doesn't exist at compile time. The bytecode must be interpreted to know it the code is really correct or not.
cloud crypt
JIT compilation is pretty much combining compilation and interpreting
smoky turret
This talk really gets into the weeds of python, interpretation vs compilations, JITs, etc https://www.youtube.com/watch?v=I4nkgJdVZFA
its a good one
grand crag
thanks for the link !
smoky turret
basically yeah, but its pretty nuanced
prime estuary
Not really. The problem is that you can't use 99% of classical optimisation, since the language is so dynamic.
grand crag
python is slow because it cannot be very optimised
prime estuary
For example you can modify the builtins module, and then suddenly len() could do anything.
solar grove
https://github.com/ezquire/Interpreter this is college code but its 80% mine lol be gentle
grand crag
and there are some many steps during the creation of an object, or when you set / get an attribute...
smoky turret
taken from the talk, the fact that cpython allows c-extensions makes it impossible for cpython to be fully JIT'd
solar grove
yall?
smoky turret
as opposed to somthing like pypy, which does use a JIT, and is faster than cpython
solar grove
taken from the talk, the fact that cpython allows c-extensions makes it impossible for cpython to be fully JIT'd
@smoky turret nothing is "impossible" we have magic powers lol
smoky turret
naw, c-extensions make it impossible
solar grove
how would we do that?
prime estuary
The problem is that the C-API exposes a lot of internals. For example it gives you direct access to the PyObject pointers in lists, which means CPython couldn't for example optimise integer/float lists by just storing an array. PyPy does on the other hand.
solar grove
why are they lists of pointers? is there a pep for this?
smoky turret
the indirection makes everything easier
I imagine its been lists of pointers since probably python 1
solar grove
why not a graph? think like graph ql
prime estuary
Because they can hold any object type.
prime estuary
Ints, floats etc are all regular full objects.
solar grove
for sure
that makes more sense now thanks
so are they template objects? i thought it was a c interpreter lol still new to python
prime estuary
Pure C.
prime estuary
The structure is roughly like this:
struct PyObject {
long refCount;
PyTypeObject *type;
...
}
The type is a pointer to the type/class object, which has function pointers for all the possible operations you can perform.
solar grove
lol i lightweight know @boreal mantle_ through a program we did together to get hired
whoops sorry dude wrong person
@peak heron is who i meant lol
what is ref count for ? instantiation?
solar grove
ok so number of instances
prime estuary
Yep. When it hits zero, it can be freed.
grand crag
just like smart ptr
solar grove
i have auto all over that code base i shared lol
when i discovered safe ptrs i was so happy lol
smoky turret
I had to do a lot of C in uni
solar grove
errr not auto lol
smoky turret
Im quite glad those days are behind me
solar grove
dude i have been coding in one form or another since i was 8 lol i live in C and i love ARM assembly (need to brush up)
i got my "professional" education start with C++ though at American River College in Sacramento, CA
i was an early adopter of gmail and facebook yadda yadda all the shit lol its my hobby and now its my career finally lol
so im happy to have this discord, i need an outlet, im driving my gf crazy lol
i used to run a phishing scam on habbo hotel if anyone remembers that game lol now I'm dating myself
hi yall nice to meet you all
smoky turret
hell of a story dude, nice to meet ya
solar grove
you too! i have lots of stories lol been going to private school my whole life because my parents didn't want to medicate me lol I was doing calculus in 8th grade too and dissecting frogs and shit lol
fun school
i guess you call that privilege lol
i'm currently working with a senior engineer at Apple to build a wall mounted app controlled laser pointer cat toy that uses computer vision to map your room so you can control it from an app. I like to build shit lol
shhh I didn't tell you that lol
no stealsies
smoky turret
lol
solar grove
im only half joking lol that's my IP lol
well ours
claiming it now lol
we might actually need some help, I think all our servo code is python so you all might be a good resource
we're using an rpi to control and arduino and a servo then linking the app data to AWS with a small node.js server running on the rpi but I think the servo code is python, still gotta work out the details
signal tide
@solar grove iCat?
solar grove
bigglesworth 😉
signal tide
That doesn't sound very apple
solar grove
well its not an apple product lol
signal tide
Very ot thats what they want you to believe
gloomy rain
@solar grove Refrain from posting memes/gifs in on-topic channels, please.
solar grove
noted
gloomy rain
Thanks for understanding.
solar grove
haha its all good I have been on forums for a very long time
I just want to know where I stand
I skimmed the rules but I read docs all day so I didn't quite get everything lol
gloomy rain
No worries.
solar grove
so why isn't python built on assembly?
if you can build it on C you can build it on assembly
C is a nice interface and all but you can write an interpreter that interprets stuff other than C code
I know its overhead and it needs to be built but its possible
thoughts?
i'm envisioning a bare metal assembly scripting language
if you want a gentle introduction to assembly this is my professor's book available for free https://bob.cs.sonoma.edu/IntroCompOrg-RPi/intro-co-rpi.html
radiant fulcrum
pretty simple: Assembly isnt cross chipset
undone hare
Because making the whole python interpreter and compiler in ASM would be a nightmare
radiant fulcrum
its chipset specific which means a different set of instructions for Intel, AMD and ARM based CPUs
solar grove
yea I know but you could make it
undone hare
Working with lower interfaces takes way more time and effort
radiant fulcrum
not to mention the MASSIVE amount of time it would take and the fact it would be nearly impossible to maintain
radiant fulcrum
it pretty much is
its limited by practicallity
in the fact that its not practical
gloomy rain
I mean, this is basically how native compiled languages work. You need to implement a different compiler for every architecture.
solar grove
like most things lol
I wish everything was practical
I mean, this is basically how native compiled languages work. You need to implement a different compiler for every architecture.
@gloomy rain that's just tradition
gloomy rain
How so?
solar grove
its just how the old beards did it
I was taught by them
my first C++ prof wrote stuff for the std lib and lived in Sac and built guitars
he was a carpenter by trade
gloomy rain
I'm not sure exactly what you're referring to. How else would you make a native binary compiler?
gloomy rain
Whiteboard how to make a native binary compiler without having to implement a different one for each architecture?
flat gazelle
it is unviable to maintain 3 separate implementation of a program
solar grove
why?
gloomy rain
@solar grove That doesn't make sense. Each architecture has its own instruction set. You can't get around that.
solar grove
yea cause it hasn't been built yet
sorry for being crass but how tf do you think your iphone got built?
gloomy rain
No, each architecture has been built and they all have different instruction sets.
In a factory?
solar grove
In a garage
flat gazelle
because no one can effectively contribute since everything needs to be written in triplicate, and all contributors must know all of the instruction sets, makes it harder to test, especially since you cannot easily emulate different insn sets. The common way to do this is to write something that already has the various compilers, such as C
gloomy rain
Apple was already pretty big by the time they designed the smartphone, so it probably wasn't done in a garage.
solar grove
right I know all that lol
because no one can effectively contribute since everything needs to be written in triplicate, and all contributes must know all of the instruction sets, makes it harder to test, especially since you cannot easily emulate different insn sets. The common way to do this is to write something that already has the various compilers, such as C
@flat gazelle
uhh your fucking motherboards were built in a garage
like literally digital transitors
gloomy rain
Still don't see what your point is.
solar grove
my point is stop saying its impossible
flat gazelle
you cannot write assembly portably, that is indeed impossible
solar grove
no its not
gloomy rain
I mean, if you propose something that's logically impossible, no innovation in the world will change that.
flat gazelle
what you do is you make a language that then gets converted into each of the relevant instruction sets
which is C
this is a solved problem
gloomy rain
I don't need to think about it.
Yeah, I mean, the C language abstracts away the hardware-specific instruction sets, but you still need one C compiler implementation per architecture.
solar grove
look I just got my degree in 2019 I'm inexperienced with all the new shit
flat gazelle
C is not new shit
solar grove
It has new features
gloomy rain
It's literally 50 years old at this point.
solar grove
so does C++
flat gazelle
CPython afaik is written in C99
gloomy rain
C I think barely changed for a long time. C++ has new features, but that's not really relevant to what we're discussing.
solar grove
see that is something I didn't know
CPython afaik is written in C99
@flat gazelle
I have read Khernigan and Ritchie
I have a BS in Comp Sci from Sonoma State for god's sake
I know my shit
undone hare
C is 48 years old
flat gazelle
C does get new standard features, but adoption is slow and they generally do not affect the core language that much
I am not sure if there is actually a fully compliant C11 compiler out there
solar grove
"If you can't code it in assembly, you don't understand it" - Mark Gondree, Sonoma State
flat gazelle
if you can't code in python, you don't understand it - lakmatiol, python discord
flat gazelle
also, ASM is much worse a language than C for actually getting code done
undone hare
I don't see how this quote is really revelant tbh
radiant garden
Seems like an oddly backwards view to take. Surely one can make a good program without having to worry about its low-level implications, especially if most of the program consists of "business logic"?
solar grove
its about understanding the fundamentals
undone hare
So if you import a library in python, you should be able to recreate it in ASM?
It doesn't make much sense
flat gazelle
also, there is interpreted language that runs on embedded systems. FORTH
gloomy rain
It certainly is advantageous to have a conceptual understanding of the abstraction stack down to the bare metal. I wouldn't say that it's absolutely required, though.
radiant garden
Generally speaking, abstractions are for the benefit of the developer. The scope of one's project is relevant in dictating its level of abstraction.
solar grove
Well during a hackathon at Sonoma my homie nathan wrote an extensible version of brainfuck and an interpreter for it
radiant garden
Well, yes. It's dead easy to implement.
solar grove
I abstract all my knowledge
yea good for a hackathon
I'm also all about DX
get me going
gloomy rain
@radiant garden It would be a waste of time to concern yourself with the implementation details of a given abstraction as you are working on an application using that abstraction. But, it's useful to understand the whole abstraction stack on a conceptual level, and especially the abstraction layer below the one you're working on.
solar grove
I want a toolbox and I want to play
flat gazelle
y'know whats even more interesting for mathematical curiosity about how simple computation really is, rule110. It is turing complete and thus is capable of all computation, but so rudimentary that you can write its interpreter in about <10 short lines
I am honestly not sure what your point is by now
solar grove
idk just high and pontificating
gloomy rain
Maybe this would be a good time to refocus back on the topic of the channel then.
grave jolt
i support the orange people
gloomy rain
This is a channel for discussion of the Python language, its features and implementation.
flat gazelle
you really don't tbh
radiant garden
Well, keeping up with the topic, CPython should scrap its codebase and rebrand to ASMPython. :v)
solar grove
that's a discussion to have too
scripting is great but idk if its the best, I love that it democratized code but its like playing with fire
radiant garden
Scripting languages like Python exist in their appropriate domain.
solar grove
the right tool for the right job
tawdry gulch
Python is scripting language?
gloomy rain
@radiant garden I don't think that would actually be advantageous. Modern C compilers are so good at optimization, it probably wouldn't help to manually implement the interpreter in ASM. Rather the opposite.
tawdry gulch
I guess it could count as one
solar grove
its not about knowing all the tools, its about knowing where to bang the hammer to make the machine run again
gloomy rain
Completely disregarding the fact that you'd need to implement a different one for each architecture.
radiant garden
@gloomy rain Why, then it could perhaps be more advantageous to rebrand to SUBLEQPython instead. (Going back to the topic of low-level esolangs)
gloomy rain
I'm not familiar with that one.
flat gazelle
VerilogPython
radiant garden
It might filter the userbase to only those on a SUBLEQ computer, but it's a sacrifice worth making in order to understand the inner legos.
solar grove
@gloomy rain i wasn't saying it was advantageous I said it would be fun to think about lol
gloomy rain
I was responding to Olivia, so I'm not sure what you're referring to.
solar grove
oh whoops sorry misread that lol
@radiant garden if you want to talk low level esolangs i'm down
tawdry gulch
BF Anyone?
radiant garden
I've designed a few (not necessarily low-level), but I think it falls outside the scope of this channel
solar grove
DM?
radiant garden
Prefer not (but do check out https://esolangs.org/wiki/User:RocketRace)
gloomy rain
Feel free to discuss other languages, esoteric or not, in the off-topic channels.
visual shadow
my two cents is that one should be careful with slippery slope arguments, it's easy to say that "abstraction is a hand wave, and one should actually understand it all" without realising that the very nature of that argument is also another form of "hand waving". There's a certain level of depth at which we can understand a topic without having to go into all details, and be able to perfectly utilize and implement the knowledge where it actually matters with just that level of understanding. There's also the saying "Don't lose sight of the forest because of the trees"
visual shadow
it's good to dive deep and understand things as you see fit. but that should be a pursuit of it's own, and not something that's assumed to be critical for someone to be competent in python.
solar grove
very true, that's why I love Python I switched to it for interviews specifically because it is so expressive and I can get something going quick
flat gazelle
I used to dislike high level languages, but programming is much more fun if you can write code in accordance to how you think, rather than what the machine needs to do in order to accomplish it
solar grove
I used to dislike high level languages, but programming is much more fun if you can write code in accordance to how you think, rather than what the machine needs to do in order to accomplish it
@flat gazelle might even be more than that, high level languages have their place and so do scripting languages. So does Spanish and French lol its all just language and people
radiant garden
Cognitive "noise" is real.
solar grove
Python feels easy because it feels like the language you know, if it was in Croatian you would have a hard time lol
visual shadow
honestly, using python gave me a better understanding of the low level languages as well, i could finally see how the dots connected, while having an abstraction that just "made sense".
gloomy rain
There are certain situations where understanding of implementation details can be important, but it's relatively rare, and not having that knowledge ahead of time isn't necessarily a critical problem.
solar grove
There are certain situations where understanding of implementation details can be important, but it's relatively rare, and not having that knowledge ahead of time isn't necessarily a critical problem.
@gloomy rain This is why I am into Dev X
its ironing out those details
gloomy rain
What is that?
solar grove
like User Experience it is Developer Experience
I work at a startup in SF lol
making it easier for devs to dev
gloomy rain
Oh, never heard of that term before.
unkempt rock
is there any new idea to build
solar grove
I wanted to be a Linux Kernel dev for so long but I got more mature lol
there is always a new idea to build
You all know Linus Torvalds wrote Git to make his dev x better right?
he got sick of his codebase getting axed
so he wrote Git
unkempt rock
idk
unkempt rock
nerver mind just asking
gloomy rain
This isn't really the right channel for that.
solar grove
DM me lol if you really want to build something so do I
gloomy rain
If you need help coming up with project ideas, you can ask in #python-discussion
solar grove
Its not about that lol I thought this was #internals-and-peps
gloomy rain
The topic of this channel is:
Discussion on the use cases, implementation and future of the Python programming language including PEPs, advanced language concepts, new releases, the standard library, and the overall design of the language.
solar grove
yea I've read all the PEPs you peeps just dont know me
unkempt rock
plaese accept my apologies
gloomy rain
No worries.
unkempt rock
i dindt know this
solar grove
not trying to flex but this has been my hobby since I learned BASIC as an 8 year old, I have been around
gloomy rain
@solar grove If you want to discuss your background as a developer, please do so in off-topic.
unkempt rock
i ve never been 8
solar grove
that's all I wanted to know
lets talk Python
why are the tokens broken out they way the are?
unkempt rock
my tokens?
solar grove
do you guys really want to discuss the grammar of Python or not?
I am a linguist in my heart
unkempt rock
sorry i need to sleep
solar grove
sleep away, if anyone wants to discuss Python as a language DM me, I don't want to clog up the chat
true ridge
why are the tokens broken out they way the are?
@solar grove what do you mean by that?
solar grove
each character is considered a token right?
true ridge
no
true ridge
for example, a 150-char long string might consider a token
solar grove
not necessarily characteres
true ridge
or -> arrow operator
solar grove
but there are explicit control characters exactly
true ridge
yes, generally tokens are 1, 2 or 3 characters long but as I given the example, they don't have to be. NAME (identifier), NUMBER, STRING are the basic examples of the tokens that rule don't apply
true ridge
I feel like we are not talking about the same 'tokens'
solar grove
are you talking about the physical character on the screen like :
cause that's a token
true ridge
Well, I'm talking about 'tokens' as in the input of parsers
solar grove
yea dude I've written an interpreter lol
call it a token or whatever
its a control character really
true ridge
why are the tokens broken out they way the are?
@solar grove I still don't get what you are trying to ask here
solar grove
why does python have its particular grammar?
spark magnet
that is not what people mean when they say "control character"
solar grove
why isn't it different?
that is not what people mean when they say "control character"
@spark magnet I know that, everything is a control character expand your definition
its all bits and bytes and nibbles
this is why fundamentals are important
"/n /t /0" I know all that shit
spark magnet
why not use the definition other people use?
solar grove
its still just memory
it "controls" shit in memory
interpreters "interpret" shit
so that's why I call it a control character
spark magnet
once you say "everything is an X", then X is a useless word.
spark magnet
ok, also, once you say "whatever", the conversation is kind of over.
solar grove
sorry I'm from norcal it a verbal crutch
spark magnet
well, "the language isn't important" also kind of prevents discussion
flat gazelle
it is very important to use precise definitions when talking about math
solar grove
I call everything "dude" and I say whatever
yea true
I know differential equations
I get it lol
I was a math guy first
precision is important but I'm not a precise person
I respect it but I'm a tornado, its just how I work
I need to speed up to slow down
trail warren
@spark magnet I was looking at your python aware lecture - wanted to clear up a few things concerning it.
1, I'm confused about compilation to bytecode of a module. I've tried using dis.dis() on an imported module, and it only showed code concerning the defined functions/classes inside the module, not any of the statements inside the module itself - do the module statements themselves get compiled to bytecode too or not? I assumed they do, since it's mentioned somewhere around 15:00
spark magnet
@trail warren i think you mean you don't see the code that is inside the functions?
trail warren
no, I don't think so. I run dis.dis(asd) (asd is name my module, that contains some print statements at the top, then there's two functions declared - def testicek(): and testicek2():
When I run dis.dis(asd) it prints something like:
Disassembly of testicek:
13 0 LOAD_CONST 1 (1)
2 STORE_FAST 0 (x)
spark magnet
maybe you can pastebin the full module and dis output?
trail warren
sure
here is the module
https://pastebin.com/c9F7g2qR
here is the output from IDLE:
https://pastebin.com/1pvvGReR
spark magnet
oh, hmm, i guess dis doesn't show the module statements, it disassembles the objects it finds in the module.
trail warren
but regardless, they exist somewhere right?
spark magnet
dis has changed a lot since I did that talk
trail warren
So just to be sure - when the compiler goes through a module it creates separate byte code objects for the module statements, and separate code objects for each function, correct?
spark magnet
but maybe it never did this? Because the code for the module isn't saved anywhere, it's just executed when you import it.
yes
you can see the whole thing if you use python -m dis that_file.py
spice pecan
>>> help(dis)
Help on function dis in module dis:
dis(x=None, *, file=None, depth=None)
Disassemble classes, methods, functions, and other compiled objects.
With no argument, disassemble the last traceback.
Compiled objects currently include generator objects, async generator
objects, and coroutine objects, all of which store their code object
in a special attribute.```
Modules don't have a `__code__` attribute, so you can't disassemble their bytecode. You could, however, view and diassemble `.pyc` files generateddis.dis functions similarly when disassembling classes, it simply disassembles all methods and nested classes in it
trail warren
alright, great, thanks. I'll try that.
The second thing I was quite unclear about was "frames" - I understand them in the classic sense of being an "object" on the stack, that holds various things like local/arg values, a return address, and also data where sort of data is in the previous frame (i.e. what the address of the first local variable is, in the previous frame, or something like that).
When talking about frame objects, I'm not really sure what you mean -
1, are these separate objects, that exist on the heap?
2, You also mention that they contain data about the code block that is actually executing and the current line of code - so I guess those are all contained in addition to the data I described above, like return address etc.?
spark magnet
yes, frame objects are on the heap. and they contain references to the code object, and the bytecode offset that is currently executing
btw, these questions came from this talk: https://nedbatchelder.com/text/aware.html
trail warren
So do these frame objects actually contain all the data I mentioned? Return address, local values etc?
And there also exists a separate "frame" on the stack, when executing the code, right?
spark magnet
the return address is a referencee to the caller's frame. The locals are a reference to a dict full of locals.
In CPython, there is also the C stack, which has the frames for the C code that is interpreting the bytecode.
trail warren
Hmm.. sorry I'm still very confused about that. So are the frame objects actually necessary? Or are they there mostly for debug possibilities etc.? Because I don't see how we actually "need" them.
spark magnet
@trail warren they are the Python call stack.
how else would returning from a function work? Or where would the locals be?
flat gazelle
@still reef this is not the correct channel,ask in #web-development
spark magnet
the python call stack is a linked list of frame objects
trail warren
Couldn't that be done by using the C stack?
spark magnet
i don't know if it could. it;s not.
trail warren
If the C stack isn't used for it, what's it used for?
teal yacht
it is used
spark magnet
the c stack keeps track of the C code that implements the bytecode interpreter
i have to be afk for a few hours... others will help
trail warren
of course, thanks a ton!
still reef
@still reef this is not the correct channel,ask in #web-development
@flat gazelle Thanks a lot .
queen thunder
can someone please help me with web scraping using python
mint forge
heady mauve
git's mixed use of stdout and stderr is frustrating
spark magnet
@heady mauve are you using --porcelain?
dusky kettle
sacred yew
wrong channel
normal pond
Hello everyone!
Is there a way or tool to protect Python code or obfuscate it like dotnet obfuscations techniques ?
undone hare
Hey @normal pond, see #❓｜how-to-get-help
south plaza
Is there a module I can use to copy or read the text of a tooltip that is displayed from hovering over text or a button/image in my browser?
undone hare
Hey @south plaza, see #❓｜how-to-get-help
marsh hare
will there be a macro-alike-function for python
Something like
import module
module.something() ```
Will be replaced by
module!.something()```
marsh hare
well just asking , thanks for participating 
brave badger
There's a PEP for macros although I'm a bit meh on it
spark magnet
@marsh hare why is it important to do that in just one line?
marsh hare
well that's a good question , just wanted to know about it , btw tbh with my thoughts it is kinda useless
true ridge
https://pypi.org/project/import-expression/
@marsh hare I've written a clone of it in the past, as an example to my 'syntax-manipulation' library. https://github.com/isidentical/freesyntax/blob/master/example2.py
marsh hare
Oh sure I'll definitely have a look at it
cloud crypt
@true ridge ahahaha do you remember the example
true ridge
Indeed
heady mauve
@spark magnet no I have never used porcelain and until about 5 minutes ago I thought it was just an abstraction for gitpython.
boreal umbra
tempfile.TemporaryDirectory doesn't define a __del__, but the docs say that the directory related to a TemporaryDirectory object is deleted when the object is deleted. I can't imagine how else they would have that functionality.
peak spoke
Looks like it's done through weakref.finalize in this case
true hollow
did you thought about another way to make the cla?
imo asking for your real name and address is invasive
spark magnet
@true hollow idk which CLA you mean, but this probably isn't the channel to discuss it.
true hollow
psf cla
spark magnet
try asking about it here: https://discuss.python.org/
spark magnet
the python-dev mailing list? https://mail.python.org/mailman3/lists/python-dev.python.org/
wanton smelt
Hey folks, are we allowed to post blog links? I've started writing one related to Python and I'd like to share it with everyone.
gloomy rain
@wanton smelt We generally don't like self-advertisement unless it's relevant to the context of an ongoing discussion. Also, for future reference, questions about the community itself, and its rules, are best asked in #community-meta.
wanton smelt
@wanton smelt We generally don't like self-advertisement unless it's relevant to the context of an ongoing discussion. Also, for future reference, questions about the community itself, and its rules, are best asked in #community-meta.
@gloomy rain Thank you for the response...I didn't find anything specific regarding this in the rules page so thought I'd ask. I'm not really advertising anything, I don't make any money off of it nor is it any kind of service. It's just an article that is meant to help Python programmers 🙂
gloomy rain
It's a nuanced question. I will discuss it internally with the staff team, but hold off on posting a link for the time being.
wanton smelt
Sure, I don't wish to do anything against the rules and the blog is hosted on Hashnode so even the page doesn't have any ads as far as I know. Let me know!
gloomy rain
@wanton smelt Sure, I'll get back to you soon.
wanton smelt
Thank you
trail warren
I think I'm sort of starting to get it... in say c#, generally if you have some sort of "object" allocated on the heap (inside main function you have a local variable x, with x = new SomeClass(); or something), then somewhere in ram of the actual native/machine code of the app that is running, the address of this object is stored.
But in python, I guess there's no such thing, and instead it's something more along the lines of:
CPython runtime takes care of everything - if some sort of byte code makes a call of some function, like say x = function() ( technically I mean the the corresponding compiled byte code statement of that), then basically the runtime looks for function() in some dictionary object.
If it finds it, it then take the code block reference inside, creates a new frame object for this new function call (again, this is inside the runtime/vm), and starts interpreting the code block. Eventually, when it returns, the value is saved to some temporary memory address of the runtime, the runtime looks for the x variable, and then attempts to save the value to it.
Point being, the VM/runtime really acts as an actual virtual machine that's reading the byte code, one line at a time. It's just like a machine that doesn't doesn't use low lever assembly code, but relatively higher level byte code. But besides that it really functions very similarly, going through the code one line at a time. (Unlike say .NET, where the VM actually compiles large blocks of code, those are run, and sometimes control goes back to the runtime etc., and it might do some garbage collection or something).
gloomy rain
@trail warren As far as I'm aware, C# is JIT compiled, so it's compiled to bytecode ahead of execution, and then the bytecode is compiled to machine code during execution. Whereas Python is translated to bytecode, which is then interpreted by the VM instruction by instruction. So unlike C#, Python bytecode is never compiled to machine code.
spark magnet
Python will be JITted if you run it with PyPy.
gloomy rain
@trail warren Yeah, that's what I said, right?
But right, what I said applies to CPython, not PyPy.
trail warren
Right I'm talking about CPython through the long ramble I wrote too.
spark magnet
ok, it's tricky to compare Python and C#, since Python has a few different implementations. But I understand you now.
but I doubt C# and Python differ that much in their execution model.
You said, "in say c#, ... somewhere in ram of the actual native/machine code of the app that is running, the address of this object is stored.
But in python, I guess there's no such thing"
Every Python object exists at some actual machine address.
trail warren
Oh right, I mean everything always does... but when I say "machine code" .. I really mean ram that's associated with the actual "process"/app that's running. In C# there is such a thing - there's the "ram" for the runtime and the machine code for it, that keeps track of things like garbage collector etc, and then there's the actual memory and compiled code of the app you're building.
spark magnet
how is that different than Python?
trail warren
Well I'm assuming that Python doesn't actually have compiled machine code for the blocks you're running?
(CPython again)
spark magnet
CPython doesn't, no. But you were talking about data in ram I thought?
trail warren
I am, yeah? Not sure what you mean now.
fallen slateBOT
:incoming_envelope: :ok_hand: applied mute to @ruby dirge until 2020-10-18 20:03 (9 minutes and 59 seconds) (reason: mentions rule: sent 94 mentions in 10s).
north root
!ban 767409004530892810 spam pinger
fallen slateBOT
:incoming_envelope: :ok_hand: applied ban to @ruby dirge permanently.
spark magnet
compiled machine code is about how the code gets executed. How is that related to the program's data in ram?
trail warren
Oh right, I see, what I meant is that with C# / .NET, I think there's an actual separate call stack for the "app" you're building, and for the runtime. I think? I'm not sure, but conceptually I think it makes sense to think of it like that.
timber dagger
in C#, the bytecode (IL) is generally compiled into machine code before it gets executed (which helps with making things run faster), if that's what you're talking about
whereas CPython doesn't do that, the only machine code that runs is pre-written C going around a loop executing python bytecode instructions
gloomy rain
@trail warren Each thread in an application, whether its a native binary executing machine code, or a VM interpreting bytecode, will have an individual call stack keeping track of function calls.
That stack will of course be stored somewhere in memory.
timber dagger
yep
trail warren
What I mean is that with CPython, there really isn't another "subprocess" running that's "separate". There's just the bytecode instructions. So it has to be the runtime that keeps track of all the objects/data.
Where-as in C#, there's the compiled machine code, a call stack etc for it, and it has references for objects. The CLR might know very little about them. It will do stuff when you use virtual methods etc., but for basic operations, the CLR doesn't really need to do anything.
Does that make sense?
@gloomy rain I see. I thought that's the case.
@timber dagger right, that's what I mean.
I think I'm being too confusing, sorry guys. It's just weird to me to think that ... there's no "machine code" for your actual app, that would be aware of the references/addresses of variables/functions. It's probably too obvious to you, because I mean.. of course there's not, CPython interprets the byte code, there is no "machine code of your app", so it might be a bit too obvious to you?
gloomy rain
I'm not very familiar with the specifics of how .NET JIT compilation works. I found this quote:
At runtime, a just-in-time (JIT) compiler is responsible for compiling the IL into native code for the local machine just before a method is executed for the first time.
This might imply that each function is JIT compiled individually and that the CLR does in fact keep track of the call stacks of the application itself. But I can't really say for sure without digging into this further.
spark magnet
@trail warren it is something to wrap your head around, coming from C, that your program is never in a machine-code form. But short C# or Java programs might also never get to the point of being JITted. The execution isn't that different.
trail warren
@gloomy rain I don't think it does, but the CLR definitely keeps track of how many times a method has been called etc., for recompiling/optimization, if nothing else.
timber dagger
i'm pretty sure that the stack trace is kept around somewhere in a form that can be interpreted and manipulated by the CLR because i've seen C# stack traces
i'm one of the most active modding helpers for a game written in unity so i see multiple of them every day, they definitely exist
trail warren
Yeah you're probably right, I have no clue.
spark magnet
There must be a C# stack, in addition the implementation-language stack
gloomy rain
But yeah, in the case of CPython, only the VM itself is aware of machine addresses for the memory used in the Python program, and it won't be "aware" of what the significance of those addresses are for the specific program.
trail warren
@spark magnet When you say "But short C# or Java programs might also never get to the point of being JITted." that sounds interesting, when does that happen?
spark magnet
In CPython, there is the Python stack (a linked list of frame objects) and the C stack.
@trail warren my understanding is that evvery JIT has a warm-up time, and no JITting happens until code has executed for a while without JITting.
timber dagger
the C# JIT implementations i've seen don't do that
spark magnet
ok
gloomy rain
That might vary between implementations and circumstances, I guess.
timber dagger
errors from a method with invalid IL will often show up with the method not in the stack trace because it hasn't even started being executed yet
but yeah it probably varies, there isn't necessarily just one way to do JIT compilation
trail warren
I mean I'm not sure I get that... "until code has executed for a while without JITting?" how does code execute in .NET if it hasn't been JITted? Is there some sort of basic interpretation approach for the first few lines/when JIT engine is warming up or something?
spark magnet
@trail warren i could be wrong about .NET, but in general, JITting is an optimization, so there are differences in when it happens
trail warren
I see.
gloomy rain
@trail warren I guess the implication would be that JIT compilation might only be worth the effort of compiling if you're going to execute the function multiple times, so unless you have reason to believe that it will be executed several times, it might be faster to just interpret the bytecode.
trail warren
Yeah I guess taht would make sense.
spark magnet
whats the father of inumerate
@unkempt rock huh?
trail warren
"inumerate the first"
unkempt rock
is it an iterator
gloomy rain
I don't know what you mean by "father"
trail warren
I think he means the parent class? Not sure that makes sense though.
gloomy rain
It doesn't, because enumerate is a function.
trail warren
Anyway, thanks a ton for the discussion guys.
gloomy rain
@unkempt rock If you have questions about Python, please see #❓｜how-to-get-help. This is not a help channel.
trail warren
I think I might start reading some books on CPython and how it works.
unkempt rock
no i just cant figure out why we cant print enumerate without making dict of it
timber dagger
enumerate is actually a class in CPython
heady mauve
If I have a library in my python path and a library in my python installation folder, will import prefer the one in the installation folder or the one on my python path?
timber dagger
its base class is object
unkempt rock
aha thank you btw
gloomy rain
@unkempt rock enumerate produces an iterator, yes.
So you can't index into the result.
raven ridge
And you can't print an iterator because printing it requires consuming it.
unkempt rock
got it
raven ridge
And it would be really surprising if printing an object modified that object.
unkempt rock
how
gloomy rain
heady mauve
thanks
gloomy rain
Though, use a virtual environment, so you don't even have to worry about this.
heady mauve
gotcha
gloomy rain
@wild sandal This is not a help channel. See the channel topic.
cloud crypt
#microcontrollers seems like a suitable one for you, though
wild sandal
gleaming rover
@unkempt rock this is a channel for language discussion; you might want to try #❓｜how-to-get-help.
sacred yew
unkempt rock
Ok 😄
finite moss
Can I ask a doubt?
gleaming rover
Can I ask a doubt?
@finite moss (I don't mean this in a condescending way) in standard English, the more appropriate word is "question"; "doubt" is generally used more to convey a lack of trust, as opposed to a lack of understanding/knowledge.
anyway, this channel is for discussion of the Python language itself
so if what you wanted to ask would be on topic, then sure; otherwise, you can consider #❓｜how-to-get-help
but I suppose it'd be hard to tell without asking in the first place, so what problem do you have? 🙂
leaden meteor
hey guys does anyone here know why powershell refuses to prompt inputs if I'm using the turtle module?
I realized my problem wasn't a syntax or code related problem so I thought I'd ask here
extra info here #help-broccoli
i apologize if this is the wrong channel for this, I wasn't able to find any answers anywhere else
candid stump
save method of pillow library save the images but if we pass quality parameter it compress the image size, does anyone know which algorithm it use to compress image
smoky turret
have you read the docs?
also, just a guess, but almost certainly some flavor of jpeg
candid stump
yes i read the docs but doesn't help
teal yacht
The docs say the additional arguments are ignoreb if the file format converter doesn't support it
So it uses whatever algorithm you specified with the file extension
prisma cypress
Hello people, I don’t know if this is the right channel, but will be happy to redirect.
I’m starting a new project and was hoping to plan it out instead of jumping straight into code.
I also don’t want to stuck too heavily on planning. I was wondering how should I go about it?
To be more specific, I’m writing a data pipeline that scrapes data and puts it into a data warehouse for later retrieval.
I was planning on doing ADT for a pipeline object and then a scraper object. Or should I just write a script first and refractor from there?
smoky turret
more planning will usually always be better than less. The easiest way to map out a project is to (after thinking over some general structure) is to just write all the topmost level logic, call functions that dont even exist yet and implement them later
def main():
thing = get_the_thing()
while some_condition(thing):
use_the_thing(thing)
if __name__ == "__main__":
main()
``` kinda like this, obv with actual good names that describe what you are doingthen go through and start implementing those functions, still calling things that dont exist, until the functions become simple enough that you can actually start writing real implementations
prisma cypress
Ahh makes sense.