I'm leaning towards LBYL too. I guess I could've used collections.abc.Iterable (thanks SO) to check whether iterable is iterable.
#internals-and-peps
grave jolt
peak spoke
the if does look better in some cases imo compared to a normal try if doable; but for some cases contextlib.suppress makes it just as nice
unkempt rock
For me it entirely depends on how simple either or is. If I have nested ifs that will all result in the same error, why?
peak spoke
had to LBYL once for performance on dicts when I was doing an in check every few microseconds because handling the exception took substantially longer
unkempt rock
Hrm
somber halo
try:
self.iterable = iter(iterable)
except Exception as e:
raise TypeError("first argument to constructor must be iterable, got:"
f"({type(iterable)}) {iterable}") from e
^ changed that
grizzled vigil
Yeah, in that case it makes sense to LBYL if the if x in y is frequently false.
grave jolt
After the fix, the if makes it symmetric with the second check in this case. It's also shorter and less cryptic.
grizzled vigil
But if 90% of the time, you expect the key to be in the dictionary, it tends to make more sense to use try ... except KeyError
grave jolt
if not isinstance(iterable, Iterable):
raise TypeError("first argument to constructor must be iterable, got: "
f"({type(iterable)}) {iterable}")
if not isinstance(cache_size, int):
raise TypeError(f"cache_size must be int, got: ({type(cache_size)}) {cache_size}")
@grave jolt ok, looks good to me.
peak spoke
I think it for some was some key handling in the kivy qualifier for the last codejam
so potentially whole keyboard of keys I did not have in the dict
grave jolt
Is there a built-in python decorator that enforces type hints at runtime?
So that I could write
@typecheck
def __init__(self, iterable: Iterable, /, *, cache_size: int)
I tend to prefer just having type hints over isinstance() checks, but it's not necessarily wrong by any means. Especially in the case of isinstance(cache_size, int).
Not that I'm aware of
But I wouldn't be surprised if it was implemented in some 3rd party lib
grave jolt
I implemented it myself in my almost-ADT lib, it helped me a bit
somber halo
I just use type hinting everywhere nowadays and have mypy to do static analysis.
grave jolt
I want to rewrite it one day so that it's actually usable
somber halo
don't really use those isinstance() nowadays
grizzled vigil
There's a significant amount of resistance though for including any form of strict type checking in stdlib, so I don't think anything like an @typecheck would be around anytime in the near future at least.
unkempt rock
I watched a talk today where the guy stopped his talk to complain about type hints haha.
grave jolt
Alright, I'm going to bed. It's literally 5am here.
grizzled vigil
gnight
peak spoke
When was json added to the stdlib?
somber halo
what where his compaints?
grave jolt
gn
somber halo
5 am?
unkempt rock
Night, thanks for working with me a bunch
somber halo
are you in Russia?
grave jolt
you little stalker
It's 4:00..12:00 in Russia
somber halo
cheeki breeki
marble vale
indeed
grizzled vigil
A little bit of chatter is fine, but let's try to keep it more formally on-topic in this channel
somber halo
nah, your time zone isn't too far from mine.
grizzled vigil
Feel free to discuss timezones and such in #ot0-psvm’s-eternal-disapproval
somber halo
chill dude.
either way @unkempt rock, what were that speaker's complaints about type hints?
marble vale
Btw where can I ask about a slight confusion I have with escape sequences?
grizzled vigil
We're trying to keep this channel specifically focused on formal Python language discussions, unlike python-general which tends to get more distracted. So please don't tell us to "chill" @somber halo, that's not helpful for anyone.
@marble vale #❓｜how-to-get-help
unkempt rock
https://youtu.be/rrBJVMyD-Gs?t=1922 -- the speaker on disliking type hints and linterss, at pycon
I cant explain it from a code review perspective like he does
So just let him say it.
somber halo
My intention wasn't to be patronizing at all @grizzled vigil, I just don't see what's wrong with releasing a little bit of steam here and there
grave jolt
That's what #ot is for
grizzled vigil
@somber halo That's the point of the off-topic channels, which is why we have 3 of them.
somber halo
If you (and others?) don't consider that acceptable, I'm OK with that and I'll refrain from that.
In order to avoid this in the future, you might want to pin a message to this channel to specify that.
it should make the "social contract" of this channel clearer : )
it's just a suggestion.
grizzled vigil
The main purpose is that if we don't try to redirect off-topic chatter, it very quickly devolves into that constantly and makes it difficult for others to read over the conversation to get anything out of it. You'll find that the same mostly applies to the other non off-topic channels as well, although we don't enforce it much in python-general.
north root
this channel is very strictly on-topic because we don't want this channel to become #python-discussion. it gets very hectic in there. in the past, #python-discussion and #internals-and-peps were both one channel: #python-discussion. that channel was barely ever on-topic, so that's why split it into to channels. this one is strictly on-topic, but #python-discussion is loosely on-topic, for newcomers and such
that's just some insight on how it used to be
grizzled vigil
That's a reasonable suggestion @somber halo, it might be worth mentioning that in a pin to make it clearer.
@north root Would you mind if I pinned your recent message? I think that pretty much sums it up quite well.
north root
sure, go ahead
grizzled vigil
this channel is very strictly on-topic because we don't want this channel to bec...
deft pagoda
@grave jolt
from collections import deque
class PeekDeque(deque):
def __init__(self, iterable):
self.iterable = iter(iterable)
def __iter__(self):
return self
def __next__(self):
if self: return self.popleft()
raise StopIteration
def peek(self, n=1):
while len(self) < n: self.append(next(self.iterable))
return self[0] if n == 1 else list(map(self.__getitem__, range(n)))
def __bool__(self):
try:
self.peek()
except StopIteration:
return False
return True
how about this version
grave jolt
!e
from collections import deque
class PeekDeque(deque):
def __init__(self, iterable):
self.iterable = iter(iterable)
def __next__(self):
if self: return self.popleft()
raise StopIteration
def peek(self, n=1):
while len(self) < n: self.append(next(self.iterable))
return self[0] if n == 1 else list(map(self.__getitem__, range(n)))
def __bool__(self):
try:
self.peek()
except StopIteration:
return False
return True
i = PeekDeque(range(8))
for x in i:
print(x, i)
fallen slateBOT
@grave jolt :warning: Your eval job has completed with return code 0.
[No output]
My idea was to make an iterator that shows a little bit of itself
Like it's teasing you to get more elements.
deft pagoda
there, added the __iter__
grave jolt
I guess that works as weel.
deft pagoda
that seems to be unraveling everything
grave jolt
I'm too lazy to test it from the mobile app.
unkempt rock
It all started because I was complaining that you cant __repr__ a generator and have to look at <generator at 0x7dsfds8f7dsf> in the i shell
grave jolt
I guess that works, but the peeking on bool is not obvious at first glance in __next__
But maybe I'm just too sleepy already
unkempt rock
Its a pretty good solution.
grave jolt
Took me a while to figure out why it works
There might be a bigger problem with inheritance here.
grizzled vigil
Yeah, I agree that that relying on the custom __bool__ in __next__ might be a bit on the implicit side.
grave jolt
It's a deque, but you can't popright of append anywhere in it.
grizzled vigil
It's a clever solution though :-)
deft pagoda
from collections import deque
class peekable:
def __init__(self, iterable):
self.iterable = iter(iterable)
self.peeked = deque()
def __iter__(self):
return self
def __next__(self):
if self: return self.peeked.popleft()
raise StopIteration
def peek(self, n=1):
while len(self.peeked) < n: self.peeked.append(next(self.iterable))
if n == 1: return self.peeked[0]
return tuple(map(self.peeked.__getitem__, range(n)))
def __bool__(self):
try:
self.peek()
except StopIteration:
return False
return True
def __repr__(self):
return f'peeked([{", ".join(map(str, self.peeked))}])'
ok, back to composition
grave jolt
I absolutely agree with aeros
deft pagoda
it's something with the __repr__ of deque
grave jolt
Can you do that in one line without using /e(val|xec)/, though?
pliant tusk
@grave jolt yes
grizzled vigil
Also, I would say that it would be singnificantly more readable to have return self.peeked[0] if n == 1 else tuple(map(self.peeked.__getitem__, range(n))) split into multiple lines, that's getting a bit out of hand for a ternary :p
deft pagoda
i agree, that was done to appease machadojpf though
grizzled vigil
Ah, the good ol' "it could be on one line"?
deft pagoda
yep
grave jolt
stnuoc ytilibadaer
deft pagoda
bless you
i think i like the ... repr better though, this repr just shows anything that's been peeked
you could return a default value if you peek too far
grave jolt
You can also implement it as a function that returns a generator with custom __str__/__repr__
I think that's possible
somber halo
no need to appease me, I just suggested something.
grave jolt
Alright, good morning everyone. In a reverse sense.
somber halo
and in my experience, that ternary is not big at all compared to some that I've seen before. But of course, this is subjective 🙂
abstract talon
op = input("Enter an operator: ")
num2 = float(input("Enter second number: "))
if op == "+":
print (num1 + num2)
elif op == "-":
print (num1 - num2)
elif op == "*":
print (num1 * num2)
elif op == "/":
print (num1 / num2)
else:
print("Operator Invaild")```
*can anyone guess what this does or what it;s meant to be?*it's a calculator
severe lichen
Hey, I've heard that Python is getting a new parser. Do you think it will bring us new constructs/syntax sugar?
severe lichen
simply do eval(f"{num1} {op} {num2}") (spoiler - do not)
somber halo
what do you mean with that spoiler?
pliant tusk
peekable = type('peekable', (), {
'__init__': lambda s, i: [
setattr(s,'iterable', iter(i)),
setattr(s, 'peeked', __import__('collections').deque())
][0],
'__iter__':lambda s:s,
'__next__':lambda s: s.peeked.popleft() if s else (()for()in()).throw(StopIteration),
'peek':lambda s, n=1: [
[*iter(lambda:[s.peeked.append(next(s.iterable)),len(s.peeked) < n][0],1)],
s.peeked[0] if n == 1 else (*map(s.peeked.__getitem__, range(n)),)
][1],
'_try':lambda c,t, a=(), k={}, f=lambda a:a, e=Exception, r={}:r.get('r',type('',(__import__('contextlib').ContextDecorator,),{'__enter__':r.clear,'__exit__':lambda s,*a:issubclass(a[0]or map,e)and[r.update(r=f(a))]})()(t)(*a, **k)),
'__bool__':lambda s:s._try(lambda:[s.peek(),True][1],f=lambda a:False),
'__repr__':lambda s:f'peeked([{", ".join(map(str, s.peeked))}])'
})
``` @deft pagoda @grave jolti onelined it
severe lichen
I mean that it's probably possible to inject some sort of malicious code
somber halo
ah, yes. if you use just that one liner like that sure
severe lichen
And a dict of {str: function from operator module} probably shows intent better
somber halo
num1 = float(input("Enter first number: "))
op = input("Enter an operator: ")
num2 = float(input("Enter second number: "))
if op in ("+", "-", "/", "*"):
eval(f"{num1} {op} {num2}")
else:
print("Invalid operator!")
I cant read that at all @pliant tusk haha
I remember when we were playing with snekbox, you were pretty good at the one liners
pliant tusk
i am a firm believe that everything can be onelined
deft pagoda
this isn't #esoteric-python
from sly import Lexer, Parser
class CalcLexer(Lexer):
tokens = { NUMBER }
ignore = ' \t'
literals = { '=', '+', '-', '*', '/', '(', ')' }
@_(r'\d+')
def NUMBER(self, t):
t.value = int(t.value)
return t
class CalcParser(Parser):
tokens = CalcLexer.tokens
precedence = (
('left', '+', '-'),
('left', '*', '/'),
('right', 'UMINUS'),
)
@_('expr')
def statement(self, p): print(p.expr)
@_('expr "+" expr')
def expr(self, p): return p.expr0 + p.expr1
@_('expr "-" expr')
def expr(self, p): return p.expr0 - p.expr1
@_('expr "*" expr')
def expr(self, p): return p.expr0 * p.expr1
@_('expr "/" expr')
def expr(self, p): return p.expr0 / p.expr1
@_('"-" expr %prec UMINUS')
def expr(self, p): return -p.expr
@_('"(" expr ")"')
def expr(self, p): return p.expr
@_('NUMBER')
def expr(self, p): return p.NUMBER
if __name__ == '__main__':
lexer = CalcLexer()
parser = CalcParser()
while True:
text = input(' > ')
if text.lower().startswith('q'): break
if text: parser.parse(lexer.tokenize(text))
acc looks like it doesnt work perfectly :/
deft pagoda
this is calculator parser
deft pagoda
no floats though, because regex game isn't strong enough
severe lichen
from operator import add, mul, sub, truediv
op_dict = {"+": add, "*": mul, "-": sub, '/': truediv}
try:
return op_dict[op](num1, num2)
except KeyError as ex:
print("Wrong, buddy")
raise ValueError("Unsupported operator") from ex
i cant wait till 3.9 when @(lambda f:'whatever') will work
severe lichen
Parsing parentheses is a whole another ballgame
deft pagoda
> 2 + ( ( 4 * 3 ) / 12)
3.0
what is going on with that @_()
somber halo
nice expansion Oouja
deft pagoda
it's a special decorator you use inside the Lexer, Parser classes that gives either the grammar rules(Parser) or the regex(Lexer)
unkempt rock
Right, okay.
severe lichen
I swear, some libs change Python so much it looks like another language
deft pagoda
well that's a beazley lib, so it's gonna be diabolical
unkempt rock
I could tell it was a decorator and I thought I noticed a regex
severe lichen
Who's beazly?
deft pagoda
the antichrist of python
somber halo
: D
unkempt rock
it seems like he shouild give it a better name
severe lichen
Decorator syntax was relaxed recently. I'm not sure what it gives me, personally, but i'm sure #esoteric-python had a field day.
unkempt rock
@_() seems to indicate that its nothing.
pliant tusk
https://github.com/chilaxan/py_future
from py_future import jsdict
x = {}
x.a = 1
print(x) #{'a':1}
del x.a```
unkempt rock
a decorator of nothing
pliant tusk
@severe lichen yah its being implemented into 3.9 i believe
deft pagoda
_ was chosen simply because it's short --- the point was to make lexer, parser generation fast as possible
unkempt rock
I think I like that lib from the example at least.
deft pagoda
i used it for boolean logic parsing: https://github.com/salt-die/tiny-math/blob/master/truthtables/parser_lexer.py
unkempt rock
class Null:
'''placeholder null singleton (not actually a null value)'''
def __bool__(self):
return False
Null = Null()
Whats the reasoning here
deft pagoda
I do stuff like that when None is being used
pliant tusk
i needed a way for a python func to know if it was passed null without passing a null pointer into python
unkempt rock
Hrm
pliant tusk
for stuff like implementing del dict[x]
severe lichen
@unkempt rock Now it needs to override isinstance checks and throw NullReferenceErrors for every attribute access.
pliant tusk
cause internally that calls set_item(dict, name, null)
severe lichen
So Python can get authentic null experience
unkempt rock
I notice you also write over the class in the instantiation.
pliant tusk
yea cause i only want one instance of Null to exist
unkempt rock
makes sense
I should learn C/++ its hard to follow some of this type of code without understanding some basic things about it.
even if I dont become a C programmer, I should get the crash course in the ctypes and stuff. Because it comes up a lot.
deft pagoda
i use type constructor when i want a singleton
Null = type('Null', (), {'__bool__': lambda self: False})()
tbh if you go through alot of that code, youd see a lot that doesnt quite make sense
lots of it was written at unhealthy hours
unkempt rock
Well, it cant be as bad my code 😛
I know the feeling.
I will say, you seem like you enjoy the most obscure ways to phrase things generally.
Maybe thats just my ignorance.
severe lichen
Remind me, why python classes implement a lot of stuff as functions instead of methods?
Like isinstance(x, str) instead of x.isinstance(str)?
unkempt rock
I have seen it said at talks that you should "put the tool in the toolbox". Like, if you're making the WeatherCloud API and you have a function that has nothing to do with Weather or Clouds but meterorologists will be using it and looking for it, you put it in the class because thats where they will look.
Thats kind of just a side note I heard the other day.
clear hornet
Isn't this kind of a bogus warning? Shouldn't any object be able to be None?
cedar glen
It would be Optional
wide shuttle
Yeah, it requires a specific type hint, like
from typing import Optional
def something(arg: Optional[int]) -> Optional[int]:
return arg
And Optional[X] is equivalent to Union[X, None].
clear hornet
Okay, that works. Thanks. Still a lot of effort for what works in other languages out of the box.
flat gazelle
implicit nullability is a awful practice and is a source of many bugs
grave jolt
I'm not sure if positional Nones there are good.
It's not clear what they represent in the function call.
flat gazelle
yeah, those should be kwargs
sacred tinsel
ahh
if it can be None, it should absolutely be annotated as such
i do not consent to receiving unannotated Nones
grave jolt
@unkempt rock @grizzled vigil https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.peekable
It peeks only one element ahead, though.
sacred tinsel
oh my god
oh its more_
i wish this was stdlib
im interested to see the source for this
grave jolt
Yes, there's a lot of stuff there
undone hare
Well, there is a source button 
severe lichen
I feel weird about utility libs like more itertools. It gives anither dependency just because i want 1 beautiful line of code instead of ugly function.
gray mirage
Well itertools is standard library
Oh, it's literally more itertools
What, who named this thing
severe lichen
Just what's on a tin
wide shuttle
Wait until he finds out about even-more-itertools
grave jolt
even-more-itertools doesn't have an arbitrarily peekable iterable, time to make one
even-even-more-itertools coming soon
severe lichen
Were hitting levels of itertools that shouldn't be possible
BTW, are there any plans to include more libs into stdlib?
deft pagoda
i got the repr to be more like yours:
def __repr__(self):
try:
self.peek(4)
except StopIteration:
items = ", ".join(map(str, self.peeked))
else:
items = ', '.join(map(str, self.peek(3))) + '...'
return f'peekable([{items}])'
true ridge
@severe lichen no
wide shuttle
Some argued at last year's language summit that the stdlib should try to do less
true ridge
@deft pagoda did you see this https://twitter.com/dontusethiscode/status/1251339365504221184
wide shuttle
It's assigning a generator expression to _ with the walrus, meaning that it's also returned
deft pagoda
i still don't know what it's doing
flat gazelle
why is it using _ as a variable? It is not throwaway...
true ridge
@deft pagoda it is a peek implementation
deft pagoda
yeah, i get that, but why does it look shitty
what happens if all your arrays[0] have a greater id
grave jolt
jesus
lambda calculus is more readable
Oh, I remember that person.
I think I've seen two or three of his talks.
oblique crystal
This stride numpy things look very hacky overall
grave jolt
If pydis channels were people, #esoteric-python would be James Powell
true ridge
Hahaha, indeed
grave jolt
Maybe there's another possible implementation of the peeker (#internals-and-peps message) -- an 'immutable peeker'.
It's like my peeker, but it keeps the cache.
So that you can create an iterator from the peeker, and that iterator won't shift the original one.
deft pagoda
from collections import deque
NULL = type('', (), {})() # None is a valid default value, so we use this for no default.
class peekable:
def __init__(self, iterable):
self.iterable = iter(iterable)
self.peeked = deque()
def __iter__(self): return self
def __next__(self):
if self: return self.peeked.popleft()
raise StopIteration
def peek(self, n=1, default=NULL):
while len(self.peeked) < n:
try:
self.peeked.append(next(self.iterable))
except StopIteration:
if default is NULL: raise StopIteration
self.peeked.append(default)
if n == 1: return self.peeked[0]
return tuple(map(self.peeked.__getitem__, range(n)))
def __bool__(self):
"""Return False if peeked is empty and iterable is exhausted."""
try:
self.peek()
except StopIteration:
return False
return True
def __repr__(self):
try:
self.peek(4)
except StopIteration:
items = ", ".join(map(str, self.peeked))
else:
items = ', '.join(map(str, self.peek(3))) + '...'
return f'peekable([{items}])'
you could iterate over peek(n) here
grave jolt
I see
Well, you can use float('inf') for n :)
I think the if n == 1: return self.peeked[0] is inconsistent. It will break the code if n is parametrized somehow.
deft pagoda
i don't know what you mean
grave jolt
def map_first_n(fn, p: peekable, n: int):
return map(fn, p.peek(n))
then wrap it in always_iterable
grave jolt
lol
deft pagoda
i think the biggest use case is n=1
and i don't wanna to a, = i.peek(1)
that comma annoys me
grave jolt
def peek_one(self):
(element,) = self.peek(1)
return element
gross
flat gazelle
those parens are reduntant
deft pagoda
yeah, i just avoid all that for what i find more usable
peekable would be nice for those arbitrary look ahead parsers
deft pagoda
i think python parser uses single look ahead
grave jolt
My problem with that is just that the function returns values of different types for no good reason. Peeking a single element and peeking n first elements are different tasks, so they should be in different methods IMO.
true ridge
An interesting (and definietly not a real world) idea would be specializing ellipsis to custom expressions, like a macro. Something like this
#define ... *args, **kwargs
class T(Base):
def __init__(self, key, *args, **kwargs):
super().__init__(...)
self.key = key
I really hate writing *args, **kwargs every time I'm wrapping/subclassing something. It would be pretty useful to me 😆
deft pagoda
there's a good reason to return different types --- to avoid an extra comma for the highest use case
grave jolt
There will be no comma if peeking the first element and peeking the first n elements is done with different functions.
spice coral
Ok so we use local vars inside of a function to prevent using up an already established namespace, but then we have to be careful with how we name the function o.o?????
deft pagoda
there's no reason for that
grave jolt
def map_first_n(fn, p: peekable, n: int):
if n == 1:
return fn(p.peek(1)
else:
return map(fn, p.peek(n))
I think this is way worse than a single ,
deft pagoda
i don't
because that use is more contrived
than the most common use case
of a = i.peek()
grave jolt
Why not extract the most common use case in its own method?
deft pagoda
because then i repeat the same logic in two methods
grave jolt
def peek(self):
return self.peek_many(n=1)[0]
you can still peek(default=5)
i mean, i'm not changing it, you could do whatever on your machine with the code
spice coral
@deft pagoda can you help me understand my question up above
grave jolt
Maybe this will work. It's both consistent for different values of n, and doesn't require a comma with the most common use case.
def peek(self, n=None, default=NULL):
if n is None:
return self.peek(n=1, default=default)[0]
while len(self.peeked) < n:
try:
self.peeked.append(next(self.iterable))
except StopIteration:
if default is NULL: raise StopIteration
self.peeked.append(default)
return tuple(map(self.peeked.__getitem__, range(n)))
that's reasonable
@property
def _(self):
return self.peek()
there's a shortcut
wait, i got an idea
grave jolt
def __getattr__(self, name):
if set(name) == {"_"}:
return self.peek(n=len(name))
else:
raise AttributeError(name)
p._ == p.peek(1)
p.____ == p.peek(4)
Then you can easily implement the peek method as:
def peek(self, n):
return getattr(self, "_"*n)
That would make a circular reference, though.
But who said that programming is easy.
deft pagoda
def __getattr__(self, attr):
n = attr.count('_')
if n != len(attr):
return super().__getattribute__(attr)
return self.peek(n)[-1]
In [48]: a = peekable(range(5))
In [49]: a.___
Out[49]: 2
In [50]: a._a
Traceback (most recent call last):
File "<ipython-input-50-83996d449050>", line 1, in <module>
a._a
File "/home/salt/Documents/Python/Snippets/peekable.py", line 21, in __getattr__
return super().__getattribute__(attr)
AttributeError: 'peekable' object has no attribute '_a'
Is there some library that allows you to quickly fetch and store snippets?
deft pagoda
uhh, i dunno, i just keep em all in a folder
grave jolt
I mean, something like pypi or git, but a bit more lightweight. More like gist on github.
$ python3.8 -m snippets load salt-die/peekable
oh, i have my snippets in a repository on github, but i haven't uploaded recent snippets
you know, it would be cool if you could just import directly from github
grave jolt
from from_github import require
peekable = require("salt-die/peekable", version="1.2.0")
Circular dependencies will be lots of fun.
Suppose django uses github imports. Someone pushes some code with a defect that causes a circular dependency, but only during non-leap years.
Then, on Jan 1 2021, GitHub is suddenly out of service
Hm
Maybe you could build a microservice architecture this way.
deft pagoda
grave jolt
some_module = require("http://my.web.site/interfaces/...", version="v1.2")
i've never tried requests, would that do it
deft pagoda
oh wut
grave jolt
Oh
It's even async-ready
But maybe the require could also support an arbitrary URL
But isn't a token required for any interaction with github api?
deft pagoda
yeah, or username password i think
https://pygithub.readthedocs.io/en/latest/apis.html -- i still don't know how to get individual files
deft pagoda
i looked at the other one because it had a bajillion stars
grave jolt
deft pagoda
that's nice --- creating gists from the console would be pretty awesome
deft pagoda
yeah %gist
grave jolt
Get source of a function and upload it as gist
deft pagoda
can't get source if it's defined only in console though
peak spoke
I think for funcs getsource works
grave jolt
Yeah, it doesn't work for classes.
undone hare
This function looks awful, I like it
def __iter__(self):
return itertools.chain(*((cell for cell in row) for row in self.matrix))```Yup
grave jolt
Idk, it's kinda readable...
Given that I was able to guess what it does without context
deft pagoda
you don't need chain at all
grave jolt
def __iter__(self):
for row in self.matrix:
yield from row
Yeah it is understandable, not it looks awful
Oh
Oh..
Ownn
Well, thanks haha
I forgot about the yeild
grave jolt
Well, sometimes you just get a strong urge to use map, reduce or (anything from functools or itertools).
inland spire
@grave jolt does the yield from row yield element by element? or do you get a generator returned for each row?
grave jolt
yield from x
# equivalent to:
for i in x:
yield i
for _ in ((yield i) for i in x): pass
oh, that's illegal
undone hare
Well, we talked about lambdas a couple of days ago, I think lambda i: i > 1 is better than map(partial(lt, 1)) haha
coral geyser
:) lol
whole lake
Hey
inland spire
@grave jolt in that case, that's a really neat way of iterating a 2d list 🙂
undone hare
Well, that's probably worst IMO
wide shuttle
yield from has very complex behavior on top of that
It's what drove old-style coroutines
It's quite the read: https://www.python.org/dev/peps/pep-0380/
It's equivalent to:
_i = iter(EXPR)
try:
_y = next(_i)
except StopIteration as _e:
_r = _e.value
else:
while 1:
try:
_s = yield _y
except GeneratorExit as _e:
try:
_m = _i.close
except AttributeError:
pass
else:
_m()
raise _e
except BaseException as _e:
_x = sys.exc_info()
try:
_m = _i.throw
except AttributeError:
raise _e
else:
try:
_y = _m(*_x)
except StopIteration as _e:
_r = _e.value
break
else:
try:
if _s is None:
_y = next(_i)
else:
_y = _i.send(_s)
except StopIteration as _e:
_r = _e.value
break
RESULT = _r
Let's golf this function
grave jolt
I can golf it
wide shuttle
yield from
wide shuttle
If you start looking through it with coroutines in mind (the EXPR yielding values asynchronously, but you want to account for a GeneratorExit but also don't want to lose exception information in the inner most subgenerator), the code does make sense
It's just a big chunk you have to read carefully
undone hare
I also wonder why except AttributeError instead of hasattr() calls
dict(filter(lambda i: i[0] in cells, self.weight.items()))```Is that any readable too?I guess there is a better way, isn't there?
grave jolt
dict((key, value) for (key, value) in self.weight.items() if key in cells)
```?I don't know the context, though
{key: value for (key, value) in self.weight.items() if key in cells}
Well, self.weight maps the type of cell to its weight, I want to get all the possible mapping ceil -> weight to feed it into a weighted random
grave jolt
{cell_type: weight for (cell_type, weight) in self.weight.items() if cell_type in cells}
{cell_type: self.weight[cell_type] for cell_type in filter(cells.__contains__, self.weight.keys())}
Well yeah, I think I'll go for the verbose way, thanks
grave jolt
Let me refactor that in a moment
deft pagoda
# -----------------------------------------
# -------- Load Github User and API -------
# ------- %gist name file descripion ------
# -----------------------------------------
from github import Github, InputFileContent
github = Github('user', 'password')
salt = github.get_user()
@register_line_magic
def gist(line):
files = {}
line = iter(line.split())
while 1:
try:
name = next(line)
except StopIteration:
break
try:
file = next(line)
except StopIteration:
description = name
break
with open(file, 'r') as f:
contents = f.read()
files[name] = InputFileContent(contents)
try:
description
except NameError:
description = ''
salt.create_gist(True, files, description)
del gist
there you go, quick gists
undone hare
Nice
Interesting, pycharm consider that
def add_coords(coords1: Tuple[int, int], coords2: Tuple[int, int]):
return tuple(map(add, zip(coords1, coords2)))```should use `Tuple[Any, ...]` as return typehint, but it is `Tuple[int, int]`I guess pycharm can't do introspection that deep
somber halo
PyCharm sometimes doesn't deal well with type hinting
grave jolt
Although tuple(Iterator[T]) is tuple(T...), and map([a -> b], Iterator[a]) is Iterator[b], and zip(Iterator[a], iterator[b]) if Iterator[tuple(a, b)]
Wait
slow owl
so
grave jolt
Does add take a tuple of ints?
slow owl
hello
deft pagoda
if this is vector addition, it seems like tuple(int, int) is right
though dunno why you'd map on a 2-vec
somber halo
I would use a list comprehension here to be honest...
deft pagoda
i'd unpack
coarse copper
can anyone tell me how to get pygame? i have tried installing it with cmd but shows an error
im fairly new
deft pagoda
x1, y1 = coords1
x2, y2 = coords2
return (x1 + x2, y1 + y2)
inland acorn
@coarse copper you can grab a help channel and ask there, read how in #❓｜how-to-get-help
deft pagoda
parens can be erased
grave jolt
how cruel
flat gazelle
I would do
tuple(map(operator.add, coords1, coords2))
```if you want a mapthat zip is quite odd imo
deft pagoda
just a lot of machinery for such short iterables
somber halo
return coords1[0] + coords2[0], coords1[1] + coords2[1]
but yeah, generally for vectors of known size, I do the unpacking thing
somber halo
this would avoid the creation of temporary variables
deft pagoda
yeah, but its ugly
grave jolt
Why not make coords namedtuples.
deft pagoda
that would be prettier
somber halo
Also, this function seems like a function that would be called a lot of times... I would really consider inlining this operation.
grave jolt
def add_coords(a: Point, b: Point):
return (a.x + b.x, a.y + b.y)
i would just make a Vector with __add__
somber halo
Considering the context, this seems it would be much important to have an efficient function than a "prettier" one.
deft pagoda
then use numpy
grave jolt
complex(*a) + complex(*b)
might be applicable or not to his scenario
deft pagoda
complex is a good method
flat gazelle
ye, complex is neat here
deft pagoda
just use complex numbers as your vectors
flat gazelle
quaterion() when?
grave jolt
y tho
deft pagoda
at that point you can just use a rotation matrix
grave jolt
make a C extension for adding two tuples
deft pagoda
probably wouldn't save that much time
somber halo
the overhead would be too much for this scenario IMO
deft pagoda
converting from python types to ctypes probably more time than necessary
flat gazelle
in general, 4D vectors are quite neat, and scheme and julia have them builtin AFAIK
grave jolt
Maybe you can construct the function youself in bytecode for maximum efficiency.
somber halo
it would be a matter of benchmarking and assessing the best course of action for optimizing this
grave jolt
!e
import dis
def f(x):
a, b = x
dis.dis(f)
@grave jolt :white_check_mark: Your eval job has completed with return code 0.
001 | 3 0 LOAD_FAST 0 (x)
002 | 2 UNPACK_SEQUENCE 2
003 | 4 STORE_FAST 1 (a)
004 | 6 STORE_FAST 2 (b)
005 | 8 LOAD_CONST 0 (None)
006 | 10 RETURN_VALUE
I would not bother optimizing this tbh. If performance mattered, you would be using vectorized numpy operations
just pick whichever one tickles your fancy
grave jolt
custom bytecode tickles my fancy a lot
deft pagoda
FunctionDef
├──f
├──arguments
│ ╰──arg
│ ╰──x
╰──Assign
├──Tuple
│ ├──Name
│ │ ├──a
│ │ ╰──Store
│ ├──Name
│ │ ├──b
│ │ ╰──Store
│ ╰──Store
╰──Name
├──x
╰──Load
@grave jolt an example would be this https://github.com/brandtbucher/hax#example
grave jolt
So that's kinda the algorithm
instruction | stack (top is on the left)
-----------------------+--------
LOAD_FAST 0 | coords1
UNPACK_SEQUENCE 2 | y1 x1
ROT_TWO | x1 y1
LOAD_FAST 1 | y2 x2 x1 y1
ROT_THREE | x1 x2 y2 y1
BINARY_ADD | xS y2 y1
ROT_THREE | y1 y2 xS
BINARY_ADD | yS xS
ROT_TWO | xS yS
BUILD_TUPLE 2 | (xS, yS)
RETURN_VALUE |
I'll look into that, thanks
I can finally use goto in python 🥳
true ridge
haha indeed, a real goto
true ridge
probably you drained stack
grave jolt
In [2]: @hax
...: def add_tuples(x, y):
...: LOAD_FAST("x")
...: UNPACK_SEQUENCE(2)
...: ROT_TWO()
...: LOAD_FAST("y")
...: UNPACK_SEQUENCE(2)
...: ROT_THREE()
...: BINARY_ADD()
...: ROT_THREE()
...: BINARY_ADD()
...: ROT_TWO()
...: BUILD_TUPLE(2)
...: RETURN_VALUE()
...:
maybe some benchmarks?
grave jolt
Good idea
In [20]: %timeit add_coords_Akarys((1, 3), (100, 1000))
681 ns ± 42.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [21]: %timeit add_tuples((1, 3), (100, 1000))
157 ns ± 4.49 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
Actually, the tuple-map-zip one is surprisingly fast.
I replaced add with the built-in sum, though :)
flat gazelle
try the zipless one with int.__add__
grave jolt
In [25]: def add_coords_lakmatiol(coords1, coords2):
...: return tuple(map(int.__add__, coords1, coords2))
...:
In [26]: %timeit add_coords_lakmatiol((1, 3), (100, 1000))
531 ns ± 7.62 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
ye, about as expected
grave jolt
now for the readable versions...
flat gazelle
not too great, would probably get a bit faster with (*map(...),), but not that much
flat gazelle
yeah, numpy will not be faster for small things like this I do not think
grave jolt
In [32]: def add_tuples_machadojpf(a,b):return a[0]+b[0],a[1]+b[1]
In [33]: %timeit add_tuples_machadojpf((1, 3), (100, 1000))
266 ns ± 22.8 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [34]: def add_tuples_salt(a, b):
...: x1, y1 = a
...: x2, y2 = b
...: return x1 + x2, y1 + y2
...:
In [35]: %timeit add_tuples_salt((1, 3), (100, 1000))
196 ns ± 9.92 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
lmao, my bytecode version might be the fastest
deft pagoda
what why is the one named after me using indices
grave jolt
In [36]: %timeit add_tuples((1, 3), (100, 1000))
152 ns ± 1.51 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
i'm offended
unkempt rock
hi is there someone who could clear my doubt?
odd ether
In[7]: %timeit np.array((1, 3)) + np.array((100, 1000))
1.71 µs ± 4.82 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
Yeah that won't do
deft pagoda
that's not bad
grave jolt
fixed
deft pagoda
i thought it would be slower
grave jolt
Well, we have different computers
somber halo
just a tip on benchmarks
you want to disable turbo boost or dynamic CPU frequency scaling on your machine when doing benchmarks
odd ether
hi is there someone who could clear my doubt?
@unkempt rock you can check out #❓｜how-to-get-help
grave jolt
good idea
unkempt rock
@odd ether oh thx!
somber halo
that's a very common mistake I see people doing
grave jolt
Oh, I might've done that
deft pagoda
there's a few instances where i've avoided using numpy because i was using 2-tuples and i figured it would be too much overhead
grave jolt
My CPU speeds up from 2.60GHz to 3.20GHz
odd ether
i thought it would be slower
It's a pretty useless comparison though since mine and typing's machines are obviously different
deft pagoda
it's true, but it's still faster than i'd thought
somber halo
and for these type of measurements you want to not have anything else running on your machine
grave jolt
I also didn't disable turbo boost. It says 1200 here, not 2600. I don't even know who to trust anymore!
decorator-factory@df-laptop:~/Python/lazy$ lscpu | grep MHz
CPU MHz: 1197.340
CPU max MHz: 3100,0000
CPU min MHz: 1200,0000
and not using your machine even
deft pagoda
i don't think we need a clean room, a general idea is fine
somber halo
it depends on the order of magnitude of the execution time of your fuction
since we're in the range of micro and nanoseconds... a more clean room setting might be a good idea
but still
with the results here you can start having an idea of the difference in execution time.
odd ether
That's what the loops are for
grave jolt
I tried to disable turbo boost, but the solution from SO doesn't work 👀
somber halo
even if you use thousands of multiple measurements, it would still be important to just leave your machine alone while running these type of benchmarks.
But the the most important low hanging fruit here would be disabling turbo boost anyway
grave jolt
Right, my cpu jumps from 1200 to 2600 hz
true ridge
I isolate a cpu when I'm doing a benchmark, (and disable scheduler to ever schedule there). It is the most reliable way I found
these are great tips: https://pyperf.readthedocs.io/en/latest/system.html
grave jolt
disabled turboboost
https://paste.pythondiscord.com/ifizexegax.py
still similar results
numpy is ~5.4 microseconds for me
grizzled vigil
In general, you can also typically use the best case rather than the average case for most benchmarking purposes. Unless it's something with more significant variance and some degree of randomness.
Especially if you don't need complete precision and just want to get an estimate, or compare performance between two different approaches on the same setup
grave jolt
Right, because the slower results come from other things interfering with the benchmarks.
Right?
grizzled vigil
Yep.
somber halo
possibly yes
grave jolt
Sadly, hax only works on cpython :-(
somber halo
I agree also with that best case scenario
grizzled vigil
At least for 90% of typical cases, using the best case is "good enough".
true ridge
I guess with a few tweaks it should be able to produce micropython bytecode @grave jolt
somber halo
considering the results, I would go with add_tuples_unpack().
It would be much more readable compared to add_tuples().
grave jolt
Yep, bytecode is way too much
pypy3 only supports 3.6, so I'll have to type everything in here...
true ridge
@grave jolt :/
grave jolt
So, I moved the bytecode into a pypy function by hand, and it works
>>>> add_tuples((1,2),(100,1000))
(101, 1002)
for some reason it segfaulted when I tried to timeit it
grave jolt
I'm doing that now
grizzled vigil
@grave jolt
Yep, bytecode is way too much
I'm honestly not ashamed to admit that I have not personally done much of delving into CPython's bytecode, other than fairly trivial disassembly for one-to-one comparison purposes using https://docs.python.org/3/library/dis.html. Don't get me wrong, it's certainly interesting and worthwhile to learn more about, but I would not say it's for everyone by any means.
somber halo
related to this benchmarking discussion -> https://www.welcometothejungle.com/en/articles/btc-performance-python
grave jolt
HAVE_ARGUMENT
This is not really an opcode. It identifies the dividing line between opcodes which don’t take arguments <
HAVE_ARGUMENT and those which do >= HAVE_ARGUMENT.
that's an interesting opcode
is it a reference to the argument clinic?
true ridge
no, it is basically a divider
grave jolt
I understand, but maybe there's a bit of a pun
I want to have an argument
Maybe there is a more beautiful way to express this in Python:
@hax
def add_tuples(x, y):
LOAD_FAST("x")
UNPACK_SEQUENCE(2)
ROT_TWO()
LOAD_FAST("y")
UNPACK_SEQUENCE(2)
ROT_THREE()
BINARY_ADD()
ROT_THREE()
BINARY_ADD()
ROT_TWO()
BUILD_TUPLE(2)
RETURN_VALUE()
def add_tuples(x, y):
stack: *(2)x, rot2, *(2)y, rot3, +, rot3, +, rot2, tuple(2)
@somber halo Thanks for the link, I actually haven't read that article. Pretty much anything with Victor is worth a read though if you want to get into in-depth analysis w/ CPython. He's one of the most active Python core developers (being paid by RHEL to maintain it downstream and upstream in CPython for the remaing time certainly helps) in terms to hours invested and commits. In my interactions with him, I've found that his knowledge of the CPython internals and stdlib is incredibly versatile. I don't know if I could find an area of Python that he isn't at least somewhat experienced and knowledgeable with.
somber halo
Yeah, actually it was through his stuff that I started to dive in deeper on benchmarking in Python
true ridge
He is also the maintainer of the project (pyperf) I sent.
grizzled vigil
Although Yury was my primary mentor that nominated me, I'd consider both Antoine Pitrou and Victor Stinner to have been my secondary mentors, for all of the advice and knowledge they've shared w/ me over the last year.
Victor has probably mentored more core developers than anyone else
grizzled vigil
Yeah, we can move to #ot0-psvm’s-eternal-disapproval, I'll answer it w/ a mention
wide shuttle
I think it's still related to Python and how Python trains it new core devs
could be interesting to others as well
somber halo
either way, I've been working on a benchmarking methodology
based on differential measurements
and I intend to release it as a package this year, along with other functionalities.
true ridge
If you beed a beta tester, let me know @somber halo.
somber halo
thanks 💪. When I release it I'll announce on this server 🙂
grizzled vigil
@wide shuttle
I think it's still related to Python and how Python trains it new core devs
Okay, I'll move the response over here as well then.
hmm, how does that mentorship work?
Basically, the Python core dev mentorship process works by starting as an active contributor over a period of at least a few months. Then, as you start to work w/ existing core developers on multiple projects, they may either ask you if you're interested in becoming a core developer or you can potentially reach out to them. I personally asked Yury Selivanov after working w/ him on a few asyncio-focused projects.
But prior to then, I was contributing to the documentation and other bug fixes in the standard library. I also had joined the recently created Python Triage team (mainly responsible for categorizing, prioritizing, and helping w/ reviewing PRs to CPython) around a month before asking if he would mentor me.
The mentorship itself typically lasts anywhere from around 6 months to a year. It's mainly there for the core developer to provide one-on-one feedback, general advice, answer questions, and help review the PRs of the individual they're mentoring. Typically, the mentorship concludes with a Python core developer nomination, and then the existing core devs have to approve of the candidate with 66%+ yes votes (and SC approval) for them to become a core developer.
For me, I started the mentorship back in September and it mostly concluded with my nomination last month in April (https://discuss.python.org/t/vote-to-promote-kyle-stanley/3839).
somber halo
Really enlightening answer 💪 ! Thanks!
wide shuttle
Thanks, good answer!
slim island
That was very interesting - it's got to be worth pinning right?
wide shuttle
> hmm, how does that mentorship work?
Basically, the Python core dev mentorshi...
ha
grizzled vigil
No problem. :-) I'd also be glad to answer any questions about it or just related to contributing to CPython in general.
wide shuttle
Was already doing that
grizzled vigil
Haha, thanks. That's probably a good idea in case others are curious about it as well :p
fallen slateBOT
@grave jolt :white_check_mark: Your eval job has completed with return code 0.
001 | Built-in functions, exceptions, and other objects.
002 |
003 | Noteworthy: None is the `nil' object; Ellipsis represents `...' in slices.
Are the different quotes intentional?
flat gazelle
haskell does this too
undone hare
Isn't that a sphinx thing?
grave jolt
oh
I was just confused a bit because that style element isn't consistent across this string and comments.
gray mirage
it's just another way of creating a codeblock
undone hare
Just out of curiosity, @grizzled vigil how does that work before being a core dev? I guess you need to be a cpython developer, according to isidentical's role haha, but what does that mean? What can you do in particular with that role?
wide shuttle
That's our name here
It's for people involved with CPython, isidentical made contributions and is a triager on GH
grizzled vigil
@undone hare Technically speaking, there's really no official "cpython developer" role. That just involves being active w/ contributing to CPython in some form, whether it's the standard library, documentation, C-API, regression tests, etc.
Most of us get started by finding an "easy" or "newcomer friendly" issue on bugs.python.org.
And then gradually work on issues of increasing complexity as we become familiar with the workflow (which is decently well documented on devguide.python.org)
peak spoke
How would you place the general complexity of those issues?
grizzled vigil
@peak spoke It varies quite a bit, but some of the "newcomer friendly" issues are very trivial in terms to complexity. At the simplest, they may involve typos/grammar issues in the documentation, updating an example, fixing a docstring, or a very well defined bug with a low priority.
The "newcomer friendly" label was added somewhat recently (within the last year or so), and typically used to indicate issues that are reserved for newer contributors as an introduction of sorts.
The "easy" label has been around for much longer and tends to vary quite a lot in complexity, but skews towards issues that are at least decently well defined.
peak spoke
Does sound like there's a lot to learn by contributing there
undone hare
Thanks that's quite interesting
grizzled vigil
Yeah, definitely. I would say that I've learned a large portion of what I currently know just from working on CPython issues and PR review in my spare time.
somber halo
you did mention here that you're in college right?
grizzled vigil
Yep, currently in the last couple semesters of my B.S. degree
somber halo
impressive
unkempt rock
bullshit degree
No honestly, that's really amazing!
somber halo
I wish I had that insight and initiative back then.
grizzled vigil
Haha @unkempt rock, sometimes it certainly feels that way. But the several of the programming courses did provide a decent amount of introductory hands-on experience.
undone hare
That's pretty cool to know that they don't mind
unkempt rock
Really curious, what did you do differently than others to be such ahead of the curve?
grizzled vigil
Yeah, my background as a college student instead of a software engineer w/ several years of exp (like most of the other core devs) hasn't really been an issue at all. It's mostly just about being willing to put the time and effort forth
unkempt rock
I'd like to try my best to follow that
grizzled vigil
I spent an absurd number of hours on programming challenge websites, mostly Codewars
unkempt rock
haha, wow. Who wouldve thought
grizzled vigil
That gave me a decent foundation in addition to few generic personal projects, mostly involving ETL (extract-transform-load).
like interacting with a rest API, parsing the json, transforming the format into one that could be placed into a standard SQL relational database, building a basic DB controller (mostly w/ MySQL/MariaDB), etc.
I don't have any especially impressive personal projects that I published though, it was mostly just for learning/experimentation purposes.
unkempt rock
I see
yeah that's the same with me right now
honestly just learning / experimenting with projects no one cares about 😂
grizzled vigil
That's a great way to get started though, especially if it largely involves things that employers care about
(in your respective field)
undone hare
Well, you already have a github enough impressive haha
unkempt rock
I should check out aeros's github yeah
grizzled vigil
Yeah, but it certainly didn't start that way before I started contributing to Python
@unkempt rock Here's the link if you're curious: https://github.com/aeros
undone hare
There's a link it their contribution on the poll
unkempt rock
I can agree this is impressive
undone hare
You could have put an uppercase to that poor @python :>
Oh hey, I have more contrib actually haha
grizzled vigil
@undone hare Unfortunately, the @ mention for the repo is case sensitive :p
unkempt rock
what? @'s are supposed to be lowercase >;
grizzled vigil
Also, thanks. But let's try to keep the discussion somewhat focused on Python. I don't mind answering questions about it, but we could easily go down a rabbit hole with GitHub discussions
undone hare
That's the impressive part https://github.com/search?q=involves%3Aaeros+archived%3Afalse+org%3Apython
grizzled vigil
feel free to @ mention me in the off-topic channels
unkempt rock
THE ACTUAL F
peak spoke
You got me intrigued with the cpython issues; so far only made one bug report on the tracker. Is there some sort of guide for how to start things there?
grizzled vigil
@peak spoke Here's a good place to start for contributing to Python: https://devguide.python.org/
undone hare
I tried to run the python test suite once, it took ages :/
grizzled vigil
Did you run it in parallel with -j or single-threaded?
undone hare
Actually, I ran the test module, so I guess it was single threaded
Oh also, can you pin the devguide here please?
peak spoke
Let me know if this is OT for the channel now, but how does development on a large project like that take place locally? For python projects running it manually and tests is just a matter of clicking a few buttons and then trying out the wanted features
grizzled vigil
<@!184351770636582913> Here's a good place to start for contributing to Python: ...
undone hare
Tests suite are really useful on big projects like that, mostly
grizzled vigil
@peak spoke It's actually quite a streamlined process. Local testing usually involves running the set of regression tests specific to the module you're working within.
E.g. if I'm working on asyncio I'd typically use: ./python -m test test_asynco -j4 -v (with a few other args sometimes, depending on what the PR will be affecting)
We also have quite a nice CI setup that runs when you open the PR using Travis, GitHub actions, and Azure Pipelines.
But yeah, a properly configured test suite is essential for large projects
undone hare
3 services?
grizzled vigil
Yep, we like having a lot of redundancy
And then in the post-commit, it's ran in our massive fleet of buildbots. They're composed of a number of different platforms, mostly donated by companies and individuals.
See https://buildbot.python.org/all/#/ if you're curious
We also tend to alternate between required checks depending on if a specific service is having issues, which occurs from time to time.
spiral willow
for any large project, it's test suites, developers are expected to run them against whatever they are working on, the build system will run full testing most of the time
grizzled vigil
Yep, we typically run the full test suite towards the end before opening the PR, and it's fully ran in the CI and buildbots. Along with randomized order of tests and such.
undone hare
So when you commit a change, a build bot will trigger and run the whole test suite?
grizzled vigil
I run only the relevant parts though when I'm actually working on the implementation details. It would be way too time consuming to do that every time I change a small part of the patch as I'm working on something.
spiral willow
for us, developers have to opt their branch into full Azure Devops builds
grizzled vigil
@undone hare Yep, you can actually see the specific buildbots if you look at one of the recent commits:
Click on the red "x" in the top left: https://github.com/python/cpython/commit/1b97b9b0ad9a2ff8eb5c8f2e2e7c2aec1d13a330
@spiral willow Yeah, we have that step done through GitHub as a pre-commit check when the PR branch is opened against CPython; it's mostly automated.
undone hare
Are the buildbots publics?
grizzled vigil
Mix of public and private, IIRC. Depending on what you mean by "public"
The buildbot logs are fully public as far as I'm aware, but the devices themselves are mostly private to the owners.
undone hare
Well I mean, can anyone trigger a build bot?
grizzled vigil
Oh, no. The builds are only triggered on post-commit (which of course can only occur from a core developer) or via a recently added test-with-buildbots label that can be applied by triagers or core devs.
That label is mostly reserved for complex changes that can have different behaviors based on the platform, and we want to test more than just the platforms in the CI (which includes Ubuntu, MacOS, and Windows 10)
undone hare
Well that make sense
grizzled vigil
or most involved C-API changes, since we have dedicated buildbots for checking reference leaks
undone hare
If you have some spare resources on a server, can you host a buildbot for you to do some extended testing on your commits?
Well, I guess it wouldn't be that hard to setup a test runner, even if it is not a buildbot
grave jolt
Do the bots simply use valgrind or something similar to check for reference leaks, or is it more complicated?
grizzled vigil
@grave jolt It's a bit more involved. It's honestly not something that I've looked closely into, but here's what we use if you're curious: https://github.com/python/cpython/blob/master/Lib/test/libregrtest/refleak.py
grave jolt
Oh, so it also counts the Python-references, not just C-references.
grizzled vigil
@undone hare
If you have some spare resources on a server, can you host a buildbot for you to do some extended testing on your commits?
Yeah, technically you could do that, but that's not something I've ever had a strong use for. Running the regression tests locally and using the existing CI has served my purposes well enough
Although I very much doubt that I have the most efficient workflow possible, it works well enough for me :-)
gray mirage
Don't worry, everyone has someone that thinks their workflow sucks :>
grave jolt
I guess it's appropriate for me to post this here instead, since that's on the discussion that took place in here (#internals-and-peps message)
I posted an explanation of my bytecode tuple adder on the Python subreddit: https://www.reddit.com/r/Python/comments/gkzcoy/adding_tuples_a_bit_faster_by_writing_raw_bytecode/
I'd be grateful if someone could read it and suggest some improvements (like, is the explanation clear enough?)
true ridge
Well, I guess it wouldn't be that hard to setup a test runner, even if it is not a buildbot
If you can guarantee that your server won't randomly disconnected / become unavailable and have at least spare 1-2 threads and some ram you can definitely host a buildbot. The important thing is if we already have that same exact operating system + cpu arch + compiler combination there wont be any need for it.
I'm hosting a worker on a private compile farm (off gcc / free software developers) and I didnt had to touch it after I set-up the bot itself.
spice coral
Hmm is the newer version of pickle in Python 3.8.3 considered secure?
quick orbit
peak spoke
I don't believe any pickling is considered secure
unkempt rock
space or tab?
true ridge
not the correct channel @unkempt rock
unkempt rock
ok which is?
true ridge
probaby one of the of-topic channels would be more suitable
shy vine
I’m sure that would be real productive
molten onyx
I have so many different python.org accounts :s
The BPO system is really not fun to use at all
true ridge
We'll migrate to the github, probably next year the migration would be completed
PSF is actually looking for a project manager for this transition (and some volunteers): http://pyfound.blogspot.com/2020/05/pythons-migration-to-github-request-for.html
raven pike
that's good news i guess
severe lichen
Did you hear the story of lib author blocked by Github because of some comment? The whole rep went down. Imagine the same with whole Python rep.
true ridge
There are plans about backuping the whole repo state (issues / pull requests / comments everything) to a psf managed server in case of anything happens.
snow kettle
did his channel get created recently?
fossil pumice
yes.
snow kettle
i feel like i didn't see it as of 5 minutes ago
but there are messages from hours ago
i guess this is on topic for this channel: i saw that there's a .toreadonly() on memory views as of 3.8. It's what i hope to move a project toward using eventually after a lot of refactoring and discussion with the current maintainer, and it got me thinking:
are there any plans for something like a ChainIterator or ChainMap but for memoryviews?
it'd be useful for some data serialization tasks as well as memory mapped IO emulation
peak coyote
Sup guys. I've decided that I was gonna start reading the python docs after the techlead recommended it. I've learning quite a bit of ways to "hack" the python syntax so to speak in the docs. But I'm wondering what's the difference and use case for using a parenthesis to span a string multiple lines or triple quotes?
flat gazelle
parentheses do not leave newlines afaik
peak spoke
Triple quotes create a different literal (long strings) and take everything that's between those quotes including whitespace
Parentheses just use the implicit line continuation and joining of string literals
peak coyote
>>> text = ('Put several strings within parentheses '
'to have them joined together.') + '''HI
my name is bob'''
>>> text
'Put several strings within parentheses to have them joined together.HI\nmy name is bob'```Yeah. That's really interesting
What's the best practice?
peak spoke
I usually use the line continuation with parentheses because with long string you usually need to process the long string in indented code
unkempt rock
why all of a sudden my python interpreter is all of a sudden invalid XD
peak spoke
But if you've got a lot of newlines that'd be worth it probably and improved readability of general text
peak coyote
Do you any of you know TechLead?
slim island
This isn't the right channel really. Try one of the three offtopic channels
grave jolt
I wouldn't recommend following his advice literally, since you can't really know which part is serious and which is sarcasm.
somber halo
oh man, Tech Lead...
yeah, he's a clickbaiter youtuber
(ok, stopping discussing this here.)
grave jolt
I'm having a bit of an n-lemma. I've been playing around with typing things in Python for a while and looked at some other languages.
It seems to me that static typing often makes things much easier. It eliminates type error at compile time, enables function overloading and generic programming.
I also like the idea of structural typing, where a type is a set of values and not a label or a node in a hierarchy. To me it looks like a static alternative to duck typing.
Python can be, at least in part, statically analyzed, and you can do crazy stuff with static analysis (https://pypi.org/project/dependent-types/). But maybe I should just accept that Python is dynamically typed and not try to make it something-other-than-python.
gentle lodge
<@&267629731250176001> ^^
uncut sage
@grave jolt I add type hints for all my projects now, mainly for function signatures, but only after they reach a certain level of complexity or ambition. Most of the time the types are self-evident
fossil pumice
!ban 528161097371353098 spamming some youtube video across multiple channels
fallen slateBOT
:incoming_envelope: :ok_hand: applied ban to @ripe peak permanently.
grave jolt
Well, it might be a good idea to typehint public-facing functions because you don't have immediate access to their source code.
somber halo
I type hint everything
grave jolt
Maybe I should try out dynamic approaches more. After all, all the nice features like type validation, dependent types, overloading and pattern matching are still available at runtime. It's just that some of these features are destined to look ugly in current version of Python.
somber halo
and use mypy for type analysis
gentle lodge
I really miss static typing sometimes
somber halo
I mean, not everything.
gentle lodge
my dream is optional static typing. Type hints + mypy are a nice step in that direction
severe lichen
I haven't found a detriment to typing other than some stuff are too complex to typehint properly. And if the typing fails, you can simply add ignore typecheck ignore comment.
severe lichen
However, typing needs to step up the game with better syntax for optional arguments for callables and better support for duck typing and monkeypatching.
It should be easy to say "this argument needs to have .x and .y attributes".
I don't like the divide between generic collections and in-built ones (list vs List[str]), but I've heard it's fixed.
Side note - I'm currently thinking if it's worth it to properly type units.
As an example - both time and distance are floats. You can add time + time, but can't time + distance. However, you can divide time/distance, giving you the speed. I wonder if i can static type that with minimal amout of boilerplate.
grave jolt
Well, look at https://pypi.org/project/dependent-types/
I haven't tried it out yet, but it seems awesome
severe lichen
Typing, if digging deeper, is a damn arcane art. Covariance and contravariance, for example. I'm yet to grok that stuff.
true ridge
I don't like the divide between generic collections and in-built ones (list vs List[str]), but I've heard it's fixed.
Happy to say, yep it is improved in 3.9. https://docs.python.org/3.9/whatsnew/3.9.html#pep-585-builtin-generic-types
severe lichen
BTW, i've heard about typing-extensions. I assume it's 3d-party lib that extends on typing lib, but both projects are connected?
Are there any interesting features you love to use and wish they were in stdlib?
grave jolt
Well, look at https://pypi.org/project/dependent-types/
I used my own lib which supports elements of structural typing, but it's totally dynamic, so it doesn't ensure any correctness.
With its help, I was able to do this:
@overload()
def parse(tokens: str) -> str:
return parse(tuple(tokens.split(" ")))
@overload()
def parse(tokens: tuple) -> str:
return case([
(StartsWith(("c",)), lambda: parse(tokens[1:]).capitalize() ),
(StartsWith(("u",)), lambda: parse(tokens[1:]).upper() ),
(("+", Case, str), lambda: plural(tokens[2], tokens[1]) ),
(("+", str), lambda: plural(tokens[1], DefaultCase) ),
((Case, str), lambda: singular(tokens[1], tokens[0]) ),
((str,), lambda: singular(tokens[0], DefaultCase) ),
])(tokens)
It's a parser for an internationalization system that supports grammatical case and number. Examples:
+ acc male = accusative plural, male (men, мужчин, Männer)
c cat = capitalized, nominative singular, cat (Cat, Кот, Katze)
at least to me it seems beautiful semantically, but absolutely, unbearably ugly syntactically.
The structural part here is that you can supply StartsWith(("c",)) as a type hint, like tuple, and it will be treated as expected in type validation and multiple dispatch.
severe lichen
Well, the problem with natural languages is that there's damn too many exceptions to every rule
Physics/math at least tries not to do that
grave jolt
I agree, and standard localization tools often ignore grammatical case, which occurs in many languages, which makes you twist sentences in an awkward way.
I'm not trying to handle every corner case in every language. It's just an example problem.
true ridge
BTW, i've heard about typing-extensions. I assume it's 3d-party lib that extends on typing lib, but both projects are connected?
@severe lichen IIRC it was a backport of new typing features, not sure though.
raven ridge
I think it was for prototyping new typing features before they make it into typing
I know stuff has graduated from typing-extensions into typing, but I don't know whether typing-extensions still exists for still-experimental stuff.
true ridge
oh looks like it does both
The typing_extensions module contains both backports of these changes as well as experimental types that will eventually be added to the typing module, such as Protocol or TypedDict.
we got both Protocol and TypedDict, I wonder what is left in there
deft pagoda
this reminds me of general property contracts
grave jolt
I have a suspicion I won't be able to build ocaml compiler from source.
undone hare
I've seen way worst
grave jolt
Downloaded a release as .tar.gz from github, took a few seconds. Weird.
undone hare
Oh well, there is probably a huge file somewhere in the history
flat gazelle
@gentle lodge optional static typing is gradual typing. Certain lisp dialects, raku, and probably some other languages use it.
proven wing
can anyone explain me about async func....I have searched for like so many hours on net but can't seem to find anything helpful...Maybe i am still not experienced enough to know about async. It would be a very big help to me, in this free time which i get nowadays.
unkempt rock
@proven wing https://docs.python.org/3/library/asyncio-task.html
proven wing
hmm i will see it right now. and will discuss what i got
unkempt rock
they are basically a task scheduling feature in python
undone hare
I wonder why is test_socket taking so long on a source built interpreter, the whole suite took 4.47 mins, and I'm 7.05 mins into the socket tests now
worldly venture
Huh that's an interesting one in #mailing-lists
gray mirage
I would be against that
undone hare
I mean, unicode are known to not work properly on some platform, and some distros doesn't render them by default
gray mirage
yeah there are a lot of situations where unicode like that isn't appropriate
and if you want to render an arrow, well, we have ligatures for that
unkempt rock
also most keyboards don't have that character
gray mirage
Ligatures OP
oblique crystal
I think this is basically the reality
sacred tinsel
I still don't like ligatures because it makes me feel like I don't see the actual code
style guides would just end up forbidding it anyway
that's probably true lol
gray mirage
sure, you might not like ligatures, but it's hard to argue that it doesn't fill that niche for people that might like the unicode char
sacred tinsel
no, I wouldn't dare argue that
filling that niche is exactly what ligatures do well
peak spoke
!pep 615
fallen slateBOT
peak spoke
this looks like a nice addition
worldly venture
oh please
peak spoke
Didn't manage to get the reference implementation to work on win but I'm sure people who have to work with TZs will appreciate something in the stdlib, timezone support was somewhat minimal so far
undone hare
So far the hardest in trying to contribute to cpython is finding an issue to work on haha
true ridge
I wonder why is test_socket taking so long on a source built interpreter, the whole suite took 4.47 mins, and I'm 7.05 mins into the socket tests now
An interesting fact is that test_socket is the longest file in the stdlib
undone hare
For some reason python -m test test_socket complete in less than a minute
true ridge
It took 26 seconds for me (on a very old computer with debug build)
undone hare
23s for me
Do you have any advice @true ridge on searching which issue to tackle? For now browsing through recent easy issues didn't worked too well, in most cases a solution was already developped
true ridge
I can suggest you to create a search filter in the tracker that sorts all issues with the message count. In general the higher message count means the problem would be hard to solve
you can just go through this list and try to find an issue that looks interesting
a detail would be that, if the issue opened by a core dev/triager, the rate of acceptance is higher then normal ones
and you'll get fast reviews
undone hare
Ah that's pretty cool, thanks!
boreal umbra
So any and all require what be passed to it be a single iterable rather than supporting *args
but there are no other arguments
can anyone think of a reason why *args shouldn't be supported?
sacred tinsel
hmm, good question, I think if you have conditions like that'd they'd be commonly checked using and or or, right?
true ridge
All of the use cases I can think for any/all can be done with the current version. I dont see a need there for *args. Other than that, there is a specific optimization in builtin function/classes if they only take 1 argument. So probably that is the second reason it is not supported
sacred tinsel
any and all are good for supporing variable amounts of conditions
peak spoke
It would probably require creating some sort of iterable anyway since it wouldn't be python *args
and the current version ought to be a simpler implementation
boreal umbra
The reason I ask is that any and all apparently support short-circuiting, but if you have a number of functions you want to use to determine True or False, you'd need to use a generator or something to make sure that only the bare minimum get executed.
Though I guess that using *args wouldn't necessarily solve that.
I guess it could also be argued that if you know exactly how many conditions you're checking, you should just use chained ands and ors
sacred tinsel
those will short-circuit too
hollow geode
@boreal umbra The reason is simple actually: all([]) would be ambiguous if it allows *args.
sturdy timber
@deft portal This isn't really the right channel for that sort of question, see #❓｜how-to-get-help for how to use the help system.
This channel is more for discussion of the python language and it's features than specific help with code
whole obsidian
how long will python be around for
flat gazelle
Far longer than any of us think
narrow kettle
Somewhere between 10 mins and heat death of the universe
whole obsidian
why not 9 mins 59s @narrow kettle
lost nexus
is dict subscription faster than .get?
zenith topaz
Good question, I assumed it would be since .get does more than just subscription.
peak spoke
because get is a normal function call it has the overhead of one
and the get needs to go through the descriptor protocol to get bound
lost nexus
i see
peak spoke
https://paste.fuelrats.com/pewikajagi.py for some timing
deft portal
@sturdy timber sorry didnt mean to
lost nexus
would you prefer using collections.defaultdict(lambda x: None) to return None with subscriptions or simply use get with a normal dict?
in terms of efficiency
flat gazelle
Definitely just use get
lost nexus
reasoning?
flat gazelle
Also, do not optimize that much except for academic curiosity.
lost nexus
it's for curiosity, dw.
flat gazelle
You are not avoiding function call overhead, because you are still using a lambda.
What may be worth exploring is
defaultdict(type(None))
NoneType has a 0arg constructor that returns the Singleton instance of None
lost nexus
I'm gonna guess this is much faster than a function call
flat gazelle
Probably not much faster, but maybe somewhat faster
flat gazelle
In [1]: k = type(None)
In [2]: l = lambda: None
In [3]: %timeit k()
111 ns ± 5.09 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [4]: %timeit l()
190 ns ± 34.2 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
```at this speed, the measurement is not quite accurate, but that is a greater difference that I thoughtDamn
oblique crystal
what are general opinion on multiple returns for functions? At one poin I remeber I tended to use it all over the place, at other point I tried to always have one single return statement. And I kinda oscilate between the two
flat gazelle
If the function is easy to follow, multiple returns are fine
peak spoke
If there are too many it can be an indicator that your function is too complex, but never considered more than one return to be something wrong
flat gazelle
I personally prefer it over a result variable, as you then also have to see if the result variable gets changed anywhere after the assignment
But yes, no amount of general advice on code style will a make a too complex function simple, aside from splitting it up
oblique crystal
OK, yeah that was my take early on, that too many means to complex design
but it's good thing to know that there is nothing inherently wrong with multiple returns
grave jolt
If a single return point makes the function harder to understand, use multiple returns. But that's kinda vague.
I think multiple returns are useful in guard clauses
flat gazelle
Do note that you generally cannot "fix" a function by making it have a single return
peak spoke
huh looks like unittest still doesn't support discovering tests in implicit namespace packages
oblique crystal
Do note that you generally cannot "fix" a function by making it have a single return
@flat gazelle what do you mean by "fix"?
flat gazelle
Make it more readable/understandable. If a function is hard to follow with many returns, it will be hard to follow with one
oblique crystal
oh yeah, for sure
my main question was whether there is any widely accepted community stance like don't usre more than X returns
grave jolt
I can only say that the transition from the first snippet to the second one is not an improvement
def f():
if condition:
happy_return_path
else:
another_happy_return_path
def f():
if condition:
return happy_return_path
another_happy_return_path
thoughts about black?
oblique crystal
imo sometimes it tend to make code uglier. But if it is big project with many people invlolved it probably would help a lot to achieve a consistent style
lost nexus
I've been using it but I revert a few things often. Black uses a very short column limit, not exactly sure what, but it'll tend to split my medium-sized parameters/conditions into 3 lines and make it hard to read.
It's supposedly 100% pep compliant.
grave jolt
it says it's 88
gray mirage
79 is PEP8's recommendation but nobody actually follows that
black says 88, some projects use 100 or 120
the general consensus is that you should have some limit though
oblique crystal
oh ok, 88 for black
lost nexus
I tend to stick to 120
gray mirage
20-40 chars can make a world of difference depending on how nested your code is
slim island
120 is Pycharm's default
grave jolt
I think there should also be some sort of sublimit. Because 10 80-characters lines in a row are unreadable, at least to me. Maybe something like 50 or 60 as a target.
gray mirage
Some people use the concept of a soft margin and hard margin
like, aim to be below the soft margin, but it's OK to go over as long as you're under the hard margin and it's more readable this way
zenith topaz
I believe black does this
gray mirage
I think it does, yeah
lost nexus
overall, would you suggest using black as a formatter?
zenith topaz
I use it for all my personal stuff, so yeah.
somber halo
In the ol' typical C style, a single return is encouraged
personally, I find the use of early returns to make sense
it makes the code more readable typically
flat gazelle
Is it? I see if(!cond)return; all the time in C
With more whitespace and such, naturally
grave jolt
It's not that bad as a guard clause
true ridge
Because lack of exception handling, if (!cond) return NULL; or goto fail; etc is pretty common in C code
somber halo
Hmm, I was taking into account a recommendation from IEC 61508
raven ridge
In the ol' typical C style, a single return is encouraged
That mostly for cases where the function may have allocated some resources that it needs to deallocate. You can avoid the need to have multiple places where cleanup of resources happens if you have a single return.
flat gazelle
With context managers, this is not a problem in python though.
raven ridge
it's garbage collection that makes you not need to worry about leaking resources, not context managers. context managers just make the cleanup happen at a deterministic time.
flat gazelle
It is kind of both. Some things need cleanup beyond what the GC does
Or just in general ensuring your state is corrected after something
raven ridge
those classes are buggy unless their __del__ prevents the resources from being leaked when the object is garbage collected.
it's probably true that there are things out there that only properly clean up after themselves in __exit__ and not in __del__, but those things are buggy.
magic python
is the import system in python generally considered as confusing or not? it seems very unintuitive , just wondering if others have found that or not
zenith topaz
For packages it takes a bit getting used to.
magic python
i just want scripts to be able to use each other really
or wanted, but yeah - it seems very unintuitive. I haven't used any other language for this kind of thing though
so, I don't know if the design is crap, or if this is just how it is
raven ridge
when it comes to packages, you're currently hitting the one thing that's kinda weird: you can run a module inside a package as though it's a script, in which case Python doesn't know that it's part of a package anymore (which is why relative imports don't work, for instance). You have to run it differently if you want Python to know that it's part of a package.
if you put your code in package_name/__main__.py and run it as python3 -m package_name, things should work the way you want.
@magic python ^
magic python
@raven ridge ok fair... what i have is a bunch of scripts and they're currently being developed, so i need to import one into others and stuff
package_name/main.py
i don't think this really works for me, i have some A.py which has general functions, and want to use them in other scripts
raven ridge
@magic python it sounds like you're not using packages at all, then - and if that's the case you can always just use import foo instead of from . import foo - the directory that your main script is in is already automatically added to the module search path, so other things in the same directory can always be imported as new top level modules.
magic python
@raven ridge hrm, but i'm not calling it from the dir that the scripts are in, they're nested in a couple of directories.
god i've confused myself this evening
raven ridge
it sounds like you really should be splitting the reusable code into separate libraries that can be installed and used by your other scripts, at this point.
magic python
i'm kind of trying - this is half way house, but i need to use things and work with them as i go
splitting the reusable code
that's what i'm trying to import:')
raven ridge
you can probably get a bit farther with your current approach by manipulating the module search path using PYTHONPATH or sys.path, but you're doing things in a way that's now how they're meant to work (scripts aren't meant to import other scripts from random directories to get access to library functions) - so you're making things more complex than they need be.
magic python
so you're making things more complex than they need be.
i don't know any better really
it's easier to make a package?
raven ridge
you can make your own libraries - and even make them pip install'able, just like regular libraries. It's not too tough, and if you're working with virtualenv already it's only a small step up. You just need to add a setup.py to your library's directory, and pip install /path/to/library/dir will install it into whatever virtualenv is activated.
and from that point on you'll be able to import it by whatever name you've assigned to it, whenever that virtualenv is activated.
magic python
hrm, i'm not sure if what i have is too much of a clusterfuck or not
i have a project with this kind of developing... and another project copies this project's code over with a cp -r bash script
raven ridge
I mean, it's not going to become less of a clusterfuck without you learning the proper way to make libraries and switching to do that.
magic python
no lol, that is true
is there a tutorial to convert a project of scripts to a package?
well, just a dir with a few scripts
raven ridge
a package is just a directory containing modules, optionally with an __init__.py and/or a __main__.py
magic python
well i have a dir with scripts, so i have a package?
raven ridge
yep. the part that you're missing is that package is supposed to get installed.
so, the next lift is having a setup.py to describe the package so that something like pip knows how to install it.
magic python
seems no, because i get all these errors and stuff about relative imports beyond top level package and stuff.
yep. the part that you're missing is that package is supposed to get installed.
hrm ok - is it bad design to have scripts call each other in a package?
raven ridge
no, it's fine. that's what relative imports are for. but the top level package needs to be found, which means it needs to be installed somewhere where the interpreter will look for it. By default that's either the interpreter's site-packages directory, the interpreter-specific site-packages directory inside your home directory, or the directory that the script you're running was started from.
magic python
i've just been doing
raven ridge
so, your two paths forward are the kinda hacky one (adding more directories to that search path, so that the top level package can be found elsewhere) or the proper one (installing the package into one of those directories)
magic python
cp -r ../where_scripts_are_developed/foo/bar/baz ./tools/baz
at the moment , to bring the scripts in to the project where analysis is done
( that command is run from the root of the project with analysis )
is there a package to auto-convert a dir into a package or something?
raven ridge
a dir with an __init__.py file, even an empty one, is a package.
magic python
right - so as soon as there's an init file, it's a package, but i can't use pip until it has setup.py
also - can i use pip to install from another directory? So, replacing the cp -r that I did above?
raven ridge
yes. pip install /path/to/package works.
by default it'll try to install it into the system's site-packages directory, which would be a bad idea and not what you want. if you have an activated virtualenv it will instead install into that virtualenv, which is much saner.
magic python
yeah - i use virtual envs for everything, so that's ok
Installing collected packages: check
Running setup.py develop for check
Successfully installed check
i think it installed lol
hrmmm, it seems that the example is kind of working 🙂
i just have to have the from . import two in the __init__.py which is within check/check, and the package is called check
check
├── README.txt
├── __init__.py
├── __pycache__
│ └── __init__.cpython-37.pyc
├── check
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── __init__.cpython-37.pyc
│ │ ├── one.cpython-37.pyc
│ │ └── two.cpython-37.pyc
│ ├── one.py
│ └── two.py
├── check.egg-info
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ ├── dependency_links.txt
│ └── top_level.txt
└── setup.py
so i can do pip install -e check, then run ipython and do import check.check as c, and i'll be able to do c.two from there, which seems to be the gist i guess
thanks @raven ridge 👍
raven ridge
i'm not sure where your extra check is coming from - you should be able to just do from check import two
feel free to take this over to one of the help-available channels and ping me if you'd like to keep digging.
magic python
i nested check in check 😅 but yeah - i'll move to a help channel for anymore - didn't intend to derail
raven ridge
ah, I see it - you've got an __init__.py in the directory that the setup.py is in - either you shouldn't have that, or you shouldn't have the check subdirectory, one or the other.
raven ridge
if you don't have the __init__.py in the directory with setup.py, you'll need to give packages=['check'] in setup.py (presumably for now you've got packages=['.']
magic python
I had both ha
raven ridge
yeah - remove the __init__.py in the directory the setup.py is in, update setup.py to do packages=['check'], and reinstall it. you should be golden, and from check import two should work.
magic python
it seems that it's working 😄
raven ridge
more interestingly, this holds even with word[:10] + word[10:] - that evaluates to "Python" + "" which gives the same result.
wide shuttle
Hello! This channel is for discussing the Python language itself. We have several help channels (see #❓｜how-to-get-help) and for shorter questions and/or more informal conversations about Python we have #python-discussion. This channel is mainly meant for discussing things like the future of Python, the implementation of Python, and more abstract conversation about Python.
raven ridge
How aggressive is the culture here for moving things when a question starts on-topic and wanders off-topic?
wide shuttle
I don't think our culture is aggressive at all, but this is the one channel where we like to keep things on-topic
We have a fairly relaxed channel in #python-discussion and a maximum of 32 help channel dynamically generated
I think it's fine to have one channel where we keep things stictly on topic
That way every conversation has its place
raven ridge
I didn't mean any judgment in "aggressive" - I was more asking how firmly I should push people into help channels rather than answering their questions where they ask them.
wide shuttle
We are fairly relaxed in #python-discussion, but we really want a channel where more advanced Python topics are discussed. That was the idea of the original python-discussion channel we had (which combined this one and py-general in traffic), but it was impossible, as those more advanced conversations move slower. It meant that those kind of conversations were impossible on our server entirely
north root
it's mainly this channel where we're super strictly on-topic. there's a reason in the pins.
raven ridge
Fair enough. I'm new here, and trying to be helpful, but also to learn the culture. 🙂
north root
there was discussion on lambdas in python a few days ago
it got really heated in here
rich wharf
it's not realistic for python to add them even if I'd like it to
so for me, when I need to use some decent logic in lambdas i usually resort to using my lib lambdatools
but it's not perfect
wide shuttle
Personally, I think our current lambdas are fine for what they are. The philosophy is that if your function gets complex, you should make a function. That's what a lambda already does under the hood, it's more or less makefunction in Python, just with worse readability and less debugging info.
rich wharf
a lot of times I need the functions inline though
if I need a single function as an argument, that's fine
but for not-so-complex functions that make use of some logic, that's not good enough for me
wide shuttle
For me, that depends on what that function is. I may want to provide a function as an argument inline, but if it gets complex, I value the readability of a regular function def and then provide it as an argument.
rich wharf
it's usually for simple uses like small iteration and conditions
it usually takes only like 3-5 lines
but it doesn't call for a whole function
raven ridge
what's wrong with a 5 line function?